You would staple the poly fill down, sorta lining the inside of the box. I don't think mesh would be a good idea, you might get some kind of noise from air pushing through the mesh.
a change of say 10 percent (one pound poly fill) wont affect the sound really but making the box larger which is what poly fill would do would make your tuning on your port lower than it was originally