im going to make a new box, ported with a port tube. at first i was going to go with a square box, but to save space i thought about going with the second picture. will this sound any worse/better than the square box??
Enclosure Shape While it is always a pretty good idea to stay away from perfect cubes, they don't necessarily have to be avoided like The Plague. Due to the very small dimensions and generally low cutoff frequencies used in most mobile subwoofer systems, there is little chance of generating standing waves in the enclosure (standing waves can cause nasty response fluctuations). For a standing wave to exist, the distance between parallel boundaries must be 1/2 the wavelength of the frequency at which the standing wave exists. Considering that sub-bass waves vary from 56.4 feet (20 Hz) to 11.28 feet (100 Hz), the generation of a standing wave is going to be virtually impossible.
Any standing waves that might be generated by upper order harmonics (caused by distortion) in the enclosure can be readily absorbed with the addition of damping material such as polyfill (available at your local cloth store--it is used to stuff pillows and quilts) or they can be broken up with strategically placed bracing within the enclosure.
In short, don't worry too much about shape. Make the box to fit the space you can allot to the enclosure and forget about it--there are more important things to worry about... like bracing.