I'm reporting back with a few more places to buy vinyl in NYC.
First, keep in mind that MANY used music stores in NYC (and there are numerous used music stores in NYC) also carry a selection of used vinyl ... it may not be a full on stonker collection, but many will have a few hundred++ LPs.
There's a store called Academy LPS, they have 3 locations in NYC. The store on 77 E. 10 St., NY, NY 10003 [212-780-9166] is pretty much EXCLUSIVELY used vinyl, every genre except classical. They have 2 other stores (18th St, 6th St), though I'm not certain of the media carried in those stores.
Another store Bleeker St. Records (different from Bleeker Bobs) 239 Bleeker St., NY, NY 10014 [212-255-7899] also sells vinyl (including 45s).
The Virgin MegaStore on Union Square has a section of new Vinyl.
St. Marks Street in the East Village has a bunch of used music stores some of which carry Vinyl.
I'll add to the list as I come across additional resources.