Wow, I'm almost in the same exact position - looking for a CRX for no more than $2k. I've found a few, and one was sold litterally 2 hours before I was going to buy it. They can be found, but dilligence is key - keep looking and be willing to wait a while to find the deals.
I would reccomend Ebay to you. Most cars are pretty spendy, but there have been a few nice deals, but then location becomes an issue. Lucky for you, there are a lot of cars out east. I'm in MN, and finding anything close to home is difficult. Your case might be better. But again, be willing to put in the time and effort to find them. It might sound liek a gamble, and it can be, but Ebay sellers are legally bound, and most sellers are decent. I have a friend that bought a '96 Prelude on there, along with a JDM H22 and all of your standard bolt-on mods with great success. His girlfriend also got a '97 Civic EX that likewise is (was, till she was rear-ended last week) in good shape. I'm a believer. Give it a shot.
I also reccomend going to local you-pull-it yards (assuming they're legal in NY?) and seeing what they have out there for parts. You'll find a lot of deals for cars that aren't running optimally, and knowing where you can find fast and cheap replacement parts can save you money, although you may have to initially buy a car thats in rough shape. Finding parts for a D series should not be hard, just more time and effort needed since you're on a budget.
I wish you luck! I've been on this boat for 5 months and the seas are rough. But it can be done.
Keith
