Propane heat is reasonably price and as said, more stable than #2 heating oil in terms of cost. The units obviously operate on much the same principle as Natural Gas heaters and are quite efficient.
Not sure how you feel about stoves, but you can always offset your heating costs down the road, if they're high, by installing a modern clean burning wood stove or fireplace system. The modern fireplace system can be ducted much like a furnace, but have clean burning technology and big old glass windows. Of course you know that's kinda my thing, so of course I"m advocating them
All that said though, propane is a nice home heating fuel. They come and fill the tank once a month and otherwise you just pay the bill. Plus the tank is outside, which is nice. Having a oil tank in the basement sucks.