Alright, I just got my Honda Tuning and it actually does have an article about an Element that was swapped with a 3.5 RL motor, converted to RWD, turbo'd, and is now being used for drift racing, so it is possible. Keep in mind though that this is all being done by a professional racing team, and they said it took about 500 hours of work to fab everything up for the motor swap and rear-wheel drive conversion.
So unless you're a master fabricator and mechanic, this is still going to cost you a buttload of money, probably upwards of 10 k once all is said and done.