I'm a fan of both drag and autocross, and I can tell you that Autocross is tougher than drag.
But Drift owns all.
Well, I say that, but I haven't done Rally. My friend has a rally car, Audi 5000S, but I've never been with him doing it. I could say that is the baddest of the bad ass sports on wheels.