Hey Nova, yeah, personal preference. Definitely the most limiting factor is the guy behind the camera. The difference between the D40 and D40x is 6mp and 10mp. Not that Megapixels matter that much.
The camera you saw me using last Saturday was a Nikon D50. Personally I prefer the D50 over the D70/D70s (which I also owned before) because of size. The D40 also has a nice size to comparable to the Rebel XT and XTi, but personally I think those are a bit too small.
If I had the money, I would go D80 all the way.
