Hi Brad, Welcome to ORE!!
I owned a 3.5lt manual pajero for a while and it was great fun! pretty reliable too. I was using it to get to work and back and the odd trip over over the gumfields or out to Pouto and south head. put some 33's on it with out any problems and it would go most of the places I was game enough to have a go at.
If you have a reasonable sized wallet I would go for the Pajero.
I thought my wallet was a resonable size...but after stopping at the petrol station every other day for a couple of years, and then doing the clucth, cam belt etc etc and the last straw was a scare with a possible cracked head - I was seeing dollar signs and threw the towel in and got a Suzuki Samurai.
Not sure about the Vitaras, but the samurai is friggin awesome to work on! There is room to get your hands pretty much any place you like and you can pull the gearbox with a couple of spanners and dont even need a jack!
Bottom line: If I had the money to throw at it I would definitely go for the pajero again. Pretty comfortable on the road, fairly capable off road, heaps of torque/power & a strong drive chain. Avg petrol about 15l/100km on road.
The suzuki is brutal both on and off road - frequently come home with bruises! but its cheap to run, parts are crazy cheap, and soo easy to work on! Just needs more power...