The real problem with that era of 4-runner was the terribly inefficient automatic transmission in conjunction with the 3.0.
It sounds like you like the truck, and it can be a capable rig. I would keep the mods reasonable (dear lord, don't go bigger than 33" tires), and keep your options open to upgrade to a new rig at a later date.
Swapping in a 3.4 is going to cost you more than swapping into a 3rd gen 4runner, as well as negating a resale value.
This made sense a decade ago, when a third gen was still 20-25k, but you can get them now for 5k, which is cheaper than keeping your rig and swapping in a new engine.