I have a '97 80 & recently acquired a '04 100. I love both and made sure to wait a couple months now to post so that I wasn't posting during my honeymoon stage with the 100. I would recommend the 100 for what you would would be doing.
The 100 is much more refined, better braking, significantly more drivable power, better fuel economy, the seats are more comfortable, the rear seats recline, rear air-conditioning and larger inside. 17 mpg on trips for me.
Now, the 80 is better at off-roading in the off-camber twisty stuff. This is because of the solid axles allowing more wheel travel. 12 mpg on trips for me.
The 4Runner is a great vehicle but is not an equal with the 100 in terms of off-roading and durability. They may share the same engine and transmission for certain years, however that similarity ends with those parts. There are also other items to keep in mind that are critical for longevity and durability (U joints, drive axles, hubs, rotors, calipers, cooling system and the frame). The 100 has more interior cargo space. The 4runner is easier to find for purchase and the initial acquisition is at a lower cost.
The truth is you would likely be happy with either one, my recommendation would be to get the Toyota Land Cruiser, but I'm biased.