Engine changes alone would be reason enough for me to buy a "new" Hilux. And as you point out, the Tacoma's "generational change" wasn't anymore extensive by comparison...and some of those changes, like the engine, were for the worse.
I actually have driven both, in long duration, trips (with a good mixture between offroad and onroad)....the Hilux rides and handles better. More comfortable seats, better low-end torque, bigger cabin space, and a suspension that rides well (especially with a load).
The Tacoma still drives like a truck, no doubt about it. But its rear end sags way too quickly (limited payload) and the cabin & seating position just aren't that comfortable for long duration trips, at least for normal-sized people. The whole argument that the Hilux is too "spartan & truck-like" for the North American market is a bunch of nonsense. It's ride & handling are perfectly in line with how the Tacoma and Tundra handle and the NVH is, if anything, better since the engine doesn't need to be stressed out to get decent grunt.
Tacoma's are popular (for the time being) because they are affordable, reliable and because they cater towards the the truck-lite crowd...people who want a truck but don't want a full-sized monstrosity. The reality is that the payload and mediocre engine make the Tacoma a compromise between daily driving and some light overlanding or truck-like duties (towing, hauling)....It really could be a lot more competitive if Toyota were to upgrade its capabilities to be more robust....which is what the competitors (Gladiator, Ranger, Colorado) are doing with great success right now. The Ranger and Gladiator are close to matching the Hilux's capabilities, so the idea that there is no market for an over-built, highly capable midsized truck is nonsense.