I get that for a rock crawler or something on an SUV/pickup, but I don't see a dual wheel set up slowing something down or limiting ability on something as big as an Earth Roamer.
You're probably right in the great majority of situations. There are a few times it can get ugly, though, the most prominent of which is getting a good-sized rock wedged between the two tires. Wedge it just right and there's no easy way to get it out. There's also a potential problem with mud packing in between the wheels, and there's often some sort of not-very-rugged valve extension to the inside wheel to get damaged
Another issue is if you are going to be airing down the tires for more traction. Deflating and reinflating six tires is 50% more work; not a big deal to do once a day, a pain to do often. Also, six tires cost more than four, the sizes available may or may not be as suitable for off-pavement, and it's hard to keep track of the condition of the two adjacent sidewalls. And while the effect depends on the specifics, duals generally provide more traction surface, but less traction pressure. Whether this is good or bad depends on the surface and the situation.
But you're right to think that duals won't often be the determining factor in whether you'd eventually get somewhere. The medium and heavy trucks hauling products to remote areas are on duals, so if you're traveling on a road, even a bad road, it's likely that it's workable without SRWs. Still, there are times . . .