The high speed test sets load and psi, and varies speed (increasing). Temperature is usually the limiting factor at ~200F as the rubber compounds deteriorate in performance above that.
The main thing generating heat is hysteresis in the sidewalls and tread where it meets the ground. So you should be safe on the road at low psi so long as the tire is no more squashed than it was at the max load.
When airing down offroad you're typically nowhere near the speed rating of your tire, so the heat generated (which is a little above linear with speed) will also be less. The cooling component due to airflow will also be less (varies less than linearly with speed), but the conduction to the ground (doesn't vary with speed) will be greater, due to more surface contact. Since I can't find any reliable analysis or testing of this... I dunno...
If you're concerned about it, measure your tire temperature with a IR deal.
Agree with what you said, but will add: tire deformation + thickness of the rubber determines your heat generated. i.e. a 1" thick sidewall vs 1/8" sidewall, both deformed by the same amount under (different) load, running the same speed - your thicker sidewall generates significantly more heat. Hence why E-rated tires need more psi for the same load, it's to reduce deformation.
Yes, off road it won't matter much. But as you mentioned earlier, rock crawling vs bombing down a washboard at speed running King long travels are 2 different things.
A P265/75R16 at 22psi, will retain more of its speed rating so to speak, compared to an LT265/75R16 at 22 psi. How much speed rating is retain, we won't know without temp readings.