Brake fade is mostly due to pad selection. Rotor mass plays a part, as the rotors are just giant heatsinks (my '06 has huge rotors and the nice 4-piston calipers, but the pads aren't intended for constant towing and will eventually cook if asked to stop huge loads repeatedly.) Pads that work well for 99% of everday driving conditions, as factory pads are designed to handle, won't work well duing the 1% of heaviest-duty use such as when towing a heavy trailer.
Heavier-duty pads will make a huge difference. Certainly, use trailer brakes if you're pulling a heavy load, but I would look at upgrading pads if you tow a lot.