Agreed, though the argument I always hear pertains to the off road limitations of a hard sided camper. Though I (coming from someone who's idea off road is a 1985 4Runner on 35s) feel that any vehicle that can carry even a pop up camper is already limited to off-road by the sheer nature of the vehicle.One of the best modifications I've done is to add hydro-assist steering. Besides making turning big tires easy, a side-benefit of hydro-assist is that it is the best steering damper around.
Our camper is around 11'6" tall. It is unaffected by side-winds by and large. Adding hydro-assist is going to be an early mod in all future trucks where possible.
Regarding the topic of this thread, it always surprises me that the market for pop-up campers is as large as it is. For camping for an extended period of time a hard side camper has no downsides and numerous positives as others have mentioned already.