I guess the problem with retaining an X5's on-road prowess is that the first thing on anyone's list of improvements would probably be a set of AT tires.
My wife and I are dealing with this question ourselves right now. We have an E53 X that's old enough not to be precious anymore, so it will be our travel vehicle until we decide on the right vehicle/trailer combo that will be our more permanent solution. So it's a debate on how much, if anything, we want to put into the X as a temporary solution. It needs tires anyway, so maybe it's a given that I get a set of AT's, which cost the same or less than what I'd otherwise put on it. But then I should probably replace the spare to match, which adds a few hundred for the tire and the same for the wheel. And if I do that, then the spare won't fit under the rear deck, so I'm looking at a hitch carrier like a HitchGate for $800+, or maybe carrying the spare inside. Which brings up the question of storage - maybe find a roof rack that could be moved to another vehicle or trailer later on? If so, I'd probably end up with a compromise that wouldn't be exactly what I'd want for either. Then there's the question of recovery gear. On the one hand you could argue that we won't (shouldn't) take the X anywhere we'd even run the risk of getting stuck. On the other, you could argue that the less capable the vehicle, the greater the need for some sort of recovery plan.
Right now all I've been willing to invest in are a portable compressor, puncture kit, and a battery jump pack - all of which are basics and will be useful no matter what we decide. I'll probably get a Thule or Yakima cargo box rather than a rack, since I already have a set of crossbars, and can use a cargo box on any future vehicle or trailer. I'm debating the tire question, as well as some recovery solution. A recovery kit and a come along is probably a practical choice. I could mount a winch to a receiver plate of course, with the plan being to install the winch on whatever vehicle we end up with, but I think that idea is well into the realm of overkill.
Or I could just bite the bullet and go get that Wrangler.