Were I a bettin man, I would venture that 99% of the issue is the bathroom. Either a lack of bathroom (remote camping), or a public bathroom (public camping). I know women that wont use a public bathroom, and are thus tethered to home even around town.
There may be additional reasons, and you'd probably need years of analysis to bring it out, but there it is. There are psycho disorders that are gender-specific, this may be one.