And yet, despite our shared experience of "failed" hunts -- "A bad day in the bush beats a good day most other places", doesn't it?
All hunting talk aside, for your truck situation, I will play devils advocate to the recommendations for a bigger truck and suggest that thinking critically about your desired outcome, and what build will get you there, might give you other options. For example, what does "4-season camper" mean to you? Would you consider an Alucab/GFC-style Canopy Camper with a wood stove or fuel-fired heater in it be adequate for winter, enough to take the edge off of a below zero night, or are you wanting the typical Elkhart, Indiana-style white plastic box with poor quality finishings and an overreliance on spray foam to "winterize" it? For an occasional weekend or two-party hunting trip you might find the wedge campers to be a solid option especially considering they are sometimes 1/5th the price of the Elkhart campers and far, far lighter so that they would work on a 1/2 ton truck.
And the reason I'm bringing all this up, is that as you chase higher payload, you almost inevitably also catch higher vehicle weight, and heavy vehicles are always at a disadvantage off road. Even a "mild" HD truck will be a couple thousand pounds heavier than a half ton (Just the weight of the empty vehicle is a thousand pounds difference, plus all the extra stuff you can bring). You may find that the hassle of that extra weight takes away from your experience more than a more minimalist build on a lighter vehicle would, depending on the types of places you like to go.
You will not fit a family of 5 with a typical 4-season Elkhart-style camper in anything less than a large HD truck in my opinion, but if you are wanting to hit up some trails on occasional weekends, your family might have more fun in something like an F-150 with a high payload package -- giving you F-150 weight (and durability, which is good but not as good as an HD-level truck) but with 3,000 lbs payload -- and a lightweight wedge camper build, and most wedge campers are very versatile on the interior so you can easily outfit them differently for hunting versus family trips. Depending on the size of your kids though not everyone will fit inside the wedge, but you can do an awning and room combo or a ground tent for the 'overflow'.
Whatever you go with - build it all on paper, every component - and make sure you are getting the right rig for the job. But being honest with your needs could save you a lot of money and result in a better product at the end of the day for what you love to do most.