That's a great ride. My first ride from Estes to the summit was in '86 when I was 14 with a young buddy you might have heard of...Jonathan Vaughters.
When I was living in Fort Collins, Co in the early 90s I would do that ride at least 5 or 6 times a year......leaving from Fort Collins! I'd leave my house at 4500 feet, ride to Estes and to the Alpine Visitor's Center at over 12,000ft. Then, I'd head home. I can't remember the exact mileage but it was well over 120 and would eat a weekend with a day of recovery. Damn good fun. I was also a 24 year old Cat 2 at the time. As a 37 year old Cat 2 I might not be so bold.
The climb up from Grand lake isn't so bad. The only thing that sucks about that climb is the traffic. It sucks to be rippin' down the pass only to get stopped by a station wagon from Ohio blocking the road as they photograph a chipmunk.
The scenery is better on the Estes side, for sure. I'm not such a fan of downhills. I enjoy the climbs more than the downhill but the drop into Estes is actually really nice.
If you're up for another classic climb, head over to Durango and ride the Iron Horse route. That's a great climb as well with some screamer descents. If you really seek a screamer, the drop from the Climax mine on Freemont Pass to Copper Mountain is wicked fast. That is the only time I ever really got over 60mph. That was a real chamois pooper.