I absolutely love my Challenger 40', Elizabeth, which I've had for almost a year. They are really well built boats and I'm extremely happy with how she sails (I've had her moving over 10 knots flying the #3, Main, and Mizzen). Mine is a ketch rig, which I really like, although I'd be interested to check out the sloop. I had little experience with ketch rigs before I started looking at her and read a lot about ketches and yawls in the run up to buying her. I learned that there is a lot of hatred for the rig out there, I'm glad I didn't listen to the haters; I like it for the following reasons:
It's easy to reduce sail - over 30 knots of breeze I can fly my mizzen and headsail only and still move along just fine - and I'd argue with more control than a reefed main.
Splitting the sail plan makes the boat more trimmable for women or those small of stature.
Three sails looks cooler than two.
Mizzen tricks - you can use the mizzen for a lot of fun tricks your sloop friends don't know about.
That's my ketches are great bit, as to the Challenger, I think she has a lot to offer. A few things to watch out for -
The original mainmast step on some of these boats was a welded bridge that spans the bottom of the hull where the mast passes through the cabin sole. I saw photos of someones Challenger down in Central America that had badly corroded - this is in the foreward head so figure it might have had to do with use of the shower. Mine is stepped into a large (approx 12"x12") block that is glassed in to the hull - prefferential I think, don't know of it is original, I suspect it was done when the boat's standing rigging was redone. That would be something to look at if it is accessible.
A few other beefs I have - The wiring harness on mine left some things to be desired - it would have been nice if they'd followed colour-coding convention when they ran wires - also they were quite a few splices below the cabin sole - less than desirable.
The keel is really deep, under the engine, and is almost impossible to clear of standing water - let alone clean. I think they made Challengers in a full keel model as well that might be better for this (mine is a fin keel and has a six and a half foot draft).
Mine had some issues with secondary bonding - some tabbing had from the hull at the connections to the bulkheads. Not sure why - I fixed all that with epoxy after I bought the boat. Something to look at.
And the good:
Roomy, as you noted, nice headroom, solid layup (I think the hull is an inch think!), mine has a Perkins 4-107, which is a great little engine, generally well thought out systems, nice interior woodwork, etc.
I did what you're doing when I bought mine and got some nice testimonials from folks on here - one bloke said he'd take his anywhere - now that I've had mine for a year I'd agree with that too.
Sailing a '74 Challenger 40' Ketch rig out of San Francisco