I'd go for the boat with the best hull shape beneath the waterline, the better sail/displacement ratio, and the better performer in light air.
I've seen a Columbia 24 Challenger, and it's a cute pocket cruiser design, but the wetted surface beneath the waterline is pretty major, and I got the sense that it would be anything but a light air boat. You should check a copy of "Royce's Sailing Ilustrated" out of the library, since that author owned and taught sailing lessons for years on a Columbia 24 Challenger, and loved it. He was in So. Cal., however, and had plenty of wind.
I've never seen a Columbia 22, but from pics online it appears that they are more of a fin keel design with less wetted surface. If the sail/displacement ratio was better, and the underbody was more modern, I'd likely go for the 22 for a first keel boat. It will likely be easier to sail in light air conditions, easier to dock, and possibly more responsive and fun as a first boat.
Going for something heavier with more room below may be what you want in the future, but be aware of the drawbacks before making it your first boat. Let us know what you decide.