Sure it is.
Most of us are cruisers until another boat heaves over the horizon and we then become racers! (g)
There is a point to be made there, though. I'm always suspicious of someone who says they're just a cruiser with no real interest in getting the most speed out of their boat. Do they mean it or are they just lazy?
Those whom are just racers burn out. I suspect as most do on racing alone, in any sport. For them it's not the specific mode of racing, it's the racing alone.
I think most of us are like you and much of our angst regarding choosing a boat reflects the balancing of those two often separate goals of speed over comfort. The boat manufacturers sure know about it; it's impossible to find any boat in the world that is not a racer/cruiser. Just look at their adverts!
“Scientists are people who build the Brooklyn Bridge and then buy it.”
Wm. F. Buckley, Jr.