I think it comes down to this:
People want other things more than they want to go cruising.
They want creature comforts, a certain perceived level of safety (As if cruising is less safe than commuting on the freeway). They want pets that do not do well on a boat, like horses. They want to maintain existing social relationships, the prestige that their job gives them; There is an amazing equality among cruisers, regardless of material wealth. They may want constant connection to media - a news feed, for example. They may even be unable to give up a favorite soap opera.
Or they may simply be unable to break out of their comfort zone.
Lack of money may sound like a legitimate reason but, in my experience at least, if one really wants to go cruising, money issues are the easiest to overcome.
Health issues are a legitimate concern. You have to be in reasonably good health to go to sea. Having said that, I myself am a 62 year old disabled veteran. You do not have to be young or an athlete.
But that is just my opinion and not to be taken too seriously