Not too late!
Rick, for inspiration:
My husband comes from 5 generations of Kansas farmers -- NOT the environment where you would expect a sailor. His first ever time on a sailboat was an afternoon daysail on a Catalina 24, when he was 50 years old. 5 years later we live aboard fulltime, he *teaches* sailing at the Naval Academy, and we're planning to sail off 'somewhere south' in another 3 years.
Here's our personal trajectory; may not be right for everyone but worked for us. It included buying an inexpensive older boat right away (a then-25-year-old Erickson 27) that was big enough to weekend on and small enough to learn to sail on, and cheap enough that we could sell it for close to what we'd paid for it when it was time to move up. We also invested in a one-week liveaboard/learn-to-sail charter in the USVI/BVI, very intensive and focused, we learned a lot, and had lots of fun. Expensive but utterly worth it for newbies with a late start. We moved aboard as soon as it was practical. Even though we both have land jobs and sometimes our boat seems more floating condo than sailing vessel (due to time constraints) living aboard gave us the opportunity to learn & upgrade the systems, tap into the community, and save lots of $$, so when the time comes to retire & go, we'll be as ready as possible.
Best wishes, E.