It all depends on your priorities. If your major priority is to get out there and cruise, then going with a smaller boat, that has simpler systems and fewer amenities is the way to go. If you need to have all the comforts of a shore-based life and want to have the big queen size berth, the 42" LCD tv, with DVD, hot and cold pressure water, 110 VAC available at all times, then you won't be happy.
The Pardeys have spent decades on boats smaller than 30', and put hundreds of thousands of miles under the keel of their two boats.
It also depends on how many people you are talking about. For one or two very close people, a <30' boat is possible. Once you get more than two people, you'll probably need more space at a minimum because of the increased stowage requirements.
Smaller boats have lower costs overall. They have the ability to anchor in more places, and require less draft generally. They're often less likely to be attacked by pirates. They're easier to handle in many ways. They're usually slower. They're not as comfortable. I wrote a bit about this in a previous thread I started
I've noticed that some people on this site seem to discount the idea that a smaller boat can be the ideal boat for a person. I guess it has a lot to do with what you got the boat for.
If you want a floating condo, with all the pleasures and conveniences of modern life, including the microwave oven, the big screen TV, and the washer and dryer, you can't do that on a smaller boat.
If you want to live aboard a boat and run a business from it, a larger boat may make sense. You need to have living space as well as office space. I know a man who runs a very successful company from an office on a 34' catamaran... where he lives and has his office.
However, if you want to sail to far away places, and live a life that is based around sailing, a smaller boat may make a lot more sense.
Look at what boats some very well respected sailboat designers chose. Many chose smaller boats for their personal sailing craft.
Capt. Nat Herreshoff designed for himself the 26' "Alerion III". When Capt. Nat was in his seventies and living in Florida, he sailed a 30' K/CB "Pleasure"
Joel White sailed a Bridges Point 24 named "Ellisha" after his grandaughter
Phil Rhodes sailed a wooden 25 footer named "Nixie"
Carl Alberg sailed a 26' Pearson Commander named after his wife "Alma"
Bob Perry sails a 26' Cirrus called "Perrywinkle"
Part of the problem with a lot of the mass media is that they are driven by advertising dollars...and the larger boats are where all the money is. Look at the pages of Blue Water Sailing, Cruising World, and those magazines, it would seem that you can't sail across an ocean in a boat smaller than 40' in length.
Part of the problem is that many people confuse cruising with chartering. Most long-term cruising sailors I know have tried to simplify their lives and have gotten out of the rat race, and the rat race's need to compete with the Joneses.
However, I believe you can go and do a lot of sailing, and even sail long distances in relatively small boats. In fact, this was in fact the norm until not too long ago.
Look at Tania Aebi, who sailed a Contessa 26, Donna Lange, who is in a Southern Cross 28, Pat Henry, who was in a Southern Cross 31, have all circumnavigated the globe... okay, Donna's not quite done yet, but she's in the home stretch... The Pardeys sailed aboard two different boats, Talesin was less than 30' LOD, and Serrafyn was less than 25' LOD. Webb Chiles circumnavigated four times, once in a Drascombe Lugger, but that's an extreme example IMHO.
John Vigor's book, 20 Small Sailboats to Take You Anywhere, lists boats all smaller than 35' LOA. Bigger isn't necessarily better... if your goal is to sail and cruise for as long as possible... then a smaller boat may make far more sense.
A smaller boat costs less to buy....leaving you more money for the cruising kitty.
A smaller boat costs less to maintain....making the money in the cruising kitty last longer.
A smaller boat often can go more places than a larger boat. Bigger boats don't gunkhole well.
The small boat is often easier to repair. The hardware needed on a smaller boat is often simpler and more reliable than that on a larger boat. A manual windlass is going to have fewer maintenance problems than an electric or hydraulic windlass; an electric winch is going to need more maintenance than a manual winch; the electrical system is often far simpler, and require less work to repair; stepping the mast often can be done without a crane, and so on.