Leaving aside for the moment whether or not you and your family are truly prepared to make this (huge) step.. to the basic question of "how big?"...
Many boats in the mid 20 foot range claim to "sleep" 5 but clearly with 5 aboard full time they would be quite unbearable. Plenty of 40 footers will boast 5 or more berths but not all will be decent sea berths. Add to that, your kids are all teens and soon to (if not already) be taking up 'adult' space, yet they're still teenagers makes this close proximity more challenging still.
Your funding level is going to go a long way to determining which is the better boat for you as well. Though I'm loathe to recommend a cruising cat, perhaps in this situation such a boat would give you the space, a possibility of a vestige of privacy and room for everyone to "spread out". However these things are very expensive and may well be outside your budget. And they're not for everyone.
In a monohull, frankly with such a crew I think you're looking at 45-50 feet or so, and even then it would be a lucky find to get a boat that truly fills the bill. Many of the larger Benes and other mainstream boats, especially charter versions, offer a lot of accommodation (Moorings 50 sold out of the charter fleet perhaps) so maybe that's one approach to consider. Again, you're looking at quite a bit of change.
Not to say it couldn't be done smaller/cheaper, of course it depends on how well everyone gets on, ultimately how the voyages go and the challenges presented along the way.
You give no indication of your experience so far, so this whole thing could be a total pipe dream, or it may have potential for an amazing lifestyle - but it's not quite like buying the biggest motorhome you can find and setting off down the highway...
".. there is much you could do at sea with common sense.. and very little you could do without it.."
Capt G E Ericson (from "The Cruel Sea" by Nicholas Monsarrat)
1984 Fast/Nicholson 345 "FastForward"