Personally I would put ease of sail handling at the top of the list. My kids are 2 and 3 right now so one of us is always on kid watch. The boat we have access to (not own) is a San Juan 7.7 (26') and all we can do is motor. The sails are set up such that I cannot sail the boat alone and my wife is too busy with the kids to help much. Problems with the San Juan are as follows (with solutions). The
jib is set up with a luff foil where a rope on the forward part of the
jib slides in a track on the forestay. Very popular with racers as it makes the
jib more efficient. The problem is that someone has to go forward to feed the sail into the foil to raise it and when it comes down it is not attached to anything and therefore goes flying. You want roller
furling on a family cruiser. The mainsail has the same problem, you'll want
lazy jacks to keep the main under control.
Winches require some one to tail and someone to crank. I can do this by myself, my wife can't. Self tailing
winches should be very close to the top of your list. An auto pilot so the helmsman can leave the helm for a minute to tend to sails etc will be a big help. As far as comfort below, that is very dependent on you. We have spent most of our married lives (16yrs) hiking, backpacking, and bike touring. For us, having an enclosed head with a flush toilet is luxury. I am currently setting up our boat with one settee that folds up into bunks with lee cloths for our two kids while we have the v-berth. The ameniites on a boat will not equal your house until you get to a boat that is very large and expensive. I would look for a boat that will comfortably sleep the four of you. Extra berths are installed at the expense of storage and functionality. If you are weekending I would look at using some solar showers in the cockpit to clean up with. Keeping a head that is also used as a shower clean can be a real pain and that much moisture inside the boat can be a problem. Something else to look at is ventilation. You will have four people living inside a small compartment. It will become hot and humid without lots of air exchange. At least some opening ports will be very helpful. Good luck on your search.