The boat should be sail-able by crew number minus one. That is, your wife needs to be able to bring it back to harbour alone with you incapacitated below. The advice on how long that is has been going up over the years. It used to be 35 - 37 ft, now its over 40 or even 50 ft. So I don't think that length is a deciding factor any more. In practice, I think parking the boat is the real issue, the bigger it is, the fewer parking places there are.
It is expense that makes most people settle on the final boat choice. For a couple, I think below 30ft is undesirable and above 40 ft unnecessary, but if you can afford it, go for what you fancy.