We've been through 24, 28, up to 40 feet, and now down to 35. The current boat still feels 'small' and manageable for two after the 40 footer.
Depends on your situation, of course, but we wanted to be easily able to coastal cruise and live comfortably aboard for up to 2 months at a time, and the ability to host occasional guests (usually couples) without hardship. For us this size seems ideal.
We've looked at upgrading (but based on age, not size) and there are plenty of 32 footers whose interiors outstrip our boat now, but having a hard time justifying what we'd have to pay vs what we'd actually gain in real, tangible improvements. The biggest thing we'd lose is storage (lots of it), the only 'real' tangible gain would be a much larger, spacious cockpit.
So we're 'settling' for what we have (barring a lotto win