Where to draw the size line
is a tough question. I tend to think of it in terms of what can I single-hand even when I'm cruising with family. I've been with some people male or female that I'm sure can keep going longer than I. On the other hand I've been with some where after the first wave over the bow they go below and are done.
So if you have a very small crew, two or three especially if they are not very experienced, I think that the chance of you in reality being single handing is very high. Between seasickness, exhaustion, injury and fear your crew options can go down very quickly.
So if you agree with the above assessment the next step to think about is how much are you going to rely on power winches
What it the biggest anchor
you can manually haul up?
What it the biggest main sheet you can hoist?
You may decide the extra size is worth it but you are making a choice that you have to rely on the power. Not necessarily a bad choice but a significant one, at least in my opinion.