I think, for how I'm likely to use the boat, a windvane would be the way to go. I know the autopilots can be power hogs - and you can always add them later.
But it seems like most used boats in the 34-40 foot range have autopliots installed by their previous skippers. I've only seen a few with vanes. Maybe your point about needing an autopilot when raising sails is something to ponder...