I've put a lot of effort into getting my boat to sail by itself. This is typically the bulk of my activity while singlehanding. At first I used a simple tiller mounted electric autopilot that came with the boat. It works well but responds so slowly I use it only in benign conditions. It still gets used every now and then.
Wanting a more versatile system that could be used in bigger seas and didn't require electricity (I have only one 60W panel to generate amp hours) I graduated to sheet-to-tiller arrangements as described in John Letcher's self-steering book.
This is an extremely simple 'downwind in light airs' set-up. With other points of sail and windspeeds the arrangement of mechanical advantage or disadvantage can get a little cumbersome, leaving a cat's cradle of lines in the cockpit. Still, it's better than nothing and can work quite well.
The basic theory is to tie either your headsail or mainsail to the tiller and use it for input. As your boat tries to round up into the wind the force on your sails change. The force your sail exerts on the line pulling on the tiller usually must be amplified or dampened to pull just the right amount. The tiller goes back to neutral or slight weather helm by tying elastic cord on the other side. The pulling power of the elastic cord must be adjusted also, with cord size or strength, depending on conditions and arrangements.
This summer I added a windvane to my boat. I was fortunate enough to pick it up used for about 22% of the cost brand new. I'm not sure I would pay full price for a brand new windvane they're so expensive. Here is my Fleming Global Minor 301:
This thing is fantastic and very easy to use (especially when you're used to sheet-to-tiller arrangements). With this system you try to balance the sails as much as possible and then tie the windvane to the tiller. The big air foil goes edge into wind and if the boat steers off course the side of the air foil becomes exposed to the wind and blows over. This moves a rod which twists a rudder looking like blade which trails vertically through the water. When this blade twists its side becomes exposed to the water shooting past and gets lifted up towards the surface. This lifting moves those pulleys and the attached ropes, tied to the tiller, correcting steering.
The windvane is a closed loop system of which the tiller is one part. The air foil is the brain of the operation and the blade is the muscle. Miles and miles of hands free sailing is the result! With both the windvane and sheet-to-tiller it is important to remember that you are steering a course relative to the wind, not the compass. As the wind changes direction so do you.
Arranging your boat to steer itself is worth every effort. The time away from the helm can be used for other important tasks or simply to enjoy the sail. And getting your boat to steer itself requires problem solving and analytical thought concerning the forces at work, making you a more knowledgeable skipper.