I am hearing all kinds of comments here about how poorly furling headsails perform. I just can't agree.
I sail well over 100 days a year in the pacific northwest. We get some pretty snotty days on occasion. I regularly single-hand in 25-35 knots of wind, and regularly sail in gusts to 40 /45 kts.
I run with a main that has two reefs - the first one is moderate, and the second is quite deep. My genoa is a 135%, with a cruising cut rather than a racing cut. This means that the foot is fairly high, with a deep upward angular cut toward the clew. This means that in light air it doesn't drive the boat as hard as a low-cut racing clew, but I can see where I'm going under sail, and I don't have to furl as soon in progressively heavier air. The cruising cut Genoa also maintains a better sail shape as it furls than a racing cut sail, which is an excellent attribute.
In 35 kts I can sail broad a reach with everything up, or to windward with a single reef in the main, and the genoa partially furled to slightly less than 100%. In sustained winds over 35 kts, sailing to windward ,the second reef goes into the main, and I would furl the headsail to somewhere in the ballpark of 50-70%, but every boat is different. The 100% that you were mentioning in 40kts of wind is probably a little overpowered, and a little less square footage might have reduced the heavy luffing that you were encountering. And yes, sheet in HARD.
But seriously, if you're getting harmonic vibrations while sailing to windward with a partially furled genoa, I am going to suggest that there simply may not have been enough tension in your rig, and you may not have made your sheets hard enough. Add to this, too much sail area, and you've significantly increased your chances of having to make a trip to the sail-maker after your outing, to put your sail back together.
To eliminate the harmonic vibrations you were encountering, there are a few things you can do. If you have a backstay tensioner, use it. If you don't, increase your Genoa halyard tension, and harden your mainsail. If you don't have enough tension on your rig and sails in high winds, you are jeopardizing your sails and your rig, period. This could get very, very expensive in the space of a heartbeat. Breaking your rig would be very sad indeed, and you wouldn't be the first guy to do it.
You might want to have a professional rigger look at your rig and ensure that the tension is correct, or get your hands on a rig tension gauge, and find out what the right stay tension is for your boat for heavier weather. Unless you are a racer and have a lot of quickly adjustable gear, if you sail in higher winds, I'd leave the rig tuned for heavier weather and worry less about light wind performance.
If you are trying to reduce the drive from your main (this will only really work with a main that is not fully battened), slide the traveler somewhat to leeward, while maintaining main-sheet tension. This will deliver best results if your traveler is well forward. It will allow the Genny to spill some wind into the luff of the main, partially depowering it to give you some wiggle room while giving you some backward rake on your mast, increasing your forestay tension. This isn't a great long-term sail trim (it's a quick 'cheat', but works in bigger puffs (say over 35) on a day with average windspeeds that are a little lower.
I did like the comment by one of the other sailors, to sheet in the genoa HARD, and conceivably apply some tension to the windward sheet; I haven't had to try that second part but I can see it being slightly helpful.
Sounds like heaving to was a good strategy at that moment, to ride through the gust.
Ultimately, if your genoa won't hold a decent shape when partially furled, I would suggest picking up a good used storm jib. All of us could stand to have one, especially those who are not dedicated fair-weather sailors.
When it's forecast to blow anything over 25 kts, remember: Forecasters are right about 50% of the time. The other half the time, they are WRONG. It might blow a lot harder right where you are, than the overall forecast. I have seen gusts over 60 kts with waterspouts, on a day forecast to blow 15-25 kts, but it was an extremely localized phenomenon - right where I was. Anyhow, if your boat doesn't behave all that well in 25+ kts, bite the bullet; buy some smaller canvas, and use it when it's blowing.
Remember, it's preferable to add canvas when you are bored, than to have to reduce it when you are terrified. Leave the dock ready to sail with less area to give yourself a margin of error, and plan to increase sail, rather than vice-versa. When do you reduce sail? The moment that you are wondering whether or not you should!
Also, whether or not you will make any way to windward with a double-reefed main only in such conditions will really depend on your rig's design, and the design of your boat under the waterline, not to mention how clean your bottom is. Only 1 way to find out - give it a whirl when you aren't exposed to a lee shore!
I need a bit of advice and hope that more knowledgeable salts could help; I went out today single handed for a half day sail on a 40’ monohull. The for-cast was for 20 to 25 knots in Moreton bay, Brisbane. All was fine until I noticed a storm approaching. About 15 minutes later I noticed the water ahead was white as the rain bucketed sideways. I was already on second reef on the main and the Furling jib was reefed to around 100%. The oncoming squall was pushing me toward a lee shore about a mile away and so I decided to head into it instead of galloping with the wind at 8 knots towards the lee shore. I was also thinking that the wind wouldn’t be more that 30 knots - pretty dum really. In heading into the wind I thought that I would cope by sheeting in the jib (not too tight) and the mains and simply feather the boat close hauled as I have done many times. This worked fine until the wind started to gust to 40 knots and the half furled jib began to beat in an intermittent and uncontrollable manner. I managed to keep it calm for much of the time but every now and then the rig would vibrate with the shock loads administered by the jib.
It seems to me that the windage of furled part of the jib is large in 40 knots of wind and maybe the forstay is not tight enough (even though I manage to point well). I didn’t want the rig coming down and so I heaved to; this calmed things down with a drift speed of around 3.2 knots and 10 minutes later the wind dropped to a respectable 20 to 25 knots again.
The shudders in the rig scared the hell out of me and I was wandering what others have to say with respect furled head sails and sailing into stiff 35 to 40 knot winds. Should I have furled the jib completely and simply relied on the mains which was behaving well; the argument against the main only I guess would be that I wouldn’t be able to heave to nor even make head way.
Would love to hear if others have had similar rig vibrations from a furler head sail or indeed what they might have done differently.