I had the outhaul as tight as all the strength I could muster.
Thats your problem !!!!!!!!! youre not sailing, you're testing the ultimate tensile strength of the sail material, and are your're not 'shaping' the sail to get the 'optimum' out of it.
If you take ANY piece of cloth and *overstretch* it along any 'edge' it will by 'bias realignment' of the fibers form a large CREASE (on a sail called a 'girt') close to that edge. ...could even look like a full baby diaper.
You dont state if the foot is attached to the boom or if its 'loose footed'.
A mainsail that cut for slugs or 'boltrope' attachment to the boom will usually have EXTRA material added to make a smooth 'connection' between the 'working' lower panels for the intended purpose of allowing sufficient draft in the bottom panels. This is called a shelf-foot and allows for maximum draft along the foot of the sail and 'closes the space between the 'working' part of the lower panels to the boom - the thought being that the shelf foot would prevent 'short circuiting' of the air from high pressure (windward side) to the low pressure (leeward side). Without the shelf foot, the sail at the foot section will be very FLAT.
A loose footed sail doesnt have any cloth between the boom and bottom panels.
Both configurations are equal aerodynamically, the loose foot easier to shape the bottom sections.
Both configurations can be permanently deformed by too too much tension on the outhaul, etc.
Putting all the strength you can muster in ANY sail shape adjustment is a sure-fire way to permanently deform the sail ... the sail material will exceed it elastic limit and will soon be permanently stretched out of shape.
Flat sails are for 'speed' -like 'high gear' on an automobile.
Full drafted sails are for 'power' to punch through waves - lower gear.
Neither shape will ensure that you will be at optimum when beating !!!!!!!!
You need to carefully shape the sails in comparison to 'some metrics' to get the most out the boat ..... eg. making any trim/shape adjustment and then checking with the SPEEDO to validate if that change of shape or trim was beneficial or not. When beating, reaching or broad reaching a more prcise 'instrument' to qualify the 'metrics' is a GPS with VMG (velocity made good) function. VMG will optimize the boat speed AND the BEST course angle of the 'beat'.
For each and every combination of wind speed and wave condition there is ONE and ONLY ONE 'shape' that is optimum. If you arent using a FULL set of tell tales so that you can at least visualize if the 'shape' is working well\ ... you are only guessing. A FULL set of tales is: Mainsail - midcord and leech; Jib - luff & leech ... and a ROW of 4-6 tales about 1/4th the way up from the foot to ensure that you are STEERING correctly to keep the precise angles of attack.
Beating optimization comes from
1. The sail plan *shaping* is controlled by observing the tell tales .... the BEST ever method of sail shaping via tell tales is ArvelGentry.com
---->magazine articles -----> and then 4 articles of a series: Checking Trim on the Wind, ---> Achieving Proper Balance, ---> Sailing to Windward, ----> Are You at Optimum Trim?
Only after you get near perfect shape and trim ... then will sail close hauled like a banshee ..... and the sails will be near perfect shape simply by using the tell tales and the SPEEDO (or VMG).
2. a very balanced helm with very little 'weather helm', a rig that is properly tuned so that the forestay with jib attached and windloaded .... doesnt sag off to leeward. Eg. optimization includes absolutely correct backstay tension so that the leading edge luff shape that the sailmaker cut into the luff is EXACTLY matching the 'expected sag' in the forestay for the conditions present. Once the rigging in near perfect 'tune'/tension - the forestay sag is controlled by the backstay tension.
If the mast is raked reasonably well, the way you control weather helm is to adjust mainsail halyard or cunningham tension .... more halyard tension moves the position of maximum draft more forward and flattens the leech sections and vice versa.
Guessing or following 'rules of thumb', ancedotes, etc. ... sorry that wont get you anywhere but slow, overly heeled, skidding off to leeward, overpowered, underpowered, 'cranky' or 'dull' boat ... etc. Shape and trim based solely on a full set of tell tales. Any time you adjust the tension on ONE side of a sail, you usually have readjust the tension on the opposite edge of the sail, verify each change with the speedo (or VMG). When sail shaping, always TWO adjustments, never just one. Anytime the sail is showing 'girts' ... you are 'over-tensioning' something.
If FLAT sails were the choice for beating .... we could use plywood sheets instead of sails.
If you have to gorilla-tension a sail to get the proper optimized shape .... it needs to be taken to a sailmaker for a 'readjustment'. :-)
Hope this helps.