1. Traveler at center, and Boom WAY out due to the gusts and not wanting to heel!
2 Where were the jib cars placed? Pretty close to fully forward
Hmmmm, as I step though this, I'm thinking that I made some mistakes!!!!
1. there is no way that a rope-vang can totally control a boom from lifting with the traveler on CL and the boom WAY out. This can cause the main to suddenly power-up via artificial BIG draft occurrences as when the boom lifts, even if the leech sections become 'open' due to the rise as the BIG draft then occurs at mid -cord. Typically in such conditions without the added down-support from the mainsheet on a dropped traveller, and especially if the gust has 'rotation' to it, this can happen. Better to 'blade-out' via the traveller and keep the main flat than to be vulnerable to a 'power-up' from a lifting boom.
2. Jib Cars forward will increase 'power-up' also as any release of jibsheet pressure will cause the leech to be the last section to 'unload', leaving the midcord sections vulnerable to 'power-up' and overall sail instability. A bit 'aft' of normal cars will allow a better 'blade out' either by slow jibsheet easing or helm/steerage.
If the leeches are the last to 'unload', the CE goes WAY aft!.
Question? when this windward broach (?) occurred, do you remember what the main and jib looked like immediately before the broach happened ..... a BIG "S" shape along the cord length of the sail?
FWIW, In lightweight ILYA scows, if the sails arent severely flattened, the leeches open/slack, and the boom isnt almost 'directly over' the traveller car
such power-ups can result in a severe 'dunking' (sometimes to windward) when that "S" shape develops in the sails to warn of an immediate upcoming 'power-up'.
If this is the case for your boat suggest 'playing the traveller
' to keep control of the main and to keep the boat on her feet; and, leave the mainsheet alone/cleated, except for 'emergencies'. Forget the mainsheet, play the traveller to keep the main 'bladed' and under control in the 'rolling gusties' .... and its 'faster' too.