The guys who passed you were almost certainly racing in the RCRA/NPSA Wednesday night series. I've encountered a lot of racing skippers who (falsely) believe they don't have to give way to a leisure sailor during a race. When I'm not racing I just try to avoid boats in a regatta for that reason.
I think if I had been in your situation I would have yelled "leeward boat" to the skipper of the other boat. If they didn't respond to that, words would be exchanged but I'd have tacked off to avoid going into irons as they took me up.
I was in that race last night (and I swear it wasn't me!!!) and it was quite a wild ride!
Beneteau Oceanis 400
formerly Lippincott 30