Get the sails down...the pfd's on and motor into the wind.
I've only been caught in a squall once, and this is what the captain of that boat did. However he motored only hard enough to maintain way. The rain was coming down so hard you couldn't see more than a few boat-lengths in any direction. The wisdom of his actions was proven when the squall let up and, lo and behold, there were boats all around us. None closer than a few hundred feet, but still...