Same thing happen to me years ago on my Santana 22. A buddy plugged the hole while I drove to the chandlery to get the underwater epoxy. It's a two part system in clay like form that you mix together and stick in the hole.
That experience taught me to always keep that underwater epoxy stuff and a variety of wood plugs on board.
Btw, I wouldn't trust plumber's putty. Go online to either West Marine or Defender.com and look for something like Marine-Tex.
FAQ's for products in the Marine-Tex line