Thanks for the feedback. SD was right about the binnacle comment.
I found some box speakers that will fit in my jib sheet/winch handle storage cut outs (is there a name for these?) in my cockpit. I'm thinking that may be a better solution. I have two concerns with that layout. One, the speakers will be facing each other across a narrow cockpit. I think if I angle them slightly toward the front this should not be an issue. Second, they will likely take some abuse from winch handles and jib sheets.
On the positive side they will be fairly well protected from the elements and other than screws and holes for wires no additional holes in the fiberglass.