I'd also think about grinding out and re-sealing the top of your board above the pivot hole where it shows damage. You are lucky your swing keel/center board has not swelled as this damaged area clearly is below the water line and would allow water to intrude into the core.
I hear you. The thing has lasted 38 years tho. What ever water that gets in there obviously drains/drys out before winter has a chance to do any damage. I would worry about "fixing" it such that any opening is large enough to allow water in, but too small to allow it to come out or dry out. The core is a very hard foam, almost like bondo hard. If you look at the pic, you can see some delamination at the pivot hole. To seal it all perfectly may not be possible. There is a saying in the electrical trade that goes: "If you want to make something water proof, drill a drain hole in the bottom". Attempting a "fix" could spell disaster in this case. I'll try my luck as is.