I Would guess that what ever wood was used under the mast area in the core has gotten wet over time and turned soft
The cure will be to cut out and replace
That's exactly how it appears to me too. Seen it many times. A good fiberglass guy can fix it to where you would hardly notice.
That's not to say that the whole house hasn't sagged a little bit too. Boats are soft.