Fiberglass (GRP) is very strong in tension but not so strong in compression. It's not the best choice for a compression post.
The advice of a marine acrchitect / engineer should be sought. Modifications to the deck, chainplates and standing rigging would likely be required. If your only reason for doing this is cost, it may be cheaper to replace the mast.