If you look at most fractional
rigs you will see that the mast is considerably further forward than most mastheads. Also since the boat is designed with this
rig in mind the keel is likely further forward as well.
The biggest problem with this idea is that you are shifting the center of effort of the
rig and depowering at the same time (if using the same stick). This will likely lead to increased weather helm and reduced performance. Even this approach would still require a fair amount of mods to the existing mast.
To turn a typical masthead boat into a good frac would most likely require shifting the mast forward, and making the main proportionately larger to compensate for the lost sail area of the smaller foretriangle. All of this has to be done with the intent of keeping the center of effort properly aligned with the center of resistance of the hull and fin.
Relocating the mast in the boat is a huge deal - chain plate/bulkhead relocation, moving partners if keelstepped, mast support if deck stepped, and doing all this in a structurally engineered fashion would be a huge undertaking.
It may help to know what boat you are contemplating doing this to.
If you really want a frac it would probably cost less to sell and buy another design that's better suited to your needs.