The Bill Garden design Formosa 41 originally had quite a radically raked mainmast ... looks like up to 7 degrees of rake on the 'spec'. drawings. Many of such radically mast raked boats from that era now usually have their masts 'straight up' for 'sail/helm balance' considerations ... and if now carried more 'straight-up' will radically/directly change the boom angle.
So first, the pregnant question becomes, how much rake do you normally carry in that main mast
? That approx. rake angle will be needed before I can give a reasonable 'guestimate' reply and one based on generalized 'sailmakers data' for the F41.
Id need to know the construction of the mainsail ....
.... woven dacron sail cloth?
.... does this sail have a luff bolt rope (a three strand dacron rope inside a 'sleeve' at the luff?
could even be that there is an 'exposed' three strand 'rope' sewn directly to the luff ... yes/no?
.... What is the luff length of the sail (my data book omits the F41)?
.... How old (yrs.) is this sail?
.... and how much 'extra strain' do you add to the mainsail luff when 'raising' this (dacron?) mainsail?
.... if you don't 'stretch out' that bolt rope, explain briefly how you raise that mainsail.
Two things possibly going on here that I can 'guesstimate'; but, I would need some/most of those questions answered before giving my 'best guess'.
If this sail is a bolt-roped woven dacron sail, and is more than about about 3-5 years old and the sail has seen a LOT of use .... probably its not the angles that the reef positions are making but rather your boltrope in that mainsail 'may' be grossly shrunken, the luff length is now MUCH shorter than when 'new' and as a result the boom 'may' be drooping at the aft end, giving the appearance of being 'normal'; BUT, if the mast is no longer raked to the OEM design the boom may be giving a 'visual' cancellation, etc.
This is quite complex; but, the amount of rake in that mast is the key here on what is exactly happening - as you SEE it ..... Plus 'how much strain' you are adding to the halyard .... AFTER that sail is 'just up' when raising.
My first suspicion is that your mainsail luff has shrunken over time (quite common on aged boltroped sails) and the dissimilar shrinkage of that bolt rope is the reason for the apparent different 'visual' angles you see when 'full up' or when reefed. When raising this mainsail you probably winch the halyard so that the sail is 'just up'? correct?