The mast is straight when checked at the dock, but your shrouds may be out of tune, allowing more fall off on one tack, than the other. Also, check the symmetry of your jib car positions as MarkSF mentions.
Something is assymmetric about your setup, that's for sure.
S/V Old Shoes
1973 Pearson 30 #255