First of all, most cruising boats have their knotmeters installed off to one side of the centerline and so tend to read differently from one tack to the other. Beyond that, there may be differences in the jib sheet lead (The block on the track that the jib passes through) from side to side so you may want to check that. Roller furling jibs have heavy sunshields on one side of their jibs. I have seen that change performance from one side of the boat to the other. Other things might be a warped or assymetrical keel or rudder.
Could also be your mast isn''t plumb port/starboard. (It can be straight on both tacks, but if it leans a little to one side to start with, that side may be the slower tack. If you''re counting jib car track holes, as a further to the above post, make sure each of the pairs of holes are the same distance from the headstay on both sides.
Thanks for all the help. We did think, all last season, it was the knot meter being off center. But last weekend it also was true on the GPS, so we ruled it to be a true performance problem. But we will check the other things as well.