Help me settle a bet!
I agree completely with your statement, but I want to "defend" my take on this:
I could be wrong, but I believe the term "lee shore" is purely nautical in origin and use. I don''t believe non-boaters standing on land would describe the shore onto which the wind is blowing and the waves are crashing as "lee" anything. In fact, people I know who live on islands (and are not boaters) will often talk about visiting the windward side of the island and seeing the waves crashing. To them (on the island) the side where the wind is coming from is the windward side. When you think about it, they are simply applying the same convention we boaters use while aboard.
I just did a Google search of the term "lee shore" and "windward shore" and found numerous examples of usage describing the two sides of island coastlines (even more with the terms "lee coast" and "lee side") where the leeward shore is the calm side and the windward shore is the rough side. [yes, I know that finding numerous examples of usage does not necessarily make it correct]
My points are simply that (1) it is easy to understand why there is confusion, and (2) the non-boating public appears to be applying the frame-of-reference definition.