As I understand polars, they are saying that the speed on the curve, is the best you will do in those winds. They are not saying that you can always go at the high of a point sailing. So if I understand what you are asking, yes, sometimes you can not point as high, nor want to because of sea state.
I have seen polars with different sea state comparisons too. Then there will even be different rig configs depending upon sea state also. IE, a full main and 110 vs a single reef and 155 for the same wind condition, one being better in 3' chop, the other in 1' waves. Or as I saw once for the V70 follks 40-45knots of wind vs how high the waves were, they had 3 or 4 sail opions, from single to triple reefs with varying jib sizes.
She drives me boat,
I drives me dinghy!