It is if you're rowing into a strong headwind. More then once I was unable to make any headway rowing out to the boat. This was Monroe Harbor in Chicago and we were the third row in from the breakwater, maybe 1/3 of a mile. One time I rowed for over an hour to get to the boat but never made it. And I was young then.
Yea, that would be a bummer. I'd have to spring for an outboard in that situation. It takes me under 5 minutes to row from the dinghy dock to the boat regardless of conditions. I'm the white sailboat almost dead center.