Rudder post slop & "thunking"
If you were to wiggle the rudders of the boats in your yard this fall, my guess is that 9 out of 10 are loose and have excess play. Some boat manufacturer had the foresight to incorporate a bushing system into their design to correct the situation you''ve described when the rudder tube assembly has too much play. In the fall at haul out inspect your boat to determine if such a bushing system was originally installed. I don''t know where you''ll get them as the sailboat portion of the S2s business is no longer in production although the company still makes stinkpots.
Finding bushings for your boat may be more difficult than finding Weapons of Mass Destruction in Iraq. That said, Harken does offer a limited number.
There is a stuffing box on some inboard rudder designs, not all. If one does exist on your boat, tightening it will not make much difference.
I doubt if enough force during normal sailing could be generated to crack the rudder tube, it''s probably more annoying than structurally dangerous.
I replaced both upper and lower bushings in my boat a few years ago; it is the 1 out of 10 that is nice and tight :-) I was able to get the bushings from the boat manufacturer.