I go over the side at least twice a summer to clean the prop. Besides loading the engine, I lose at least half a knot with a dirty prop. We often swim off the boat and cleaning the prop, checking the centerboard and rudder are normal maintenance things. I keep a masks and fins on the boat for just this reason (and in case I snag a crab pot). I use a putty knife. Works great; sometimes I skin a knuckle, but nothing significant. I like the gloves idea, though.
Sabre 38 "Victoria"