In 50+ years of boating I have only gone overboard twice when it wasn't intentional or an anticipated possibility. Canoe racing excepted. Once as a teenager driving a class B hydroplane when the steering suddenly failed. The boat turned sharply to the left and I went straight ahead. It was so violently fast my hand did not trip the crash throttle. I survived. That is an interesting and funny, unbelievable, story. The second one was last year at our mooring in Seal Harbor when the lifeline parted as I was reaching for the pickup float. Life vests both times. WW canoe racing requires worn pfds. Plenty of possibilities to swim. Class III rapids are much more comfortable when you are wearing a life vest.
Staying aboard is the plan. That is why our sport is not called swimming!
If you have any experience with an avoidable, tragic, accidental death you understand the remorse surrounding that irreversible event. Current equipment is not too bulky to be worn comfortably. Wishing you could undo an accident that ended the life of a friend or loved one is a permanent reality. Selfish is one of the words that comes to mind when I read of people who have died while boating and were not wearing life vests, leaving their loved ones behind to grieve. For ever!
My politics are Libertarian leaning Independent. Hands off! Don't regulate my personal decisions.... But, if you want to sail with me, you will wear a life vest. It is a loving thing to do.
I am more comfortable in just shorts too, when the conditions are appropriate. But!