People would not do it if it was not challenging and airplanes fall out of the sky pretty regular
But just for perspective - driving a car on a busy highway at rush hour is pretty risky too. And no one thinks that's unusual.
This powerpoint from US Naval Academy
(look at page 6) - puts driving at 30 deaths/100k participants, riding a bike at 0.09, sailing at 0.03 (all normalized for average hours of participation per person).