Yacht racing will never be a major spectator sport in this country, because people who have never sailed can't relate to it. In order to really get into a sport, you have to relate to it. You can't really relate to a sport unless you have played it, and most people in this country have never sailed. Even many cruising sailors don't understand the appeal of racing a sailboat. People who have never played golf often can't understand it's attraction, because they don't appreciate how difficult it is to hit that little white ball, and make it travel the right distance in the right direction. People who love a sport don't care whether spectators enjoy watching it. They play the sport purely for the love of the sport.
Yacht racing is not a physical sport. It's a cerebral sport. It's an extremely complex sport. The winner of a yacht race is not the one who is strongest or most fleet of foot. The winner is the smartest racer who knows the most, and who accurately reads the wind and the course conditions and uses that information most efficiently to move the boat from point A to point B.
Smack, I hear what you're saying, but I think I know enough about you from your participation on this forum to be able to say with confidence that, if you ever get on a racing sailboat and participate in some races, you'll be bitten by it. It's difficult to describe the thrill of sailing in the queue before the start of a race, weaving in and out through dozens of big, expensive boats, and maneuvering the boat into the best position for the start. As the start nears, the boats are all maneuvering aggressively to get into the one ideal spot to cross the line, but they obviously can't all be there at once, so they start yelling rules at each other, and it becomes a test of wills. The mark roundings in a big race are exciting, as 3 or 4 boats hit the mark together, and fight for room and position. Good racers will be constantly adjusting sail trim, playing the sails in and out to maximize boat speed all around the course, and to keep the boat driving in light air or in choppy seas.
Crew on some racing boats in a few races with decent winds, and you'll be hooked. Most people don't enjoy light air racing, because it's the most difficult to do, and not many people are good at it, but, if you learn how to keep the boat moving in light air, you'll love it as much as racing in big winds. Yacht racing isn't about speed - it's about skill.