I remember listening to one of the female members of the GLSS (Great Lakes Singlehanded Society
http://www.solosailor.org/) complaining about a worried neighbor boat callin the Coast Guard on her when she went out for a late night sail. lol
I wouldn't worry about what others think, especially if you're gonna singlehand. To those who don't get it, I doubt you could explain it to them.