I have made the trip to the Bahamas from the south shore of Long Island 3 times most of the time by myself on the boat. When to leave depends on your tolerance for cold weather on a boat. I do not like cold weather so my preferred departure time is as soon after Labor Day as I can have the boat ready.
My preferred route:
1) First night just inside Fire Is. Inlet.
3) Atlantic City
4) Cape May
Those are all fairly comfortable legs - roughly 30 miles
5) Through the Cape May canal and into Delaware Bay. Long day all the way to Chesapeake City in the C&D canal. This only works if you get the tidal currents right in the Delaware Bay. Don't try to fight them.
6) Now you have started your cruise. Take your time and enjoy Chesapeake Bay. Lots of great places to see on your way to Norfolk.
You will really enjoy your first trip down the ICW, leave yourself plenty of time to do it. Get several guides, I will let others recommend which. Plan on 35-40 mile days (statute miles on the ICW). You will get a good introduction to ICW bridges and etiquette as soon as you leave Norfolk. There is a burst of bridges to get through.
Learn passing etiquette and bridge etiquette. (and signals).
But most of all, have fun. It is a great trip.