I seriously doubt that with the prevailing, summer winds that you could make VA Beach and return home in just three weeks--especially looking at your sailing experience. If you ran the outside with favorable winds and very few tacks, you're still looking at nearly 550 to 600 miles each way. That's a fairly long trip for a bunch of guys with very little experience.
Were you going to sail at night as well as during daylight hours? If so, the trip may be doable, but in these heavily traveled waters you will need radar in order to do this safely.
And, as stated earlier, once you get to Virginia Beach, there are just two inlets--Lynnhaven and Rudee, both of which are pretty much inaccessible to sailboats. Rudee Inlet is too shallow to even consider entering and Lynnhaven is deep enough, but the bridge clearance is just over 30-feet if I recall.
Also, if you stay on the outside there are very few inlets south of Cape May that are sailboat accessible. You can get into Roosevelt Inlet at Cape Henlopen, but the next one to the south that is navigable is Ocean City Inlet, which has to be entered at slack tide because of the currents. Farther south, Chincoteague and Wachapreague are doable because neither has bridges to contend with, but Wachapreague is constantly filling in and quite shallow at low tide. The next one south is Great Machapongo Inlet, which again is strictly a slack tide entry. Sand Shoal Inlet is a piece of cake, but there's nothing there to see other than tidal marsh, greenhead flies, black flies and mosquitoes. New Inlet can be entered if you really know the area, but there are constantly shifting sand bars in front of the inlet that you must be able to skirt--it's a tricky entrance and not much water. Once you clear Cape Charles, VA you can sneak under the High Rise Bridge, then sail across the mouth of Chesapeake Bay to Norfolk, but you'll have to navigate around the Inner Middle Grounds, a massive flat that is situated just above the Chesapeake Bay Bridge Tunnel and is only a couple feet deep. It's best to stay outside until you reach the 4th Island, then cross the tube and head to Norfolk--it's a lot safer.
Going by your crews experience and the number of pop-up, afternoon thundershowers that time of year, I would make a right turn at Delaware Bay, head for the C&D Canal, then travel down Chesapeake Bay, where you can find hundreds of save havens when those unanticipated storms pop up. Not only is it a lot safer, there's a lot more to see on the trip south.