This is another great stop after your "off shore" trip down along the NJ coast (which I've done a bunch of times). It's very historic, has great beaches, can
anchor inexpensively in front of the coast guard station, or stay at a slip, and has great restaurants.
Beyond that, the Chesapeake Bay is a great cruising area and worth spending time researching and visiting...even beyond Annapolis.
Sounds like fun...have a good time.
Moe