Unfortunately, Maryland's and most of Virginia's state parks now require admission fees, which range from $5 per person and up - even if you come in by dinghy. The park rangers can be real SOBs when they see you landing at the beaches and will tell you to leave in no uncertain terms. Even if you shelled out the entrance fees, the parks are overrun on weekends and holidays - it's not a pretty scene at all.
I think you may be able to land at the Hart/Miller Island Complex without paying a fee, but not positive about this. Of course, the only way there is via boat. Also, keep in mind there are no dinghy docks or piers at any of Maryland's state parks where you can tie up for a day, or even a few hours. In some parks, such as Janes Island, Gunpowder, Sandy Point and a few others, they have marinas, but they are exclusively for slip holders that pay in advance, and most are situated in very shallow water.
Many of the waterside restaurants will allow you to tie up to their piers with your boat or dinghy if you are dining there, and some, if it is late in the day, will allow you to stay overnight at no charge.
Kiptopeke State Park, on the southern Delmarva Peninsula, is a great park to explore with miles of fantastic beaches. There is a great anchorage behind the Concrete Liberty Ships, and you could easily swim to the nearby beach, but knowing what I know about the shark population there, I wouldn't recommend it. There's also a nice public beach at Cape Charles, but you will have to pay to stay in the City Marina and it ain't cheap. Good restaurant, though an very nice, new marina.
Good luck on your quest, and I wish I could be of more help,