We sail out of Southall Landings on Salt Ponds just north of Hampton. I've found my dingy quite useful, even in the marina - I can use it as a cleaning platform for working on the hull. I looked a good bit at brands/prices before I purchased.
Check out Sea Eagle. They have a large variety of inflatables. My dingy is a 4 man (SE 8, I think), very reasonably priced. I added a motor mount and seat assembly about a month ago, for an electric trolling motor. Looking forward to trying it out in the next month or so. I got the motor with battery & a solar panel from Float Green. (You may be able to get some discounting at both locations, if you mention my name as a referral.
We've yet to do much 'exploring' outside the lower Bay - but I know there are numerous places to put the hook down. Marinas can be pricey for overnight - the lowest I've seen advertized has been about 1.50 a foot.
Suggestion - think about joining a yacht club that's a member of Yachting Clubs of America. You can then get reciprocity (space available) at other clubs at no/low cost. I'm a member of the Old Point Yacht Club at Fort Monroe (opyc.org), just $40.00 a year.
Check them out as an example.