Hampton/Norfolk and Portsmouth all have done away with personal property tax. The main thing is to try to find someplace with easy access to the Bay as inside the tunnel crossing is pretty heavily traffiked. Here's a
link to info on the downtown Hampton marinas. You might also consider the norfork side as willoughby bay and little creek have lots of marina slips and quick acccess to the bay. One of my favorite marinas is
Salt Ponds which is right on the Bay in the Hampton area but there has been a lot of shoaling in the entrance channel (and dredging). Don't know what you draw, but it is definitely a nice marina and worth checking out.
As the the "will I enjoy it" question. The area and the rest of the bay offer a lot more to do and see and more open sailing than Oriental (nice as it is!). We cruised the bay for 20+ years and still had lots of spots we missed! The only real downside to the Chesapeake from my perspective is the stinging nettles which make it impossible to jump off the boat and go swimming for most of the summer in most places. Marinas with swimming pools are highly recommended!