Would suggest that you pick up a copy of the Guide to Cruising the Chesapeake Bay (spiral bound - published by Ches. Bay Magazine). It has lots of good info for cruisers including detailed info on entrances to creeks and shoreside stops. If you want detailed info on anchorages there is Shellenbergers Cruising the Chesapeake: A Gunkholer's Guide. Both are wonderful helps to cruise planning. We keep a copy of the spiral bound one on the boat and take Shellenberger whenever we are out for longer than a couple of days.
As far as your stated destinations, I agree with whoever said skip the easy overnighters and go farther south. If this is your first time cruising, don't be too ambitious but there are as stated, a lot of good destinations that are easily reachable.
As far as local info. Worton has a nice anchorage, some good marinas and a decent restaurant. I don't think they let you overnight at the docks at Betterton but there is a nice anchorage in Turner's Creek. Georgetown is a long motor. Rock Hall is really 2 spots - a nice anchorage (and good marinas) in Swan Creek and a nice harbor around the bend.
Rhodes 22 Fretless
Rock Hall, MD