That's my home turf
First of all, get yourself the Embassy Guide for Long Island Sound, put out by Maptech. It's got tons and tons of information on everything about Long Island Sound, including harbor charts, recommendations on navigation, anchoring, dockage, places to go, things to see, where to eat and provision, etc.
Also get the Eldridge Tide Book. It's great for Hell Gate, and it's helpful for many other areas on Long Island Sound for currents. The reality is that if you are going to stay in western LIS, rather than the eastern end, current is not a big deal once you get through Hell Gate.
In terms of harbors that I would recommend, here are my thoughts, somewhat west to east:
1. Port Washington (Manhassett Bay). It's a very easy harbor to enter, and it's got a little bit of everything. You can anchor in protection (very good holding ground), you can pick up a mooring, or you can go dockside. Ashore, there are good restaurants (Louie's is a local favorite and a virtual landmark; it was featured in Meet the Parents), some shopping, and it's generally a nice town. You also can get fuel here.
2. City Island. This is an eclectic spot that struggles with whether it's part of the Bronx (it is) or is an island on Long Island Sound (it's that too). There are tons of restuarants of all sorts, everything from 4 star to sub-McDonald's. There's some marine supplies, funky shopping (not chic), a good ice cream parlor (Lickety Splits), and some cool stuff to see. It's not the best place to stay overnight if there's any weather. The harbor is exposed, and though you easily can take a mooring, you will roll and won't be uncomfortable. We go to City Island for day trips; tie to a mooring for a few hours, spend the day wandering around, and then head across the sound to Port Washington, about 4 miles away or so.
3. New Rochelle. This is a bit of a tough spot to get into, and depending on what you're into may not be worth it. There is no anchorage to speak of, and if you go you most likely will need to be dockside at one of the local marinas. There actually are two harbors (New Rochelle Harbor and Echo Bay), both of which require some attention to enter. I mention NewRoc, as it's known, because a few years ago they built something called NewRoc City, which essentially is an entertainment mall. It has everything from movies, to an arcade, to ice skating to restaurants. It's a bit of a long walk from the water, or a very short cab ride. If you have youngish kids, give this some thought, as it will be a good diversion for them. If not, you might want to take a pass. If you do go, and you go to New Rochelle Harbor, you can get fuel very easily.
4. Mamaroneck. We actually live here, and in my opinion the town does not get enough recognition as a place to visit by boat. All right at the head of the harbor you have restaurants, bars, a movie theater, shopping, supermarkets (a little further of a walk), marine supplies, ice cream parlor, beauty salons, and even a Shiatsu-Spa! The downside is that it's not the easiest harbor to find a place to stay. The town maintains a floating dock float in the harbor for guests (it's not tied to shore; it's like a mooring in the harbor, but it's a floating dock tied to two pilings). There is anchoring in the outer harbor, but that won't be so comfortable if there is any wind out of the south. There also is transient dockage at the Brewer's Marina.
5. Oyster Bay. This is a great harbor in a lot of respects. It's pretty, home to many celebs, very easy to enter, excellent anchoring in a number of different places, moorings available, and walking distance to town (where there are some restaurants and such). Most people go to Oyster Bay to enjoy the harbor and water itself. The fuel dock at Oyster Bay Marine Center generally is considered the best place to get fuel in the area; a huge floating dock, with a number of attendents to help, and it pumps high volume so you get good quality fuel.
6. Port Jefferson. Another harbor that's easy to enter with easy anchoring, moorings and dockage. The town is right at the head of the harbor, and it has everything you might need or want (save for repairs). It also has two very secluded places to stay where the scenery is great. Just to the east as you enter (to your port), there is something called Pirate's Cove, or sand city, depending on who you ask. It used to be a sand quarry. There are private moorings in there. There is no cost, and the rule is that you can pick one up, but if the owner shows up you might have to leave (in all our years of going there we've never had to leave). You then can go ashore and climb the bluffs for breathtaking views. Opposite Pirate's Cove, to the west after you enter the harbor (to your starboard), there is another area just as beautiful (I actually forget what it's called), but there are no moorings and you anchor.
7. Milford. There is no anchoring or mooring as a practical matter. You can anchor behind Charles Island, just outside the harbor, but it's not very comfortable in any wind at all. If you go to Milford, you go dockside. There are a number of places that are very nice, and they all are walking/dinking distance to town.
Those pretty much are the highlights for western Long Island Sound. There's much more to the east, and there's even more in the western Sound that I haven't covered. I strongly urge you to get that Embassy Guide, as it will tell you everything about every harbor. It's a great place to cruise because there are so many places to go, all within a day sail of each other.
If you have any questions or want some more local knowledge, feel free to PM me.