Nick - I have made very similar trips but leaving mostly from further east on the LI side of the Sound so I am not too familiar with Stamford. We usually do the opposite and head out to Block stopping at ports on the LI side and then on the way back stop along the CT coast. However last year on our way back from Newport and Block Island we stopped in Stonington, CT which is in Fisher's Island Sound and is a very nice little sailing harbor with some very impressive boats in it and is an easy sail to or from Block. You also have Mystic right there as well which is a fun little town with the whaling village and all of that to do. The next night we stopped in Clinton and would prob not go back because we were the only sailboat out of about 300 powerboats. The cruising guide advertised it as being a boaters heaven... it did have a pool and hot tub and a kind of outdated restaurant but that is it so personally I would skip Clinton. Other places in CT that is within a half day sail from Port Washington that we liked is Milford which is very clean, pretty town with plenty of restaurants right off the dock, Norwalk, and Captains Cove Marina in Black Rock which has more of a party atmosphere.
The trip is plenty doable and you should have a great time! I would get a LI Sound Cruising guide so that way if you feel like making a longer run one day or stopping short, you can look up all the places to stop in as there are plenty. My gf and I will be headed east in August and are going to try some new spots in CT on the way back. Have fun!