Time of transit and close access to the LIRR being a consideration, you might want to look at:
Manhasset Bay: can walk from train, close to restaurants (by water and land) hardware stores, a local west marine.
Steppingstone Marina in Great Neck, town run marina
(allows non-residents for small prem) great park setting
with concerts sat/sun in the summer, launch service
5 am to 11-12 pm. right on sound facing City Island a mile across the sound.###disclosure...I'm partial, my marina.###
Both above are on the Port Washington line of the railroad, which means no Jamacia,about 30 minutes from Penn station.
Oyster bay further out but waterfront is close to railroad.
If you consider Manhasset Bay or Steppingstone let me
know I can show you around. Hugo