I know it's easy to find nice bay beaches in the Peconic by car, but I'm new to sailing and I'm looking for a nice quite beech where I can anchor my 22' Catalina (30" draft) and do some swimming and\or fishing.
How about off the north side of Robins Island?
Off Jessups Neck?
The long beach by Pine Neck in Noyac Bay?
A little farther afield would be Northwest Harbor. You just need to watch the tides carefully rounding North Haven or the north channel around Shelter Island.
I am more familiar with the South Fork but there have got to be some pretty cool spots on the North Fork and Shelter Island for anchoring and swimming as well.
That may be problematic, since you are not allowed within 1000 yards of any state beaches (to keep boats away from swimmers) and that may also apply to county and town lands/parks. Then there's the private land, and property owners aren't going to let you ashore at many of them.
I wouldn't call it impossible--but I'd call it unlikely. Look for a popular and accepted anchorage, within reach of some public shoreline.
Folks on the East End tend to be a wee bit territorial about their waterlines.
I can't contest hellosailor about East End folks being territorial - I know they can be to the point of absurdity.
Robbins Island is privately owned so perhaps not the best place for making a beach landing. I don't see why you can't anchor and fish there or swim off the boat.
Jessup's Neck is a state park and sticks out almost 2 miles into the bay and is surrounded by sandy beach on both sides. It is a long hike out to the point from the road so you would likely not see many people out there at all. It looks to be about 7 nm. from Mattituck, a little longer if you need to get to the NE side of the neck to dodge the usual SW winds.
Remember, they have to catch you first so have fun exploring. You can always keep your VHF tuned to 16 to monitor if anyone is ratting on you.