I think I would try to stretch the budget a bit and look for a 30 something footer for your needs. Some boats that come to mind for you to consider are the Catalina 30, Sabre 28 and Oday 322. The Sabre would be the best sailor of the three but the trade off is space and I think 4 adults would be problematic on an overnight. Our first boat was a O'day 322 and I think its a boat that can meet your needs if you can find a good one.
The C30 is one of the most popular boats in its size range and I think it offers a lot of what you are looking for, though its a long way from a Sabre 28 in terms of aesthetics.
I took sailing lessons on a Tanzer 22 and while they are a well regarded boat, I think you'd quickly find yourself limited in what you could do based on your stated use. The only time a boat feels too big is close to the dock and you get over that pretty quick. Since you already know the basics of sailing, my advise would be to buy the size boat that meets you needs and get some instruction on that boat until you are comfortable manuvering around the docks and then just go for it.
s/v Palmetto Moon
1991 Catalina 36