You wouldn't have a problem sailing over there with decent weather. My friend just recently rode a Seadoo from Avalon to Marina Del Rey so a 17' boat is completely doable.
The problem you may find is getting a mooring in Avalon. Especially around 4th of July. Avalon is a madhouse around that time of year and if you don't know someone who owns a mooring, there's a good chance you may get bumped off a mooring and have to leave. Also, the smallest moorings are 30' so you'd have to extend lines from your cleats (bow & stern) to the hawsers or use the bow hawser and tie off the stern cleat on the spreader line. Sounds confusing but you can look up Catalina moorings online for a diagram.
The anchorage at Avalon is another problem. You have to anchor in 80-100' of water. I know I don't have enough rode to anchor there.
My advice is if you want to go to Avalon (which is a beautiful place), go mid-week either early or late in their busy season so you can practice catching a mooring and learning the lay of the land. People constantly, I mean constantly screw up in that harbor but if you are careful and do a little research, it's not that bad. One perk of going to Avalon (or Isthmus Cove) is that if you don't have a dinghy, you can call for a shore boat to pick you up at your mooring.