We've gone over to Catalina in December several times. Weather is about the only issue, as some years we have had 80 degree days and some trips get cold and wet. But the island is great, and the crowds are gone.
We always head over to Catalina (Cat) Harbor as it is the only really sheltered harbor on the island. That way, if something comes up, at least you have some shelter. The down side of going to Cat Harbor is you have to walk about a quarter mile to the little community and there are not as many services . So if you are dependent on a shore boat, there is going to be a wait. But the trade off is you have the place pretty much to yourselves.