SB Island is a nice place to visit, provided you are good at anchoring. The only place to anchor is at Landing Cove, below the Ranger Station. Just keep an eye on the weather, and if a Santa Ana fills in, bug out as there is not a lot of protection. The bigger issue will be fog as there seems to be a marine layer that has come in. And there is an active shipping channel between SB Island and Catalina in addition to the shipping lane that is along the coast. However, it is my favorite of the Channel Islands group.
Isthmus Cove is also good, but depends on the weekend. If you are going on Memorial Day weekend, give it up as it will be packed. Cat harbor is a good back up as well.