I've never been on the south side of Manitoulin, but from the maps, it does not look like many sheltered bays to
anchor, which would result in sailing through, and a little tough to do single handed. It looks like it's exposed to West, SW and South winds ( all prevailing ) which would pound all night at
anchor.
If you haven't been to Baie Fine, then it's well worth the time spent getting there and staying for a day or two. Several ridges to hike once you're there. Beyond that, theres all of the North Chanel and lots more to explore and sail. Rather than trying to make a route out and around, make it the destination and spend the time to explore the area.
