Welcome to leaking chainplates.
It never ends.
I just loosen them in turn, and re-caulk.... a messy, sticky process.
Loosen the bolts but do not take the nuts off. Prise the chainplate and the backing plate off the sealant. Take one nut off, and drift one bolt out. Tie a stout piece of string through the hole in the chainplate or backing plate (whichever one is outboard). The string will prevent the plate from dropping into the deeps if you drop it.