Yes, you should move the connections from the Echo Charge and the Inverter/Charger to connect directly to the batteries without a switch.
Also, I assume that your '87 Catalina has a Universal M25 engine. If so, that 220A fuse on both battery circuits, is WAY too big.
Figure the highest load that you expect to see on the house bank (every light, fan and appliance turned on - say ~50A), and double it. That should be the size of your primary fuse on the house bank.
I have the same engine (different boat, and no inverter) and frequently start my engine on a 90A MRBF. I have a 135A MRBF on the starting circuit, but forget to put the switch in the correct position
. I have yet to blow the fuse on either circuit.