If rotating the battery switches causes the failure to be intermittent, suspect a bad switch. I would not stop there, however. You should thoroughly check every wire and connection between your batteries and the engine starter motor. Make sure everything is clean, free of corrosion, and all connections are tight.
Don''t delay in resolving this or you will likely regret it soon. Good luck.