Yes, you do need to have the patience to keep the rudder amidships until you have steerage way on - putting the rudder hard over early acts like a brake and amplifies the propwalk effect.
You don't say what kind of (37'?) boat it is, but there are many popular designs out there that simply will not back up in a straight line no matter what you do.. if you have one of those then you're stuck with the kind of situation you had just now.
You may have been better off to try and spin the boat around as soon as you could - but presumably there was no room to do that.
1984 Fast/Nicholson 345 "FastForward"
".. there is much you could do at sea with common sense.. and very little you could do without it.."
Capt G E Ericson (from "The Cruel Sea" by Nicholas Monsarrat)