As you say, it depends on the circumstances of the moment. Ghosting along at 2 knots in driving rain is not our cup or tea.. motoring and steering from under the dodger with Otto remote's the way to go. But doing the same on a pleasant warm day with not too far to go is a different thing altogether.
We usually time this sort of thing with speed vs. time/distance/daylight remaining to our destination in mind, so the threshold can vary considerably.
1984 Fast/Nicholson 345 "FastForward"
".. there is much you could do at sea with common sense.. and very little you could do without it.."
Capt G E Ericson (from "The Cruel Sea" by Nicholas Monsarrat)