My idea with coming out a bit quicker was that there would be less time for the bow to be blown toward the dock. Kind of like when flying, a 20 knot crosswind is less stressful when you're coming down final at 120 than it is when you fly final at 60.
Increased way on (speed) will give the hull more purchase and thereby make the wind less effective. I think I miss interpreted your OP. If your stern walks to Port in reverse, more throttle would have helped as it would have slowed the bow's movement to Port. My boat walks to Starboard.
Regarding which way you "need" to go:
I often have complete novices back my boat out (and bring her in for that matter). Before leaving, I explain that several variable conditions have different effects on sail boats under power. "We're going to ask her to back to Port (to point the bow to Starboard, toward the channel). If she doesn't want to go that way, we will work with what she wants to do to get us where we want to go."
Usually she cooperates. Sometimes, we end up backing through a 180 to Starboard. Sometimes, she refuses to back to Port at all, and doesn't really want to do a 180, but backs down the fairway (toward the channel) like she's on rails!
On a sailboat, it's charming. If my truck did that, one of us would need to go in for serious help!