Sounds like classic prop-walk. The key is to plan ahead for it. If you know that the stern is going to kick Starboard on the way out, then you have to get creative on how you come in to the slip in the first place, or use warps to get the stern where you want to go. A diagram would help us all brainstorm your best approach.
Why not back into your slip and come out forwards? That way you'd have plenty of prop-wash over the rudder to get you turned in the right direction on the way out.
If you plan for it, you can think of the prop-walk as a feature, not a bug. It's like having a 90 degree stern thruster. It makes you crazy maneuverable at low speeds as long as you plan to only want your stern to go to starboard! For example, you can spin your boat around in its own length counter-clockwise with the wheel hard to port and using short bursts of forward and reverse, but you can't turn it around the other way because of the propwalk. So just always go counterclockwise.