It's a weep hole. There are two seals in that pump: An outer one to keep water out of the engine and an inner one to keep oil in the engine. That weep hole is between the two. You need new seals. Btw: Don Moyer (Moyer Marine) recommends you put a bit of sealant around the seat of the "outside" seal before installing it, so any minor dings or imperfections in the seal's seat doesn't leave you with water still dribbling out (as happened to me).
And note carefully that the seals go in a certain way: With the stamped lettering on them facing one another.