I had to do the same thing last season for a pump out fitting. I had used the wrong hole saw, at least I used one too small rather than too large. I ended up using a rotary rasp on a power drill.
Going very slow and keeping the drill at a right angle to the work suface allowed me to enlarge the hole in just few mins. Make sure you have drawn a very clear border to work out to, stop just shy of the line then finish with a small sanding drum or sandpaper wrapped around a dowel. Again, keeping the tools at a 90 degree angle to the hull will be key for this project.