I would not be comfortable using the fill in plug as a permanent solution. I don't think they are designed for that.
I concur. The "plug" that comes with most of these depth/speed instruments is meant to be temporary, and usually made of plastic and hollow, typically much less substantial than the speedo/depth instrument itself. I would not be comfortable using this as a permanent solution, or even trying to get by for a season as a temporary solution. If it fails, that leaves a pretty large hole at usually the deepest part of your hull where the water pressure is highest.
The best solution, as mentioned before, is to either find another speedo (the hole size should be pretty standard, and you can usually find a replacement) or remove the rest and glass/epoxy it closed. Alternatively, you can put an entirely new system in, even if the new hole is a little larger, if you you would like the instruments.