boat is probably too heavy for a single banger, also agree with lash comment above
The Mercury engines under 30 hp are made by Tohatsu in Japan and I compared my 5hp Merc 4-stroke to a 5hp Tohatsu 4-stroke last summer and they are indeed identical. I believe the Nissan is also the same. My hull weighs approx 2400 lbs loaded and the 5hp is very adequate and hits hull speed. My son borrowed the motor while his Honda 8hp was being rebuilt on his 30' Catalina and it did not reach full rpms with the 5hp Tohatsu. It clearly seemed underpowered on that hull.
According to Mercury the difference in the 4/5/6hp engines is not just jetting, the carb itself is constructed differently on the inside. The props can also be slightly different. I have been toying with the idea of buying a 6hp carb just to try it out.
I think you have a case of too much boat for a 1-cylinder engine. When you're talking displacement hull such as a heavy sailboat the number of cylinders makes more of a difference than hp which also helps to explain why a 4hp 2-cylinder performs better than a higher hp 1-cylinder. My Mercury 5hp vibrates considerably more than the old 8hp 2-stroke Tohatsu I had. Mercury and the dealer both say it is operating normally. I've gotten used to it. I would say it is quieter than a 2-stroke though.
You might check the valve lash as the poster suggested above. This could definitely affect performance. If your boat is 3000+ lbs plus gear I would say that that is more likely your problem and you need to upgrade to a 2-cylinder engine either 2-stroke or 4-stroke. From your original post I thought your speed over ground difference would have been greater ... I don't think you're that far off and I'd bet that a 2-cylinder engine will be night and day difference.