I'm toying with that same question myself. We have a 10' Avon with the inflatable V-bottom, but we perfer a lightweight motor too. Not sure if your dinghy is inflatable or what it's rated for--ours is 10HP max.
All that said, we're thinking about the 6HP Tohatsu (or similar). The reason is that it's the same weight as the 4HP and 5HP models, and only 14lbs heavier than the 3.5HP, but with the extended range of external gas tank and more power to push through a chop.
I don't know if we'll actually GET a 6HP--we might end up with something used or new on sale. However, our thinking right now is that it might be worth going from 41 lbs to 55 lbs but almost double the power. Just my two cents for now, and of course I might change my mind!