I have the 6HP Tohatsu for a 10'4" Achilles air floor. With one adult and a pair of kids, the 6HP will get the Achilles up on plane. With two adults, no planing. Getting the OB on and off the mother ship is interesting but do-able for one person. I always end up with a little bit of grease on my shoulder after doing it. The inflatable will take up to a 10HP, but there's no way I'd lug that sucker around except by car. If you don't have any need to entertain the kids by zipping around the harbor, then smaller might be the way to go.
Yeah, the weight of the 3.5 is appealing, and no, we don't need to entertain. But we will almost always have at least two adults in the dinghy and sometimes three. So we just want to make sure that if we do want to go explore a little further away that we'll have range and also the power to cut through a chop and into the wind if needed to get back to the boat.
I say all this without any personal experience on modern outboards. I'm still trying to figure out what the "sweet spot" is that will meet our needs.