Not meaning to be flip, but if you aren't familiar with the potential pros and cons, "yes". On the other hand, if you (as a family) come up the learning curve and at least the adults develop some skills as sailors, I'd shout a resounding "NO!"
You don't find many drug dealers, junkies, drive-by shootings, computer porn or hackers, schoolyard bullies, hit-n-runs, home invasions, or many of the daily stresses and dangers of typical "land living" at sea. You are in a much smaller and sometimes more controllable environment, for much of the time. And that which you can't control--like the weather--you can reasonably predict and prepare for.
Each port will be different, and yes, there are still real pirates in known areas and other dangers which you probably would be safest to avoid. But on the whole? There are a great many things kids can learn sailing, among them responsibility, confidence, self-reliance, teamwork and trust.
Adventuring...implies some element of chance and risk. It can be a great adventure, but for this to be SAFE, both you and your husband should have the skills, training, experience to be able to handle a boat offshore by yourself for a weekend without feeling apprehension--even in reasonably bad weather. Spouses *often* gain that experience and confidence best when they can gain it without the other half being around, i.e. by taking a "Womanship" type sailing course, so the pressures of even the best husband simply aren't part of the learning process. (And besides, that means he'll have to practice dealing with "family stuff" while you're away "on the helm" and can't be disturbed.)
Someone once asked me if I wasn't afraid of the sea and I said no. Not afraid, because the sea doesn't have any reason to come after me. It's not the boogie man. On the other hand, I respect it greatly. I know it can and will take me--or anything else it wants--anytime it wants. So I try to be very careful around it, and make sure I don't give it an easy way to take me.
But some folks are outright afraid of it, sometimes for good reason, and for those folks, it will just never be a comfortable place to be.