We had three kids under the age of 4 when we bought our first boat (a 24 footer). The best advice I can offer to you is to take it slow for a while.
We would motor or sail downwind under
jib alone to a very nearby destination/anchorage -- just a couple miles, often not even that far. Then we'd
anchor up to let the kids play and nap. We would set up a large Rubbermaid bucket in the cockpit, filled with water, for the kids to play in with toys, etc. Bring along plenty of simple games and toys (the "make believe" kind). Kites are fun too
That first year, we did not go very far. The furthest we went was for an overnight in Annapolis (about 14 n.m each way), toward the end of the season when the kids were more acclimated and it wasn't so bloody hot for them in the
life vests.
Speaking of
life vests -- we have found that
Mustang (Lil Legends) seems to make the best fit for kids. But the REALLY little tikes have trouble getting used to ANY pfd.
We found things greatly improved in most respects when all of our kids were past the wobbly toddler stage, i.e. when they got to be pre-schoolers. Hang in there.
P.S. We have some good threads on this subject already. A quick search should pull them up. They contain a lot of useful nuggets...