In the U.S. all you will need is proof of state registration. Nothing else.
Some folks try to avoid this and use their vessel's USCG documentation in lieu of state registration. It sounds good, but won't get you past the tax collector if he shows up at your dock.
As far as safety equipment goes, have one USCG-approved life jacket per passenger, at least two Type-B marine fire extinguishers, an air horn or other signaling device, and an emergency flare kit. You should also make sure that all the vessel's exterior lighting works properly.
Unlike an airplane, you are not required to conduct any annual inspections or maintain any fixed maintenance schedule. On a boat, you fix things when they break.
If you travel to the Bahamas, upon entering you must purchase a cruising permit, which is good for 12 months. This permit also includes a fishing license. From Noonsite:
"Entry fees must be paid by all visiting yachts and are as follows:
- $150 for all vessels under 35 feet
- $300 for vessels over 35 feet"
You may keep your vessel in the Bahamas up to three years, but must pay a $500 extension fee at the end of years 1 and 2 in order to remain
Finally, although no license is required for you to operate a boat, I strongly recommend that you sign up for any US Power Squadron or USCG Auxiliary courses offered in your area. They won't teach you how to sail, but they will help teach you the "rules of the road".