Good luck with your dream!
If building a boat means a lot to you then by all means go for it. If the dream is more sailing related I would strongly recommend skipping that step. Small boats are very affordable. I own a 22' swing keel trailer sailor, and I am pretty close to dirt poor. I could not have built half the boat I have for the same money, and that is just counting materials not time. It will also take lots of time to build a proper boat. Either way, have fun pursuing what you want. I've had a blast learning to sail in the last year and a half, amd am enjoying making all sorts of crazy plans for the future.