You may not want to hear this, but I don't think you are on the right track, because you are looking for too big and complicated a boat for your first boat. Lots of people make this mistake and end up not using the boat as they intended (or worse), simply because they lack experience in handling and maintaining such a large boat. You mention refitting.... you need to realize that even if you pay others to do the refitting, you will still need to make many decisions about what to fix or change, how to do it, what yard to use, who to hire to do it, how to monitor their work, whether or not they are delivering for the money you are paying them, etc.
My recommendation would be to start out with something 35 feet or less that is maybe 10 - 15 years old and well maintained, as judged by the best surveyor you can find. Then enjoy that boat while you become an experienced sailor, cruiser and boatowner. There is a lot to said for coastal cruising which is what you would be doing with a smaller boat. And if you are near the Caribbean, a lot of boats in this category would be very capable of cruising there.
Spend maybe $30-40k on this first boat and sock away the rest of your money until it's time to get the dream boat.
Learn how to maintain and fix the boat yourself. Get one with a good diesel engine and learn how to maintain and fix that engine. If you ever do long distance voyaging, you will have no one to rely upon but yourself to fix things when you are thousands of miles from assistance.
Then after a few years of this you will be better prepared to decide whether to buy that $160k boat to take you over the horizon.