I get the impression that you've never read the standard purchase agreement used with boats sold through brokers.
In the standard boat broker purchase agreement, the contract language is tremendously advantageous to the buyer. As I previously mentioned, the language allows the buyer to survey and sea-trial the vessel, after which the agreed purchase price may be accepted by the buyer, or it may be subject to renegotiation or rescission.
Yes, you can walk away unless the "acceptance date" has passed, less any expenses incurred by the broker or seller.
I don't see the standard agreement as particularly favorable to the buyer. The time is of the essence provision hurts the buyer, as does the warranty disclaimer at the end.