I fail to see WHAT the issue is. You have a written contract with the owner. You have the right to your own surveyor. The boat broker does not have to work with anyone. He or the owner must simply make the boat available for survey as per the contract. Simply advise him in writing of your intent to have the boat surveyed on "X" date with a copy to the owner and ask him to advise you if there are any problems in having the boat available/ready for inspection on that date.
I am looking at a standard broker sales agreement as I write this and it simply says:
The buyer may have the yacht surveyed at his expense.....the seller agrees that the buyer or his agents may examine the yacht and inventory in a non-destructive manner...the seller may stipulate the boatyard he is willing to have the yacht hauled for survey. The seller agrees that ....delivery to the boatyard is at sellers sole risk and expense. The buyer agrees that the SURVEYOR SHALL BE EMPLOYED BY THE BUYER AND THE BROKERS ARE NOT RESPONSIBLE FOR.....blah,blahblah.
Furthermore...the seller's failure to deliver the yacht in accordance with the terms of the contract usually lets you use the arbitration clause of the contract.
I would put the pressure on the broker and do so in writing with a copy to the owner....and YES...I would be most concerned about the true condition of this boat. Maybe CardiacPaul can shed some additional light on this situation from a surveyors perspective. You might want to PM him.