I recently had a similar experience. I just bought a boat that, during the survey, had very high moisture readings on both sides of the deck. Also one area of delamination near the foredeck. In my case, the broker was worried. We agreed to have a yard send someone to look at it and provide an estimate. We re-negotiated the price based on the estimate the yard provided.
I felt comfortable going ahead because despite the high moisture readings, both the surveyor and the guy from the yard felt that the deck was solid. The guy from the yard also felt that the delamination was not serious and said it probably wasn't even necessary to repair it (I probably will eventually anyway).
In my case I came away feeling that I got a very nice boat for a very reasonable price. I would say that if you like the boat and the rest of it looks good to you, have it surveyed.