It sounds to me like they have a case.
I assume they have a signed work order from you. You picked up the boat at the marina and sailed away. Sounds like you accepted the work as being complete to do that.
You didn't say if Doral was straightening the prop shaft at their cost or yours, or if they were paying any of the yard bill at your marina. You didn't mention any additional charges for this work so it sounds like they are trying to make good on the job. Let's face it, it would be very easy for them to say the other marina bent the shaft when they hauled your boat, but it doesn't sound like they're disputing this at all.
It's standard practice in every repair industry I've ever seen not to return property until the bill is paid. I think it's covered by Mechanic's Liens statutes pretty much everywhere. Personally I'm surprised you have the boat at all.
As for the delay in getting work done, did you call them every day
after the promised dates and ask for the updated delivery date? If you didn't, I think it might be a challenge for you to convince a Judge you were suffering from loss of use of the boat. Doral could easily say you couldn't have suffered too much since you couldn't even be bothered to call them.
You can get info on Ontario Small Claims Courts at Ministry of the Attorney General - Small Claims Court
The limit for suits is $10,000.
Bear in mind that the civil court system is different between Canada and the U.S. In Canada you can sue for punitive damages, but you will only get them if you prove
malice. "Emotional distress" won't get you anything. You need to document actual damages, i.e., bring receipts to get any money. Also, Canada uses the "loser pays" system, so if you lose, you pay your lawyer and
your opponent's lawyer.
So for a counter suit, I don't know what you could reasonably ask for.
I'd suggest talking to a lawyer about your options. I think it will come down to you're going to pay one way or another. Negotiating a deal with Doral will probably be the fastest, least stressful and cheapest way to get back to sailing.
If I might say, I'm also quite puzzled by this story. As skipper, you're totally
responsible for the safety and security of your boat and (admittedly "rookie") crew. A new engine is major surgery on a boat. Presumably the forecast the day you headed out onto Georgian Bay called for light winds, and even if it didn't being becalmed is not an unreasonable expectation for a sailor. Why did you wait until you were in the middle of Georgian Bay to test the new engine? There are no end of stories of sailors getting into trouble because they were trying to sail according to a schedule instead of sailing when the equipment and weather was ready.
Anyway, good luck with this.