I would take SD's math on the doubling fairly seriously. Unless your post is saying that money is NO object, then what the hell, go for it! If in fact you want to stay within a budget (OK, this is a boat, Close to a budget!) you should look at some fixed costs by size FIRST. Then decide what you want. For your stated purposes, a 54 seems a little bit of over kill, unless you want to impress the folks with your size.
Insurance. Would be interesting on a +20 50 footer
Slip Fees. This is an eye opener if you live in a population center! Goes up very fast in lenght....if you can find one. If you have your own empty pier, no problem. However, if you have that pier you are looking at too old of a boat for your front yard.
How handy are you? Professionals are going to cost you $75-100 an hour in most yards. West Marine however, will love you.
Hauling out. For a boat that size...not that many places that can handle her.
The list can go on.