Hmm. It's normally a heavy enough board to stay down -- about 15 lbs out of the boat. It's gelcoat & FRP over (IIRC) a putty core. I'd wager you have either (1) an aftermarket board with plywood or foam core; (2) air in the tip of the board; (3) or just a really loose pivot.
Water often gets into these boards via the pivot hole, drains to the bottom, and splits it. The core can fall out, leaving an air void. Poke your head underneath & see if your board has any visible damage. You might also see the board resting on a trailer crossmember, a common issue with Buccs and one that can notch the leading edge & result in core crumble.
It's very easy to remove the CB cap. If you haven't done it yet, you should.

Drill out the rivets and remove a screw or three, and you can inspect the trunk plywood for rot (big problem on Buccs), beef up the mainsheet padeye, and see if some nice person has installed top-hangers for your CB. These are much better than the original bottom-access pivot pin. At any rate, try to see how much circular play your CB has, and whether a too-loose pivot might be causing your CB to float up.
This is important for other reasons. Slop in the pivot makes the foil twist off upwind, causing it to stall and killing performance. I rebuilt our CB, took out all slop, and the boat is much better to sail. Also your CB could fall out, which sucks.
BTW, as long as you have that cap off, it's nice to epoxy some angle aluminum along the trunk top to reinforce the plywood and isolate its edges from water. That trunk was built by concussed monkeys.
Finally, you could just bungee the sucker down. Even snugged up on our boat, it tends to lift on reaches above 10 kts. Of course it should be up halfway or more by then, anyhow.
You'd get lots of advice at the
Bucc18 Yahoo! group. Nice pipples over there.