What point of sail were you on with the jib only? I've sailed a fin keel (C&C 24) that did not handle well in higher winds with jib alone. This might not be as noticable in lighter winds, which might be why it wasn't a problem the first time you took it out (assuming lighter winds then). If you were on a beam reach or higher and the swing keel was only partially lowered it would have made the boat more prone to come off the wind in gusts, but even with it fully lowered I wouldn't be suprised if it was pretty squirrelly.
Cape Dory 28 "Dulcinea"