The J24 is a fine one-design, if you race one-design, but IMHO is not too good for much else. It certainly is a limited daysailer - the issue is that the boat is very uncomfortable for anyone except the helmsperson. There's very little flat deck and not much cockpit and even less cabin. And forget about launching from a trailer, a J24 requires a crane to get off a trailer.
What you should look at are the S2s, for example the sweet 22 footer.
S2 6.9 Sailboat "JollyMon"
They are very well made, pe4form well, and the models with the with lifting dagger boards can be trailer-launched. They have very people friendly cockpits and cabins. They are very competitive in PHRF and MORC fleets.
If you want to race in a J24 one-design fleet, definitely get a J24...for anything else, get something else.