You can do it!
You can definitely do it.
However you should put some thought into getting the right boat, the right trailer, and developing a routine at the dock. I learned to launch a J/24 mostly by myself, but ultimately it was too much work. You don't want a fixed keel or keel stepped mast. I have a 14' hobie cat which is an order of magnitude easier to rig and launch. The J/24 took me about 4 hours to rig and launch, the Hobie takes about 15 minutes.
Actually, I'd suggest a cat as one possibility for you. You can go fast, and they don't weigh much - so you can pull it completely up on the beach yourself if needed. You're more likely to get wet however. I paid $1000 for a used Hobie with trailer and sails.