If you don't keep score, it's hard to measure progress.
If you enjoyed sailing, and then racing then by all means continue it, and don't be talked out of it.
As noted above, the discipline of following a course at a given starting time will challenge your skills, and provide feedback on your progress. Taking care to know the rules, and prioritizing safety ahead of speed is appropriate, and you can track your progress as you pass boats that you used to follow.
Learning to handle you boat in close quarters on a starting line, rounding marks and crossing others is an excellent way to learn for when you may need to recover something overboard, clear an obstruction or just get home before the bar closes.
Knowing how to point and foot and when to do both, to tack effectively and get best boat speed for your conditions, may be merely pleasing for mastery of a skill, or it could be important in a future time to get you to safety ahead of the weather.
Given a chance, crew with others, and take both newbies and experienced crew along, you can learn a lot from different opinions and questions both asked and answered.
We race a 40' 20,000lb 1962 Yawl on the Oakland/Alameda Estuary, where the courses can look like this:
We struggle to stay ahead of the Santana 22's in light air, they can tack faster and point higher, but we have a mast that gets above the shoreline obstructions and can occasionally ghost past them.