You should be concerned. Over time this leads to a reduction in sail life and wear. My preference is to get rid of the topping lift and switch to a rigid vang, but this isn't always reasonable on a smaller boat. The next option is to go with a very lightweight line that will blow free of the sail when the topping lift is loosened while sailing. Lightweight dyneema is a good option since it will have plenty of strength even at very small sizes.