Put a 100 or 90 blade on that roller, and leave it there. Then look into a Code 0 sail for the light wind days. These thing can be configured to roll up on a flexible line, and pack into a bag like a big snake.
If you get caught with the Code 0 up, you can roll it in and roll out the blade, or vice versa. Of course since the 2 rollers are right up next too each other, you cannot tack the Code 0, but you can't have everything.
If this can be made to work on your rig, you won't have to drop the blade to switch, which can big a bigger job, particularly in a blow.
Yea, we all hate/love wind