I've never seen a rope clutch used on a mainsheet, I cant' say why not, nor quite provide an explanation for why not, but I would not use a clutch. The mainsheet is the one line on a boat most often played, ie.e repeatedly eased and trimmed, and a rope clutch is not good gear for such an application.
As to size, given the above statement I'd generally go with as large a size as cleanly fits in the sheaves. If the 1/2 fits cleanly I'd use that size again. While the 3/8 would probably be fine to carry the load, the crew's hands will like the 1/2 a lot more...
I use a rope clutch for my mainsheet. It works out well with my deck layout, where all lines are led to the cockpit. I use Lewmar D2 clutches, and they release very nicely under load. Original sheet blocks were old and the loads were too heavy for my wife to ease the line, but she can do it nicely with the clutch. With new sheet blocks the loads are much lower, but the clutch still works fine. I would rather have an end boom traveler with a gross and fine adjustment lines for the mainsheet, but the current set-up works well for us.
For the size boat huguley3 is talking about, 3/8 line should be more than adequate, although it will depend on the forces on the line, which will depend on the number of blocks and mechanical advantage on the sheet. If the forces are high, a thicker line such as 7/16 is a little easier on the hands.