The final diagnosis
However, the beast is unable to fully lift the anchor plus 4-5 feet of chain without the clutch(?) slipping; the motor continues to turn, but the drum stops, or loses ground. I can get it in by providing probably 40# of muscle.
OK, answering my own question.
HyLyte came pretty close with the keyway comment.
I took the capstan apart last week; it took a heavy bar and a lot of pounding on it with a hammer to loosen the top nut (this model takes a 3/8 x 1 1/2 inch bar, the one I had was smaller). This top nut is the chromed piece you can see from the top of the capstan. It sits atop a drum intended for rope, which sits atop a bronze (or brass??) clutch plate that sits on the gipsy. The whole thing is held in place with a 1 1/2 inch drive pin with a keyway down its length.
The gypsy and the clutch plate are not caught by the key spline, so they can freewheel. The rope drum seems to be the only piece that is captured by the key spline, so it does not freewheel, and the top nut is obviously captured by the threads at the top of the drive pin.
I initially removed and cleaned the whole apparatus. The clutch plate was both greasy and covered with friction glaze. I sanded off the glaze, and while reassembling did not grease the clutch, just as I don't grease my car clutch or brakes. When reinstalled, though, the clutch was looser than ever, probably because I did not get the top nut as tight. Grrr.
I took it apart again, and noticed that the top of the key spline was proud of the rope drum by about 1 mm, and had a couple of grooves in it. Reasoning that this key spline was keeping the top nut from fully engaging the drum (and thereby the clutch plate) I took it apart once again and used my Dremel (gotta have one on board!) to cut about 4 mm of the key spline off of the top. When reinstalled, the spline now sits about 3 mm lower than the top of the rope drum (still with about 5 inches of spline engaged), and the top nut can spin down to fully engage the rope drum, and thereby the clutch plate.
Sorry I didn't get any pictures.
I tightened the top nut with the bar as tight as I could by hand, and it was still too loose. It took 2-3 swift blows with the hammer on a 10" bar and now the gypsy can lift the anchor into the rack even if it has to spin around 180 degrees.
This is a 20 year old Maxwell, and its assembly does not match that of the diagrams now on the Maxwell site. It seems in great shape though, and will certainly give another decade of service. Likely the clutch slipped and wore itself sufficiently so that the spline caused the nut to hold up.