I got back two days ago from my first BVI charter, and I had many of the same reservations the OP has. The short answer: Just go, it's really quite easy sailing, the islands are close by and there is lots of infrastructure (mooring balls, charter company help, other boats) to help you if you get into a tight squeeze. You seem to be better prepared than most, and you know what to expect having been there before!
The longer answer, touching on some of the topics here in the thread:
1) The islands are really tightly grouped together. From Road Town, the tech support boats can be at any of the islands (except Anegada) within one hour. That's a real safety net, should you need it (but you most likely won't).
2) Moorings - we found the moorings to be in good shape, generally. Some had the floater for the eye hook missing, and picking up the line was more of a challenge, but in general they were fine. Granted, it's low season and we had lots of choice. Picking a mooring up is straightforward, it's all about giving the 'hooker' time to do their bit, and that means stopping the boat right next to the mooring ball and keeping it there. Easier said than done initially, but after some practice, not hard. There were days we picked up three moorings, so you do get to practice.
3) The wind blows steady and pretty hard sometimes. We didn't see more than 20kts, but that is not an insignificant amount of wind - I'd recommend being comfortable reefing, and reefing early to be conservative and enjoy the ride.
4) Mechanical failures - we had a failure in our standing rigging (!), which prevented us from sailing for a day. Our charter company came out and fixed it in about 30 minutes. It's nice to be close to base.
5) Line of sight, charts and grounding - Getting familiar with the chartplotter / GPS and your own charts is important - but it's really not hard to navigate the islands. Running aground if you are even just a little careful should be avoidable entirely. It seems like OP is a very careful person, and you won't have trouble with this.
6) Anegada - no one has mentioned it, but it's so totally worth it if you like beaches and peace and quiet
A little more tricky getting into the harbor, but not terribly. And worth it. Oh, and beautiful.
7) We had other issues, a blown bilge pump, a lost (and recovered) dinghy, burned out anchor light. You know what - it's all part of the adventure. Just go with a good charter company that can help you out of a tight squeeze.
8) Other sailors - everyone out there was incredibly helpful, whether helping us look for our dinghy (long story) or giving us navigation tips, etc. You won't be alone!
Hope this all helps - have a blast!