The wife and I have spent years living, working and sailing in the USVI (and Puerto Rican islands, ie Culebra) both ashore and on our sailboat and as previously stated there are issues with crime, corruption, lack of services and the cost of goods other than booze.
The crime and corruption while not as bad as some down-island places (Trinidad, Venezuela?) is at a level that it really can't be compared to any stateside city. I make this comparison after moving to St Thomas from previous stints in East LA and Oakland, keep your eyes open and your wits about you in the USVI and you'll probably be alright.
There isn't any cultural scene in the USVI to speak of to be honest. Reichhold Center and Pistarckle Theater make a impressive effort to offer such but it can't really be compared to what is available stateside.
The real reason most of us choose to live in the USVI or the Puerto Rican islands is the weather, the sailing, the cheap rum :-) ... all of which are plentiful and great.
Sailing around St John
Salt Pond on St John, a great quiet getaway..
Leinster on St John, busier but still great!
Don't overlook visiting the Puerto Rican islands of Culebra and/or Vieques. Often times you can have Culebrita (off Culebra) nearly all to yourself.

-------------------------
Enjoy a sundowner from your mooring in St Thomas (Crown Bay)