Most contaminates come from water on land flowing into the ocean, and the water can still be clear. The runoff can contain chemicals like motor vehicle oil, pesticides and other chemicals, animal waste, etc. It usually flushes out of the water after a day or 2. If the water is contaminated at an extremely high level, they close the beach. That RARELY happens. Like maybe every few YEARS and usually only at a few beaches.
Some tourists get in the water where locals won't go - like the beach at the St John ferry dock. Too many boats that may contaminate the water.
I saw a YouTube video about the swimming pigs in the Bahamas, and how the pigs poop in the water when people are feeding the pigs. Most disgusting thing I can think of, yet people love it, but it's not a healthy environment for humans. That video prompted me to let people know that USVI regularly tests many beaches.