The Gem and Pearl go hand and hand. Both Tobacco and Horseshoe are nice. The caves are cool, Gibbs lighthouse. We tend to skip those since we have been there before. For anyone going for the first time I highly recommend doing a 5 hour private island tour the first day, you really get to see a lot. Our plan this time around in Tobacco Bay the first day, HSB the second and they renting a boat for 6 hours from H20 Sports.... https://www.h2osportsbermuda.com
We have rented from them in the past. Its a great way to get off the beaten path. The are a 7 minute cab ride from the dockyard