Unless you booked the tour through the ship, it will probably leave from the downtown area. I do think you are underestimating how long it will take for the shuttle bus. When I was there last summer, the line for the shuttle in the morning was around two hours. If there is another ship, that time increases. Make sure you are one of the first off the ship. Also allow at least an hour for the return shuttle before the all aboard time unless they agree to drop you off back at the ship.