Although I don't have a lot of info about New Haven, we did have it as a port for Edinburgh last summer on the Island Princess. We went to St Andrews and DH decided I needed to see the course that he had the opportunity to play several times.
It is a tender port and is fairly close to Leith where the Royal Yacht is available for touring. Here are a few pictures of the port area.
A lighthouse near the tender dock
The view of the town from the lighthouse walkway
A few shops near the tender dock.
Sorry, that is all we saw of New Haven, but most everyone went off to Edinburgh or elsewhere, I believe. We did get to drive over the Firth of Forth bridge near Queensferry where most ships dock on our way to St Andrews.