I would want to see both places, rather than longer at one. Port Douglas is smaller so you can walk from the ship to the town then from one side of the peninsular to the other, then walk along the beautiful long beach (which is what I did). Some passengers did a small boat tour independently from wharf here. Cairns is a city. Beautiful walk along the Esplanade, no beach, swim in big lagoon, lots of birds on mud flats, beautiful tropical botanical gardens. you can catch local bus to the Northern Beaches, day tours to Atherton Tablelands etc.