There are very limited crossings from Amalfi, so be sure to nail these ferry tickets down if you decide to go to Capri.
The island is very crowded, my personal preference is to grab a taxi immediately upon arrival (they will put six or so people in one taxi, there aren't enough to go around otherwise) and go right up to Anacapri, then work your way back down.
Walking between the port, Capri and Anacapri is possible (The Phoenician Steps) but it's pretty steep and, in the summer, very hot.
The Phoenician Steps - Itinerari - Island of Capri