From a local:
-There are adorable and colorful conch houses everywhere, so I would just walk around and find a spot or house you like. People here don't get pissy about pictures and the like very often - we're very used to the cruise ship folks and tourists in general!
-Hands down for fritters the Conch Shack which is right in front of Hogs Breath Saloon on Duval - CASH ONLY and ask for both sauces.
-For the Key Lime Martini, I will second the vote for the White Tarpon. I LOVE that spot!!!! Has good people watching, too!
Enjoy the Island!!!!