Here's another vote for Hotel du Vieux-Quebec. We stayed there ten years ago, and again this past fall. Just delightful. If you decide to stay there, book directly through the hotel to get their morning croissant basket.
Fairmont Chateau Frontenac is amazing. Even better if you can get off season rates which are more affordable. If you're not staying there, consider a drink in the bar or breakfast.