Since I had the same question on my first TA, it is a perfectly reasonable question! No, you don't have to wear them to dinner. I have never seen one at dinner, and at least outside of the ballroom, the were few and far between after dinner. I carted masks from home and, as two rather shy people, never put them on.