Ben, I do not even know the difference between "long sleeves" and "button-downs". Given that a fancy restaurant is somewhere I go to be polite and to socialize, not because I expect to get a lot of additional enjoyment out of the food, the additional effort of digging through my closet for clothes that I have not worn in over a year and may not fit, is not worth it to me.