Ulysse smiles and pauses to glance back at his brother, commenting, "I would have thought you would have preferred to eat together upstairs, just the two of you...?"
Other options:
no subject
Ulysse smiles and pauses to glance back at his brother, commenting, "I would have thought you would have preferred to eat together upstairs, just the two of you...?"