Maybe it is just the one-on-one situation can be misinterpreted as a date or date-like event by others. You don’t have to be expecting sex to be the predetermined outcome of a meal to just not want gossip...especially as a public figure. Not really all that crazy of a stance, but to each their own.