I think part of the reason that it was bugging me was that it felt like Caroline and Stefan basically immediately jumped to conclusions about what it meant that Elena and Damon had been at the lake house together that day. Like, god forbid it take Damon a day before he's able to bring himself to do something that is…