Things happen in offscreenville sometimes. More time has passed than you've seen on screen and they've spent time together that we haven't seen. Relax, it's fine. Felicity obviously didn't think it was too soon, her not saying it back had nothing to do with that.