Unless you’re new, no one really uses the phrase literally. It’s just as sexist as referring to someone as a dick. I’ve had multiple female friends who used it.