I have the opposite problem where I am normally in range of multiple “assistants” in my household in some areas and issuing commands will get several that will answer - even though they are -supposed to be- smart enough to know which one is closest to you and pass off interaction with that one.