Yes you can! Once you have 9 villagers in your town and you request a new villager to move in, wisp will tell you a villager will have to move out to make space for the new one. He’ll suggest a villager at random but can keep selecting “a different villager!” Until he suggests the one you don’t like