I’m a woman who is 5'10". I can tell you it absolutely maters. I’ve dated men from 5'6" to 6'7". I only prefer taller because most shorter men really don’t want to date taller than him or it will become a source of insecurity. The knife can cut both ways.
