It can be difficult, but it’s not unfair. I must say that once you get the hang of it it nails the “this game is a joy to play because it flows so well” thing that is incumbent to rhythm games being fun. So you want to keep playing (at least I do; I have 171 hours logged) long enough to get better.