Usually that is what should happened, but since I switched to iOS 7 on my 4s and now on my 5s I've been having this issue. Now that I test it again it works, but I'm sure I'll go back to the buggy state soon.

Does anyone else have issues when trying to go back to a previous song on the music app? If I'm skipping through songs and accidentally skip a song I want to listen to I'm screwed because it won't let me backtrack to a previous song only restart the current song.