N64 emulation on the Pi can be inconsistent, some games work and will run at close to full speed while other games will lag and have issues. If you're interested in trying to install a N64 emulator outside of RetroPie you can check out this article, and on this page they have a compatibility list of some of the best…

Hopefully this will help, if you're not careful you could damage your display so be cautious.