As @BirdsDontFlyIntheRain said, you can do it on a rooted device. The two ways I've done it are using XPrivacy for the XPosed framework, and OpenPDroid. I prefer the former solution.
There is actually a module buried in the settings in Android (I think 4.2.2 and above) that will accept or deny permissions on apps for…