But as some on lyric site Genius and elsewhere have pointed out, Richmond was also the capital and northernmost city in the Confederacy. So, in that case, rich men north of Richmond would be... everyone in the North, which obviously paints a far grimmer picture.