It was their duty to not fight in the Confederate army, and their duty to fight against it. Part of the whole enemies foreign and domestic thing. Nobody was doing their duty by fighting for the confederacy, that was literally the opposite of their duty. Do not enlist, and if drafted then desert; that was their duty.