He swore an oath to defend the U.S. constitution and the United States, not Virginia. Virginia’s attempted secession from the U.S. was unconstitutional and illegal, therefore they were in open rebellion against the nation to which he had sworn fealty. That is treason under any definition.