Unless you run your charger off a solar array, which would have to be quite large, you are correct. And in certain states it's the grid is powered by Lignite coal, which is the dirtiest form of the rock.
Fortunately, in Texas we have a large and growing wind power sector that will help even things out.