That's not true: they denied it almost immediately. Like QAnon, it's a hard question on whether to even acknowledge these kinds of conspiracies[0], since denying them can even help to spread them.
The Trump wing is without question populist, nativist, often conspiratorial, and caught in a cult of personality. But they're not Nazis, and by trying to conflate these things, you do a disservice to the victims of actual Nazism (note the publication I linked to, from a country of founded by Nazi survivors that has quite a few supporters of Trump).
Charlottesville Nazis likely did use it, but they're far from being Trump supporters. They deeply distrust him because of Jared and Ivanka. And they're a tiny, tiny percentage of our population - all extreme right-wing groups combined (neoconfederates, Nazis, militia, etc) are well under 1%. You'll see SPLC trumpet things like "30% rise!" without mentioning how tiny these groups are (and they have a good idea about how small they are, it's just harder to fundraise when you discuss these things).