Your assumptiom would most likely be wrong. The vast majority of punks in general are anti-racist and would gladly punch a Nazi. The ones that are most likely to actually deliver the punch are the bald guys in typical punk attire.

The original skinhead movement was very progressive, anti-racist, and there are plenty of skinhead punks still representing it in the exact same way. All of the skinhead punks in my area identify as SHARPS or Skinheads Against Racial Prejudice, and I don’t live in a super progressive place or anything.

Nazi skinhead punks aren’t seen in public much, nowadays, mostly thanks to the guaranteed violence from the people mentioned above. If they are brave enough to be in public, they will make it known with some sort of asshole patch, asshole tattoo, or doc marten boots with white laces, but I’ve never seen one at a normal punk show.

Source: Lifelong enjoyer of punk music and shows.