Its definitely a commonly used name for the mindset they are describing. There’s nothing to try to defend. As another cis het male, “the boy’s club” is nothing to aspire to, unless of course, the goal is to belittle and victimize women.
At least in a stereotypical sense, the girls club is generally a group of women talking shit about other people behind their backs, sometimes bullying other women to their faces. You don’t commonly hear about a workplace of mainly women sexually harassing the few men to the point of self harm or raping them.
The labels may be borne of patriarchal values, but the mind sets of the people IN these groups are too. In my opinion, you’re simply denying reality in favor of a progressive idea of how it should be.
You’re misconstruing the meaning and intent of the phrase to support your argument. It in no way implies or affirms that all cis het males are bigots, only the males it is directly being used against. Similarly, calling a man a misogynist does not mean that all men are misogynists.