People have been “worried about that” for the Fediverse for years. It hasn’t stopped it growing.

The main question is “what is the alternative?”. The only alternatives are corporate social media (e.g. Reddit), where you’re even more at the mercy of the admins, or some kind of peer to peer approach which may take off one day, but hasn’t really got going yet.