original link: https://tech.lgbt/@ShadowJonathan/111840265157264311

archived: https://archive.is/BMbc2

Okay, just to be clear;

Tencent/Alibaba/the CCP/whatever has specific software targeted at scraping mastodon servers, by hitting the unauthenticated “list posts from user” API endpoint, with specific account IDs, that they must’ve determined to be interesting from some moment or another.

The only way to get those account IDs is to ask the server to resolve them, thats already a step further than a normal scraping bot.

What’s more, the IPs and user agents (basically, text which tells what browser you have) all ONLY hit this specific endpoint, meaning this is a targeted, predetermined, scraping operation.

By looking up the account IDs from the last dozen or so attempts that i see here in my logs, almost each of them are some colorful flavour of trans, gay, or enby.

They are specifically targeting us queer folks with this scraping.

orignal post: https://tech.lgbt/@ShadowJonathan/111839244695370421

archived: https://archive.is/80RLB

yeah okay no

PSA for all fedi admins; block “QQDownload” as a user agent

(edit: this wont work for everything, see my replies for ip ranges)

its specifically scraping statuses from accounts with pre-scraped account IDs, meaning they know what they’re downloading

i have no idea what its about, one site says that its a p2p download manager, and another says its related to QQ, an instant messenger service from tencent

i have a feeling it might be an extension of their state surveillance system

edit: grepping logs further, it doesn’t seem like all of them have “QQDownload” in it, so also block “TencentTraveler” from accessing apis unauthenticated

edit 2: urgh… it seems that the user agents are not consistent, and it seems that a geoip database also wont really help, since some of them come from singapore, from alibaba cloud…

edit 3: im just gonna wack-a-mole it, ive set up something for our server to see these requests and for me to manually add IP ranges to ban