myersguy

Well, I’m going to start by repeating that I don’t necessarily agree that it being monolithic is necessarily a problem right now.

The immediate thought in my mind would be all of the federation logic. That’s where all of the instances seem to be lagging behind, and it seems the common fix is “just increase the workers to one billion”. Apparently that does something meaningful, but the developer in me wants to know how a few cores can put so many workers to use.

Spinning federation off into a microservice means you could deploy it on something like Cloud Run or AWS ECS, and have it autoscale as the workload demands it. Seems like a pretty prime candidate to me.

I’d be less concerned with memory (of which Lemmy seems to use very little), and much more concerned with CPU core count. I touched on it in my other comment, but I don’t understand how a few cores is supposed to handle the ridiculous number of federation workers people are setting their instances to.

Again, my knowledge of the Lemmy codebase is very small, and we could possibly host the monolith in microservices style. The point I am making is this (when it comes to scaling monolith vs microservices):

If the federation logic were split out, we could configure it to run on super tiny docker instances on Google Cloud or AWS. Any time we needed it to, it would autoscale to handle the traffic. The configuration for these dockers could be super minimal memory, no storage, and multiple weak CPU cores. This would be super affordable while still being able to handle as much traffic from federation as we ask it to. One of the cool things with Google Cloud Run is that it handles load balancing between docker containers for you (just point the federation traffic at the necessary URL)

IF Lemmy has things like background services, scheduled tasks, etc, this would significantly muddy the water (we would need each service to be able to handle being run on a multitude of instances, or we would need to be able to disable each one instance by instance). And if we just scaled by spinning up more instances of Lemmy, we would also need to ensure that only federation traffic is heading to the weaker instances that we spun up for such purpose, or we would need to ensure that each spun up instance has enough resources to handle federation traffic along with the main application.

I feel like I need to state once more: I don’t necessarily think Lemmy needs to move to microservices. Only that scaling monolith vs Microservices is not necessarily the same.

Interesting indeed. I spent some time combing through the DB and pictrs folder to try to figure it out, but I’m at a loss so far.

Wow, that is surprisingly not bad given the size of the instance!

I’ve seen this happening a lot, so I would assume there are some issues federating comments.

This isn’t really contradictory to what I said. I only wished to express that the two don’t scale exactly the same (in terms of execution)

I understand why it had to happen, but I miss seeing the old name everywhere ☹️

Lol using closed source apps with paid tiers hiding important features for media 🤣

Mostly just poking fun. It came off as an odd flex (laughing at people who choose to pay for media) while also repping software that has features locked behind a paid tier (a subscription, at that)