I know that Lemmy is open source and it can only get better from here on out, but I do wonder if any experts can weigh in whether the foundation is well written? Or are we building on top of 4 years worth of tech debt?
Definitely not a silver bullet, but should stop the app from locking up when one thing gets overloaded. I’m sure they have their reasons for how it’s designed now and I’m probably missing something that would explain it all.
I’m still not familiar enough with how federation works to speak to how easy that would be. Unfortunately this has happened all as I’ve started moving and I haven’t gotten a chance to dive into code like id want to.
It’s also not the only solution for high-availability system. Multiple monoliths with load-balancing can be used as well.
Also, a lot of people are self-hosting. In this case, microservice won’t give them any scaling benefit.
The problem with scaling monoliths is you are scaling everything, including the pieces that have lower usage. The huge benefit you get from going to micoservices is you only have to scale the pieces that need to be scaled. This allow for horizontal scaling to use less compute resources. It also allow for these compute resources to be spread out as well.
A lot of the headaches can be removed by having an effective CI/CD strategy that is completely reusable with minimal effort.
The last headache would be observability. There you’re stuck either living with the nightmare of firefighting problems with 100 services in possibly 10 locations, rolling your own platform using FOSS tools or spending a whole lot of money on something like honeycomb, datadog or new relic.
But I’m an SRE, I live my life for scalablability and DevOps processes. I know I’m biased.
I find that often the overhead from microservices is worse than any savings from dropping a megabyte worth of unused machine instructions from a binary.
When your microservices need to talk to each other (and I’m not sure how many services you could split out of Lemmy without them needing to talk to each other), you’re doing a bunch of HTTP requests that are way slower than just calling another function in your monolith.
I see this at work every day. We run a distributed monolith because someone thought microservices would be a good idea, but we can’t actually separate everything, so it’s usual for an incoming API call to make 2-3 more calls internally. It can get so, so slow.
Scaling monoliths still works fine though. Microservices are first and foremost an answer to an organizational problem, not a technical one. There is a very high chance that if you are doing microservices with less than 20 people, or let’s say even 50 people, you are doing it wrong.
Microservices introduce a ton of overhead in engineering effort required, which needs to be balanced with the benefit they provide.
Scaling shouldn’t be the first and only reason for doing microservices.
The admin of lemm.ee is currently scaling things horizontally - it is possible.
I’m a software engineer myself, but not familiar with your field. How would your practice be applied to self-hosting? I’m assuming a bunch of people with their home servers wouldn’t want to just run OpenShift.