I’m thinking about starting a self hosting setup, and my first thought was to install k8s (k3s probably) and containerise everything.
But I see most people on here seem to recommend virtualizing everything with proxmox.
What are the benefits of using VMs/proxmox over containers/k8s?
Or really I’m more interested in the reverse, are there reasons not to just run everything with k8s as the base layer? Since it’s more relevant to my actual job, I’d lean towards ramping up on k8s unless there’s a compelling reason not to.
I’m running a 3 pi cluster with k3s at the moment. The main benefit I’ve found is that all my pis run exactly the same software setup as a base so it’s easy to add new ones or replace/update one. I use a deployment management application to push my deployments too which means it’s super easy to redeploy everything if something goes funky.
That can be fun. The benefit of kubernetes is flexibility in the orchestration and (sometimes) scaling. Also the tooling in Kubernetes is more sofisticated compared to plain containers or manual services.
Kubernetes is basically just a finite-state machine that is able to manage a certain number of nodes as a pool of resources. This has added complexity compared to you managing the scheduling (I.e. I install this service on this box and this on this other box), but it also allows for much easier automation.
The basics can be useful there. The whole idea with k8s is to be able to run applications across multiple hosts in a given fleet. Your cluster can be that fleet! :)
Also k8s is in high demand in the sector, so those are good skills that could be turned into $$
I get why too. I’m a full stack (including devops) software engineer, and docker/k8s is just completely opaque to me. I’m not sure why, but I really just can’t wrap my head around it. Thankfully my current company has a devops team that takes care of it, but jeez
