I wrote a blog post detailing my homelab setup throughout 2023. It includes the hardware I use, as well as the applications I selfhosted. I also detailed how I automate my home Kubernetes cluster and how I back up my data.
In a cluster? I’m actually thinking about using Nomad between my three sff pc that I use as servers, but I have no clue as to how to sync storage between them (container side I mean, with nextcloud data for example)?
Yeah in a cluster with consul. Consul gives automatic service discovery and works with traefik so I don’t even have to care which node my service is running on since traefik knows how to find it using consul.
For the storage I went with a simple solution. I installed nfs on a machine running in nomad, and then configured the nomad clients to mount that disk. All of this with ansible so I don’t have to do it more than once.
Storage is hard to do right :(
If you can get away with it, use a separate NAS that exposes NFS to your other machines. Iscsi with a csi might be an option too.
For databases, it’s usually better to not put their data on shared storage and instead use the databases built in replication (and take backups!).
But if you want to go down the rabbit hole, check out ceph, glusterfs, moosefs, seaweedfs, juicefs, and garagehq.
Most shared file systems aren’t fully posix compliant so things like file locking may not work. This affects databases and sqlite a lot. Glusterfs and moosefs seen to behave the best imo with sqlite db files. Seaweedfs should as well, but I’m still working on testing it.
Yep, as someone who just recently setup a hyperconverged mini proxmox cluster running ceph for a kubernetes cluster atop it, storage is hard to do right. Wasn’t until after I migrated my minor services to the new cluster that I realized that ceph’s rbd csi can’t be used by multiple pods at once, so having replicas of something like Nextcloud means I’ll have to use object storage instead of block storage. I mean. I can do that, I just don’t want to lol. It also heavily complicates installing apps into Nextcloud.
