I run a self-hosted server at home on which I have run a bunch of personal stuff (like nextcloud etc.). To prevent pointing DNS servers at my home router, I run a reverse proxy on a VPS that I rent (from Scaleway FWIW).
Today I was trying to figure to what extent that exposes my data to my VPS provider and whether I can do something about it. Disclaimer: this is just a hobby exercise. I’m not paranoid, I just want to learn for my own self how to improve security of my setup.
My reverse proxy terminates the SSL connection and then proxies the connection over a wireguard connection to my home server. This means that (a) data is decrypted in the RAM of the VPS and (b) the certificates live unencrypted in the storage of the VPS. This means that the VPS provider, if they want to, can read all the traffic unencrypted to and from my home server.
I was thinking that I can solve both problems by using Nginx’s SSL pass-through feature. This would allow me to not terminate SSL on the VPS solving (a) and to move the certificates to my home server solving (b).
But just as I was playing around with it, I realised that SSL pass-through would not solve the problem of trying to protect my data from the VPS provider. As long as my DNS records point at the VPS provider’s servers, the VPS provider can always get their own certificates for my domains and do a MitM attack. Therefore, I might as well keep the certificates on the VPS since I still have to trust them not to make their own behind my back.
In the end I concluded that as long as I use a VPS provider to route my traffic to my home server, there is no fool-proof way to secure my data from them. Intuitively it makes sense, the data crosses their hardware physically and thus they will have access to it. The only way to stop it would be to update the DNS records to point directly at my home server which I don’t want to do.
Is this correct thinking or is there some way to prevent the VPS provider from seeing my data?
Again, I’m trying to solve this problem as a hobby exercise. The most sensitive data that I have is stored encrypted at the filesystem level and I only decrypt it locally on my own machine to work on it. Therefore, the actually sensitive data that would be cost me a lot if compromised is never available unencrypted on the VPS. Due to the overhead of this encryption and other complications, I don’t do this for all my files.
It sounds like you’re doing something very similar to me. I run my Lemmy and Mastodon server out of my home. I have a wireguard tunnel between that server and my cloud VPS. The cloud VPS handles reverse proxying. The information that I am most likely leaking is metadata. Metadata is surprisingly useful. In an ideal world we could secure and obfuscate everything. For the most part though, your traffic is secure and your cloud provider won’t be able to really get more than your metadata.
I don’t think it’s just metadata that’s leaking though. I would say it’s the entire content of the connection. If the reverse proxy terminates the secure connection it will decrypt the data which will be available unencrypted in the VPS. Outside of the VPS instance the traffic remains entirely encrypted.
Admittedly this decrypted data is not easy to access - you would need to have root access and be able to capture the traffic from within the VPS. But a VPS provider has this kind of access - as they run the hypervisor, they have direct access to the RAM (and possibly even a much easier way to just log in as root into the VPS itself). I think you do have to trust the VPS provider not to peek into the VPS itself. As long as you’re paying for the service, that’s probably a safe assumption.
There is no easy way to just log into the VPS as root just because you run the host. There are probably exploits that could do it but there is no simple included tool to do so.
It would probably be easier to just hook into the (likely known) VPS kernel doing something similar to the sslsniff tool from the bpfcc-tools package (hook into the openssl, gnutls,… ssl decryption functions basically).
Technically you’re correct: your VPS provider can inspect your network traffic, the contents of RAM and anything on the disk.
Bluntly: you have to trust your VPS provider, and if you’re unsure they’re trustworthy you shouldn’t use them.
(Scaleway is legitimate, bound by actual useful data protection laws, and has a comprehensive privacy and security policy.)
You could try destination NAT with netfilter/iptables (DNAT) and terminate TLS on your home server.
This way packets will be forwarded to the home server without beign decrypted on the VSP.
That’s an interesting solution. But could the cloud VPS provider still see the metadata?
You are correct. The provider owns the IP and also VPS. They theoretically have the ability to do anything within those confines. Same thing with your nameserver provider with your DNS records and the domain itself with the registrar. There’s a certain level of trust that needs to be accepted for anything that goes outside the confines of your house. The good thing is those companies have more to lose than you by breaking that level of trust.
Well, technically you only have to trust your DNS registrar minimally if you use DNSSEC and only use your DNS hoster’s servers as slave servers. The registrar could always DoS you of course by not entering new DNSSEC zone signing keys into the parent zone and the hoster by removing your zone from their servers but they can not technically publish any records in your name in that case or prevent individual records from being published (unless those are new and they just keep serving an old version of your zone).
Could you explain more about this? I’m very new to DNS, and cannot grasp how using DNSSEC prevent the VPS from tampering with DNS and how we only “minimally” rely on the DNS provider?
Also, could you explain more zone signing keys?
I actually made a mistake in my previous post by writing zone signing keys when I meant key signing keys.
Basically in DNSSEC there is a hierarchy of keys all the way from the DNSSEC root zone keys. Each DNSSEC-enabled domain has some key signing keys which have a signature signed by the parent zone’s keys stored in the parent zone as DS records similar to the way NS records are stored in the parent zone. This is done by the registrar for your domain. So e.g. the DS records for itjust.works would be stored in the works zone and the ones for works would be stored in the DNS root zone.
The domain owner can then use the key signing key to sign a regularly changing zone signing key (e.g. KSK might be valid for a year and ZSK for a month with some overlap to avoid outages). Both KSKs and ZSKs are stored in the zone itself as DNSKEY records.
The zone signing key is then used to sign each individual record in the zone. There is also a mechanism to certify that all the names in between and the records of other types do not exist but I don’t know the details of that. It is specifically designed to prevent enumeration of all existing records though.
So if you do all your zone signing on, say, your home server and only publish the zone via zone transfer on some DNS slave servers the organisation or person running your DNS slave servers will not be able to do anything other than publish it as is, publish an outdated version they received from you before (in which case you could switch out your DNSSEC keys to make that invalid) or not publish it at all (Denial of Service) but they could not publish their own records or modify the records you published.
Of course the caveat with that is that it only works if the clients actually validate the DNSSEC signatures of the zone/records but more and more do in recent years.
If you’re concerned that you VPS provider is replacing your certificates you need to find another provider.
You should also look in to certificate transparency monitoring. I get notified anytime a certificate gets issued for one of my domains.
