I’m currently traveling for months at a time and my homelab has become unreachable to me over VPN due to a unknown complication after a power outage.

Just as a learning experience for all, my mistake was that I set-up my VPN very far down the stack - as a wg-easy app inside TrueNAS SCALE’s apps ecosystem. My very important reason for doing it was that way was that wg-easy allows for setting up client devices with a QR code…

Anyway, the NAS is not booting back up nor do the TrueNAS apps. I should’ve set my VPN up right at the front of the network - on my MikroTik router that also supports Wireguard. The funny thing is I was so happy that my NAS has IPMI and whatnot but now I can’t even access it.

For now the NAS is kept powered on from what I know, it just doesn’t boot. This should help prevent bitrot until I’m back. All important files are backed up on a 3rd party service.

It’s a shame my Jellyfin and Navidrome inaccessible, but I’ll live.


Now I’m thinking about buying an UPS so that this doesn’t happen in the future. I’d like the UPS to be fanless and rackmount, so that limits me to ~700VA territory.

Devices in my homelab pull about 65W idle and spike to say 150W when everything is booting. ISP modem, router, POE+ switch, AP, NAS. I might add another 20W due to a Lenovo M920q in the future.

I only really care about NUT and graceful shutdown instead of long runtime on battery.

I was thinking about this: https://www.apc.com/us/en/product/SMT750RMI2U/

In my country I can get it with new batteries (no front panel) and a network card for NUT for a total of 180 EUR.

Would that work? Would you be afraid of leaving an UPS (it is kinda like a bomb after all) unattended an leaving your home for 6 months at a time?

1 point
*

UPDATE: Turned out that the culprit of the downtime was my switch - the D-Link DGS-1210-10P rev. B1.

The way the management web interface of the switch works is pretty unintuitive. Namely, if you change some settings in the web interface and hit save in one of the sections, the settings are saved in the volatile memory of the switch. This basically means that the settings are only saved in RAM, which is cleared on power loss. To save the settings into non-volatile memory which persists on reboots, you need to find the “Save” section at the top of the UI. This is described here: https://networkengineering.stackexchange.com/questions/20158/dlink-switch-loses-configuration-on-power-off

So basically, my problem was that the settings weren’t commied to nonvolatile memory and on a short 1 minute power loss the switch restarted.

I got an UPS anyway now, SMT750RMI2U

permalink
report
reply
2 points

If you can afford one, I would strongly recommend going with a dual-conversion UPS. A line-interactive UPS like the one you posted essentially acts as a pass-through for your mains power until it detects a power loss or a brown-out. This works most of the time, but there’s a short delay during the switch from line to batteries (just guessing, but most likely on the order of milliseconds). This might not sound like much, but you’re counting on the capacitors in your server’s power supply to hold enough charge until the UPS kicks in.

The other thing to consider is that a dual-conversion UPS also supplies “clean” power to your equipment. It essentially acts as a DC power supply connected to an inverter, so regardless of how bad your input power is, you’re always going to get the correct voltage and frequency out. I connected my old line-interactive UPS to a cheap generator at one point; the voltage and frequency regulation was so bad on the generator that my UPS continually switched on/off of battery (several times per second), and the equipment attached to it immediately shut down.

I can connect my dual-conversion UPS to the same generator, and it keeps humming along as if it was connected to mains voltage. According to the datasheet, anything from 60VAC to 150VAC, it’s still going to output clean 120V/60hz power.

They’re much more expensive. Mine is 1000VA, and if I remember correctly, I paid something like 600 or 700 USD for the UPS. An add-on rackmount battery pack was another $300 or so. It was well worth the cost, though.

permalink
report
reply
2 points

You want the ability to remote power cycle everything and to get a quick status. Get a single board computer and use it as a jumping off point. From there you can go get some smart plugs that can be connected to the single board computer.

permalink
report
reply
3 points

Always UPS everything. But also always have a simple backdoor. I generally have 1 little desktop like a NUC running some basic Win10 OS and an install of remote software like TeamViewer. It is connected to my hardware router right after the ISP router and a backup connection. Used to be LTE everywhere, now I’m half and half on Starlink. It is then also connected to the router ports needed for management but inactive.

If I have to, remote into the NUC over Starlink. I can then reboot my main ISP box. I can eventually get into my router and enable those ports which are pre-plugged in. From there I can then access all the stack management and all the IPMI ports like iLo. It’s a virtual interface through a virtual interface. It is slow, and painful. But it works.

And it works 99.99% of the time. But even then, I’ve had to do a call of shame and walk one of my friends through which button to press as I’m on the other side of the world. In my case it was also power related but the UPS I had decided to overheat. In reality over the summer, the temps were high. But also it is a super awesome double conversion UPS. The line voltage into the UPS was dropped to below standards from the utility because their grid was overworked with everyone’s AC’s. So the UPS saw this as a line failure, kicked in the double conversion and ran happily. But it did not count as a power failure, so none of my services scaled back. Essentially it was delivering 3KW of juice from the wall through a double conversion making the whole thing super hot. Eventually it shut down for safety automatically, just pulled the plug. My NUC is on a separate little backup along with the modems and an auto transfer switch which did its thing. But there was no way to press the reset button on the UPS for a critical safety shutdown like that. It had to be in person.

permalink
report
reply
4 points

The device you’re thinking of has 42 decibels of sound. You should be aware of that, I don’t think it’s actually fanless

Noise is going to be a huge factor for your home lab, so make sure you look at the data sheet for whatever you’re about to buy and check what it’s rated noise level is

permalink
report
reply

Community stats

  • 256

    Monthly active users

  • 289

    Posts

  • 2.1K

    Comments

Community moderators