cross-posted from: https://lemmy.dbzer0.com/post/14381333
(my apologies if cross-posting is frowned upon, just trying to get as much advice as possible! Will remove if it’s no good)
Hey, y’all!
Here’s the deal:
I have a server I’ve been running for a couple years, running mostly home automation and NVR stuff (home assistant, node red, frigate, etc). This was my first server and it wasn’t set up in the best way possible. On top of that, it’s starting to suffer from hardware failure. So I’m replacing it with a retired gaming computer, and I want to do it “right” this time.
So far, I’ve got it running proxmox with a couple debian VMs (thought process was to have one “primary” one that runs most of everything, and a “network” one that runs network services like nginx, tailscale, etc - I don’t know if that separation is actually important or not). I, at some point, want to run pi-hole for sure. I also need a new router, so my thought was to set up opnsense for that. I also want to build a dedicated NAS somewhere down the line, but that’s another thread for another time.
I work from home and require stable internet, and I have family that will be very upset if internet is randomly going out from my tinkering with stuff, so I think it’s probably best to have totally separate, dedicated hardware for opnsense/pi-hole. I was looking at protectli, but it seems like I’d be looking at at least ~$300 for that option, and I’m not even sure I can run both opnsense and pi-hole on it? I’d also need to get an access point since I’d be replacing my current router that supplies wifi to the house, so I’m looking at like $400 for that transition, which is much more than I’d like to pay for this right now.
I could set everything up without the extra complexity of opnsense/pi-hole and add it down the line, but then I’d be looking at yet another complete re-work of the network and reconfiguring all my automations, cameras, etc., so it feels like it’d definitely be best to just do it up front and get it done. I have access to another old gaming PC I could theoretically set up as a dedicated network box to run opnsense and pi-hole on (after buying a NIC for it), but that seems wildly overkill (it’s running an i5 and 32gb RAM, if I remember right), large (full ATX case), and power-hungry for a glorified router. I guess, in this case, I could move my network vm off the “main” server and onto this one, to truly use it as a dedicated network box, running things like opnsense, pi-hole, wireguard/tailscale, nginx, authelia, etc. But then I start getting into the territory of it being too much of a “tinkering” box instead of a stable router that I allow to handle my network and don’t screw around with, lol.
So, I seek the advice of you much more experienced homelabbers. I’m terrified to do it “wrong” and wind up having to redo everything over and over, which I know is kind of antithetical to the entire idea of homelabbing in the first place. I need to avoid, as much as possible, unstable internet. In my shoes, what would y’all do? Bite the bullet and go for protectli? Use another old PC for the network box? Just set things up without opnsense/pi-hole for now and go protectli/something else later on and just deal with having to redo everything again?
Thank you so much in advance for any advice!
EDIT: I found a Zotac ZBox CL331 locally for $100 - would that be a good option, do y’all think?
(small aside: if anyone has any advice on moving my entire home assistant instance, node-red, and frigate setups (all separate docker containers) from the old server to the new one, that’d also be greatly appreciated!)
I will assume that you will expand your homelab with time.
I will also assume that you’re not going to open ports from your home network to the internet. If you need to do that, come back here/research Cloudflare tunnels (I have an alternative to those but that will need some explaining).
Now, purchase a 4 port/6 port “Router Firewall” from Aliexpress. They’ll cost you $200 + RAM + SSD, but at the end of that you’ll have an x86 router + switch (the celeron/pentium/i3s are plenty capable to switch without dedicated hardware). Get a TPLink/Mikrotik WAP and install OpenWRT.
Now, to bring my previous point into focus: you will place this router/firewall behind your consumer router. This will effectively place you behind double NAT, but if you’re using Comcast you shouldn’t care anyway (and it doesn’t matter unless you’re planning to expose ports).
How many drives do you need for your NAS? Look at Lenovo P310s and replace the PSU with your own. RAM + storage will be your biggest expenses in a homelab.
Use your gaming PC as a server, and get a Dell/HP/Lenovo mini PC for HA with it. Don’t even think about ESXi and jump straight into KVM/Xen. Bhyve if you’re brave/want to try something new.
Use Terraform/Ansible when you can, and nothing will terrify you anymore since you can literally rebuild the entire lab inside 15 minutes of you doing nothing but waiting.
Have fun!
Not OP, but I’ve been looking into Cloudflare tunnels on my end as well and ended up not going with them because you’re forced to use their own certs so they can decrypt and see the data. I mean most likely they aren’t doing anything untoward, but it’s still a consideration with regards to data privacy.
Absolutely. Everyone should, in theory, be alarmed by the blatant push towards being the world’s biggest MiTM by Cloudflare, but if one doesn’t care about that, then it’s a very good service.
I would personally consider running my own VPS with encrypted storage and run Wireguard tunnels (with the VPS as the server and my devices/home router) as the client. If I connect to the same socket, I could essentially connect to my home network without opening any ports there. Tailscale operates on a similar concept at scale, but you’d be better off hosting headscale if you care about privacy.
I was thinking the same thing regarding VPS and Wireguard. I use Wireguard personally to VPN into my home network for remote management, but I still haven’t looked up how to make a VPS as a proxy using it. I know they can join the same network and talk with each other but what’s the best way to route port 80 and 443 on the VPS to my server at home? Iptables?
That would be correct. Typically people recommend reverse-proxies for this because of their routing capabilities, but unless you have complicated checks and other functionality you need to implement, simple NAT is the best way to go about this.
nftablescan do exactly what you need with excellent performance.Thanks! Yeah i am already using a nginx reverse proxy in a docker container to expose my other docker containers so I was thinking two reverse proxies in a row might be too inefficient. Will definitely look into nftables. Nftable rules are temporary though right? What’s the correct way to automate running these rules on boot?
Ideally, to preserve them in a file and run a command to load them at start-up. This is trivial to do in OpenRC or similar init systems, whilst systemd will require a service configured for it.