Today on “weird crap that seems undocumented on the Internet” we have DNS that doesn’t want to resolve and DHCP that is acting weird. My home DNS server decided it didn’t want to resolve things any more, and it took far too long to figure out why.
It was all working fine, then it wasn’t. It was all probably triggered by my ISP updating the firmware in my router, which seems to have switched on IPV6 DHCP and thoroughly confused my whole internal network.
So I run my own DNS at home. This isn’t for ad blocking reasons, it’s not a Pi Hole. I used to have one of those, but got annoyed by random things breaking because they couldn’t resolve super secret DNS entries for super secret tracking or APIs. Looking at you YouTube, you’re starting to get super grumpy when your ad servers can’t be reached.
No, this is because I self host my websites, and need to be able to access them from within my network. As far as you all care, “ncot.uk” resolves to a real world IP address. As far as I care, it’s on 192.168.1.xxx and more importantly, is behind an NGINX proxy, so really it needs to point to that.
Trying to loop back through your router is a route to madness, especially with ISP provided routers aimed at home users. So I have a split DNS system using Unbound and NSD.
It works, it’s great, there’s nothing wrong with it.
Until complicated Linux stuff gets in the way. I mean all that stuff that tries to manage network connections and alter the sacred resolv.conf file. At what point did people decide one Linux package should poke itself into the config of another package in a way that seemingly nobody knows about when you’re Googling?
Well, until you find some random post online where some poor person is being berated for daring to suggest a slightly hacky way to fix things, and someone with zero personal skills wades in with “that’s a dumb way, why aren’t you reading the docs and doing it correctly like this? You’ll break things”.
If you’ve poked around in that part of your machine before, you’ll know what I mean. Utilities like resolvconf, weird firewalls that get in the way and scripts linked in with the DHCP client that silently overwrite files and set up DNS servers on 127.1.1.0.
This is all magical crap that works fine on client machines, but on the machine that is a DNS server it causes no end of problems. It took me forever to realise DHCP was overwriting resolv.conf, but that some sneaky bullshit was also overwriting part of unbound’s config using DHCP config data too.
And if you’ve ever tried to diagnose broken DNS, it’s impossible.
This is your weekly reminder that cosplaying as a sysadmin can waste your Friday evening and Saturday morning, and to add your own time into the cost/benefit calculations of running servers at home or on a cloud service.
4 responses to “When did Linux get so complicated?”
I have a very similar setup, using Unbound for external queries, and ISC BIND for the internal network, with BIND forwarding non-local queries to Unbound. (On BSD Unix.)
On my GNU/Linux systems, I simply uninstall resolveconf. Then set the apt-pining to prevent it being re-installed by a dependency or some other system update.
FWIW, I recently found my OpenBSD laptop fiddling with my DNS settings. I just de-installed resolved, and now the thing works the way I want.
If your machines are going to obey someone else, you may as well run Windows… 😉
“but that some sneaky bullshit was also overwriting part of unbound’s config using DHCP config data too.”
If this is what’s happening, this is certainly a bug and should be shared, reported and fixed.
Have you reported this or shared specifics anywhere?
Thank you for your article. Please note that (using Firefox) your linked text is so dark as to be hard to read.
Good point, dark green on black doesn’t work too well does it?
This is why I’m not a web developer, there’s too much fussing with CSS. Need to figure out where WordPress themes store their link CSS colours.