But this is not a post about that, that post was great, it was long and descriptive and kind of amusing to read. And then I deleted it by accident due to some questionable UI choices in WordPress and the wonky trackpad on my tablet.
Did you know WordPress will permanently delete posts without prompting. Hit that “delete permanently” button and it’s gone. Not even Windows is that mean. My mouse pointer was a pixel into the hitbox for “Delete permanently” and I was trying to hit “restore”. And it went.
So yeah my network broke, like properly broke.
What broke? Pretty much all the fundamental core things, it was almost as bad as the time one of my core switches went a little bit mad and thought some of its ports were on a separate VLAN even though they weren’t.
If you’ve cosplayed as a sysadmin, you’ll know these situations where you can mostly connect to the Internet, except sometimes it doesn’t work and you’re off being asked to play the Chrome jumpy dinosaur game. Or DNS queries take a really long time and your whole machine crawls. Or randomly YouTube goes into 144p potato vision and your Roku claims it’s not online even though it’s playing a video.
Turns out my router died. Not properly dead, just dead enough to not route some stuff. I upgraded to gigabit fibre from less quick fibre earlier in the year. My ISP contacted me and said my 300mbit fibre package was running out and for the same price I could get their 500mbit package. Naturally I said yes, and they replied with “did you know you can get 900mbit for just a fiver more?” and I replied with “and have you turned it on yet then?”. Takes two days to regrade your fibre it seems.
My router didn’t like it, it kept dropping packets. Mostly packets dealing with the web admin interface, but probably other packets too.
And DNS died. I run an internal DNS and an Nginx reverse proxy. It’s a mess and confusing, there’s ad blockers in there too. It’s a whole thing.
Well it isn’t any more, I binned it all and put in a real DNS server and that works much better.
Also I forgot to write down all the static DHCP leases I have and decided renumbering my network was a good idea (it wasn’t). I managed to fundamentally break most things, especially anything with hard coded IP addresses like MySQL permissions, or “this is a hack to make a thing work behind a reverse proxy, rewrite this to be 192.168.1.123” type nonsense you set up once and forget about.
I think it’s working now, but I did say that earlier today before it broke again. Due to complex and stupid reasons I have to NFS mount my Mastodon server’s web folder on my reverse proxy or Mastodon doesn’t work. And if you lose an NFS server because its DHCP lease expires and it gets a new one that’s different everything breaks. I mean everything, because the poor Pi is rapidly trying to remake the broken NFS share and can’t do anything else at the same time.
I guess this did turn into a post about my broken network. It’s OK now, I know this because my Google Assistant can turn my lights on through Home Assistant, my car’s EVSE has a flashing purple light (seriously, if that thing isn’t flashing purple it means it’s lost its Internet connection, and without an Internet connection its charging schedules don’t work, so it never turns on!).
