Almost forgot before going to bed but I feel bi-weekly is a good rhythm for this.
Let us know what you set up lately, what kind of problems you currently think about or are running into, what new device you added to your homelab or what interesting service or article you found.
You must log in or register to comment.
My pihole exploded yesterday, all my fault. A couple of years ago, I created a script called via cron to update pihole’s services every other week. This was great, until now when it updated to v6 at 4am. To make matters worse, I neglected to automate raspian updates, meaning it was very out of date, and was no longer compatible with pihole-FTL (thinking back, I thought I automated it too, but I guess not).
I took an image after creating a pihole “teleporter” backup, and began formatting. In my lack of caffeine and focus, I missed that my teleporter file was corrupt after I had successfully wiped the SD card. Thankfully I had that image as I was able to mount it and retrieve my blocklists via sqlite, otherwise I would have had to start from scratch.
One good thing that came out of it (for my taste, anyway) was that I swapped the OS on the pi to fedora. No more debian around here!
Tomorrow, I plan on setting up some backup automation for my pi, as it’s the only machine missing backups at this point.
Why so hostile sounding against debian?
What does fedora better?I don’t mean to sound hostile, that’s probably my past demons coming out. Like I said in my last comment, it’s really
aptthat I hate. It would constantly break or put me into dependency hell and I haven’t had to deal with that (yet) with Fedora.I haven’t put my finger on it, but Fedora, for whatever reason, also just feels faster.
Replaced the fan with a bad bearing on one of my proxmox hosts today. For a short while I figured I was going crazy because it seemed to stop making noise when I actually got close to the server, but it finally fully gave today and I was able to identify and swap it.
I upgraded immich without breaking everything. That’s always reason to celebrate.
I’m running the Immich Flatpak specifically for this reason. It’s always one version out of date but always self updates without issues :)
flatpak
You mean as a client or a server?
How exactly does stuff get broken? Never rly had a problem bumping up the version in docker. The only issue has been the playstore version taking longer to push updates sometimes for the mobile apps.
A few versions ago I upgraded it and some default port configs changed rendering it unusable. Since my upgrades are a docker command, I had to go hunt down the error message. It didn’t take long, but it def broke the setup.
I feel you 😂
You can use https://schedule.lemmings.world/ to automate the posts. Or, given the community we’re in, you can selfhost it!
This week I’ve been doing some work on my GOG Downloader to finally back up all my GOG stuff when I buy new disks, that’s pretty much it for my selfhost/homeserver stuff this week.
I’m the one who files the most bug reports on github under a different name. Our instance runs on Lemmy Schedule, so thanks!
I didn’t know that, cool! Though I should probably talk to the mods before setting up such a thing.
Finally got my lemmy instance fully updated.
Been improving my backup scripts in advance of adding backup to a server.
Updated servers and other services.
Realised my jellyfin lxc had a maxed out bootdisk yesterday, haven’t been using it for a while. Luckily I have decent backups setup so I was able to restore a backup from late January when it wasn’t filled yet. A quick library rescan and everything was up and running again.
I finally got link warden up and running, but I’m chasing down some failures on a few websites.
Also realized that me biting the bullet for unlimited bandwidth (screw you Comcast!) means I can run archive team warrior, so that’s been going.
IMO linkwarden was a real PITA. I’ve been trying linkding and it’s been really great so far. I’ve had no issues like I had with linkwarden.
Pihole 6 broke my DNS (dnsmasq), and since I had a fw rule in opnsense to only use pihole’s DNS, and deny public DNS access, it was an early rise for me :)
Damn… DNS issue early in the morning… What a nightmare 😂! Hope you got enough caffeine.
And that’s why you have either a backup for your DNS or know whats auto-updated ;)
As you mention opnsense:
What do you mean with fw rules to only use pihole dns?
This sounds partly like a DHCP config and partly like a deny (hardcoded) DNS requests and to please use what DHCP supplied (looking at you google/amazon)I did have backups, it was an easy fix. I had a
pihole -upon a crontab for years, probably not the best idea :)FW rule accept :53 from pihole only, deny :53 from all. I had some devices with hardcored DNS settings (8.8.8.8).
Unbound broke on both of mine day one of v6 and I’ve still not gone and fixed it. Sigh.
Had a hard drive fail my main zfs array. First time I have experienced a disk failure so it was a bit worrying. Thankfully I had added an additional drive to expand the array so I was able to quickly rebuild to that drive. Currently shopping for a replacement. From now on I think I will keep a cold spare just in case this happens again. I just wish hard drives would stop increasing in price.
You save some money by buying recertified drives from Serverpartdeals.
Yea that is what I have been doing. Although it seems the smaller sizes are not a great deal anymore and I am hesitant to buy anything larger due to the long rebuild times
I’m in the process of doing an initial restic sync of my primary storage to B2 as offsite backup and while I’m at it finally got around having a look at resticprofiles to simplify my restic backups on all my systems. Highly recommend it as it reduced my mental overhead of doing regular backups quite a bit!
After having upgraded my Pi-Hole to v6, for some reason yesterday it started to not recognize any of the blocklists. So, I resetted it and now it works.
Personally I’m mostly involved with my homelab migration so there’s not too much on the selfhosting page except os updates. I set up meshmini earlier to access my thin clients via vPro/AMT but I need to configure the clients before being able to actually using meshmini. Once I’m done with that I’ll finally be able to set up Lemmy and Pine pods.
My selfhosted stuff currently works fine without me doing much which feels good and lets me focus on hardware stuff currently.
Finally moved all my lxc onto a lower-power Xeon D host, consumes 1/3 the electricity of my previous Dell R430, same essential performance.
Just swapped VPS hosts from ssdnodes to MassiveGRID. Got a pretty sweet deal, so I’m pretty excited.
Got my services transferred over this week and it’s been fun as hell. It’s interesting because I was discussing Portainer with my buddy and he has Portainer on his local PC to connect to his remote instances and with hindsight it sounds obvious of course, but it’s such a nice little setup. Just finished setting up my Jellyfin reverse proxy so I’m gonna watch a movie and chill.
I used Portainer for a while and still like it for checking out networking stuff, but try out Dockge! It’s more open sourcey and basic, but makes updating easier.
Dockge
Dockage was nice, and it was much simpler, however, I had to leverage more docker commands via my VPS with it, because there weren’t a lot of options, specifically network settings.
I’ve been working on some bash scripts to help manage my media files. I’ve been slowly working on learning more bash and I’m pretty pleased with my progress. After I finish this bash book I’m reading (can’t remember the title atm), I think I’m gonna jump into awk.
Bash is a really great shell, but consider trying out a functional shell scripting language like Elvish (which is also a shell). Syntatically it’s pretty similar and not hard to pickup, but it’s stupid powerful. A cool example is updating different servers via ssh in parallel using a
servers.jsonfile;[ {"name": "server.com", "user": "root", "identity": "~/.ssh/private_key0", "cmd": "apt update; apt upgrade -y"}, {"name": "serverb.com", "user": "root", "identity": "~/.ssh/private_key1", "cmd": "pacman -Syu"}, {"name": "serverc.com", "user": "root", "identity": "~/.ssh/private_key2", "cmd": "apk update; apk upgrade"} ]and a little elvish magic;
var hosts = (from-json < servers.json) peach {|h| ssh $h[user]@$h[name] -i $h[identity] $h[cmd] > ssh-$h[name].log } $hostsJust run the script and boom, done. You can even swap out
peachwhich isparallel eachforeachif you want to do each command procedurally–but I really love using peach, especially with file operations over many different files. Linux is fast, but peach is fuckin’ crazy fast. Especially for deleting files (fd -e conf -t file | peach {|x| rm $x }, or one thing that I do is extract internal subs (so they play on my chromecast) in my Jellyfin server, using elvish makes it really fast;fd -e mkv | peach {|x| ffmpeg -i $x -map 0:s:0 $x.srt }Find all
*.mkvfiles, pass the filenames through ffmpeg (using peach) and extract the first subtitle asfilename.mkv.srt. Takes only about a few seconds to do thousands and thousands of video files. I highly recommend it for home-labbers.
Pretty dumb example, but
peachis like 6x faster;❯ time { range 0 1000 | each {|x| touch $x.txt }} 5.2591751s ❯ time { range 0 1000 | peach {|x| touch $x.txt }} 776.2411ms
