qwerty-scripts

Git-tracked backup of system scripts and service config on qwerty (HP EliteDesk 800 G4 Mini, 10.0.50.83). Remote: sweetpea:/srv/git/repos/qwerty-scripts.git

These files live at system paths that are not inside any Docker volume or existing git-tracked directory. This repo exists purely to protect them from an OS disk failure.

---

snapraid/

Manages parity protection for the 3-drive mergerFS NAS pool on qwerty.

File in repo	Canonical system path	Notes
snapraid/snapraid.conf	/etc/snapraid.conf	Read by snapraid at runtime
snapraid/snapraid-sync.sh	/usr/local/bin/snapraid-sync.sh	Executed by root crontab

What snapraid-sync.sh does

Runs daily at 2:00 AM via root crontab. Steps:

1. Sends a Pushbullet warning notification ("maintenance starting in 5 minutes")
2. Enables Nextcloud maintenance mode
3. Stops: nextcloud-fpm, nextcloud-web, talk, immich-server, immich-machine-learning, jellyfin (leaves postgres/redis running)
4. Runs snapraid sync
5. Restarts all stopped containers
6. Disables Nextcloud maintenance mode
7. Sends Pushbullet completion notification

Logs: /var/log/snapraid-sync.log

How to edit snapraid files

The repo copies are for backup, not live execution. Edit the canonical system paths directly, then sync to this repo:

# After editing /etc/snapraid.conf:
sudo nano /etc/snapraid.conf
cp /etc/snapraid.conf ~/qwerty-scripts/snapraid/snapraid.conf
git -C ~/qwerty-scripts add snapraid/snapraid.conf
git -C ~/qwerty-scripts commit -m "Update snapraid.conf: describe change"
git -C ~/qwerty-scripts push

# After editing /usr/local/bin/snapraid-sync.sh:
sudo nano /usr/local/bin/snapraid-sync.sh
cp /usr/local/bin/snapraid-sync.sh ~/qwerty-scripts/snapraid/snapraid-sync.sh
git -C ~/qwerty-scripts add snapraid/snapraid-sync.sh
git -C ~/qwerty-scripts commit -m "Update snapraid-sync.sh: describe change"
git -C ~/qwerty-scripts push

---

audio-swap/

Nightly batch job that remuxes the surround track on Jellyfin library files into AC3 5.1 (Dolby Digital) — eliminates server-side transcoding for clients that don't speak DTS/TrueHD/FLAC. Each ffmpeg pass runs inside the jellyfin container so output files land at the right uid:gid (114:121) without host sudo.

File in repo	Canonical system path	Notes
audio-swap/audio-swap-nightly.sh	/home/aj/scripts/audio-swap-nightly.sh	Executed by user crontab
audio-swap/worklist.txt	/home/aj/scripts/audio-swap/worklist.txt	Static input — one absolute path per line

Runtime state under /home/aj/scripts/audio-swap/ is not tracked (done.txt, failed.txt, run.log, run.lock, *.bak* are gitignored). The script appends to these files; recreate them empty on restore.

What audio-swap-nightly.sh does

Runs nightly at 3:00 AM via user crontab (0 3 * * * /home/aj/scripts/audio-swap-nightly.sh). Hard stop at 06:30 between items so it never spills into morning. Per item: ffmpeg inside the jellyfin container produces <name>.remux.mkv, then mv over the original + chown 114:121.

Defensive behavior (hardened 2026-06-03 after a SnapRAID-overrun cascade burned 746 worklist items):

• Poll-wait at startup: if jellyfin isn't running yet (e.g. 02:00 SnapRAID outage window ran long), poll every 30 s until it comes up or the 06:30 STOP cutoff hits. No items are marked FAILED during the wait.
• Mid-loop guards: before each docker exec ffmpeg and on any non-zero rc, re-check that jellyfin is still running. If it disappeared mid-run, break cleanly without burning the worklist — leaves the queue intact for the next night.

How to edit audio-swap files

Edit the canonical system paths directly, then sync to this repo:

# After editing the script:
nano /home/aj/scripts/audio-swap-nightly.sh
cp /home/aj/scripts/audio-swap-nightly.sh ~/qwerty-scripts/audio-swap/audio-swap-nightly.sh
git -C ~/qwerty-scripts add audio-swap/audio-swap-nightly.sh
git -C ~/qwerty-scripts commit -m "Update audio-swap-nightly.sh: describe change"
git -C ~/qwerty-scripts push

# After editing the worklist (rare — usually regenerated by a one-off `find` command):
cp /home/aj/scripts/audio-swap/worklist.txt ~/qwerty-scripts/audio-swap/worklist.txt
git -C ~/qwerty-scripts add audio-swap/worklist.txt
git -C ~/qwerty-scripts commit -m "Update audio-swap worklist: describe change"
git -C ~/qwerty-scripts push

---

jellyfin-pushbullet-bridge/

A small Node.js/Express bridge that receives Jellyfin webhook events and forwards them to Pushbullet as push notifications.

File in repo	Canonical system path	Notes
jellyfin-pushbullet-bridge/server.js	~/jellyfin-pushbullet-bridge/server.js	Live service file
jellyfin-pushbullet-bridge/package.json	~/jellyfin-pushbullet-bridge/package.json	Node deps
jellyfin-pushbullet-bridge/package-lock.json	~/jellyfin-pushbullet-bridge/package-lock.json	Lockfile
jellyfin-pushbullet-bridge.service	/etc/systemd/system/jellyfin-pushbullet-bridge.service	systemd unit — controls autostart

Runs as a systemd service: jellyfin-pushbullet-bridge.service Listens on port 3030. Handles PlaybackStart and SessionStart events. Jellyfin Webhook plugin (GenericOptions) POSTs to http://10.0.50.83:3030/webhook.

How to edit the bridge

Edit the live file, restart the service, then sync to repo:

nano ~/jellyfin-pushbullet-bridge/server.js
sudo systemctl restart jellyfin-pushbullet-bridge
sudo systemctl status jellyfin-pushbullet-bridge

cp ~/jellyfin-pushbullet-bridge/server.js ~/qwerty-scripts/jellyfin-pushbullet-bridge/server.js
git -C ~/qwerty-scripts add jellyfin-pushbullet-bridge/server.js
git -C ~/qwerty-scripts commit -m "Update bridge: describe change"
git -C ~/qwerty-scripts push

---

Restoring after OS disk failure

# Clone this repo
git clone sweetpea:/srv/git/repos/qwerty-scripts.git ~/qwerty-scripts

# Restore snapraid files
sudo cp ~/qwerty-scripts/snapraid/snapraid.conf /etc/snapraid.conf
sudo cp ~/qwerty-scripts/snapraid/snapraid-sync.sh /usr/local/bin/snapraid-sync.sh
sudo chmod +x /usr/local/bin/snapraid-sync.sh
# Re-add root crontab entry: 0 2 * * * /usr/local/bin/snapraid-sync.sh >> /var/log/snapraid-sync.log 2>&1

# Restore audio-swap
mkdir -p /home/aj/scripts/audio-swap
cp ~/qwerty-scripts/audio-swap/audio-swap-nightly.sh /home/aj/scripts/audio-swap-nightly.sh
chmod +x /home/aj/scripts/audio-swap-nightly.sh
cp ~/qwerty-scripts/audio-swap/worklist.txt /home/aj/scripts/audio-swap/worklist.txt
touch /home/aj/scripts/audio-swap/{done,failed}.txt
# Re-add user crontab entry: 0 3 * * * /home/aj/scripts/audio-swap-nightly.sh

# Restore bridge
cp -r ~/qwerty-scripts/jellyfin-pushbullet-bridge ~/jellyfin-pushbullet-bridge
cd ~/jellyfin-pushbullet-bridge && npm install
sudo cp /etc/systemd/system/jellyfin-pushbullet-bridge.service /etc/systemd/system/  # re-create if lost
sudo systemctl daemon-reload
sudo systemctl enable --now jellyfin-pushbullet-bridge

---

nfs/

NFS exports for the mergerfs pool. Authoritative copy lives at /etc/exports; the repo copy is for OS-disk-failure backup.

The host-specific 10.0.50.47 entry on /mnt/storage with no_root_squash,no_all_squash is required for arr-stack hardlink support (added 2026-06-04) — see Raymer/docs/qwerty.md → NFS exports for the full table and rationale.

File in repo	Canonical system path	Notes
nfs/exports	/etc/exports	Read by exportfs at boot and on exportfs -ra

How to edit nfs/exports

Edit the canonical system path, apply, then sync to this repo:

sudo nano /etc/exports
sudo exportfs -ra
sudo exportfs -v   # verify
sudo cp /etc/exports ~/qwerty-scripts/nfs/exports
git -C ~/qwerty-scripts add nfs/exports
git -C ~/qwerty-scripts commit -m "Update /etc/exports: describe change"
git -C ~/qwerty-scripts push

Restoring after OS disk failure

sudo cp ~/qwerty-scripts/nfs/exports /etc/exports
sudo exportfs -ra
sudo exportfs -v