Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

The non-free software I use

Here's what I can think of for the non-free software I use, and why. It's my walk of shame.

Steam and Age of Empires 2: Definitive Edition

I play AoE2DE because I've been playing Age of Empires 2 ever since the Conquerors Expansion in 2004. I remember the fun days of Voobly. The current scene is on the Microsoft services-centric AoE2 Definitive Edition, so I have to use it.

Discord

I use Discord for my AoE2 group.

Nvidia graphics

I have some hand-me-down Nvidia card that is enough to run Age of Empires 2. That's it. I don't care about anything else it can or cannot run.

Old computer games

There are many old computer games from before my free-software days, like kids' games, that I still have the original disc or materials for. I play through some of them on occasion. I'd say most of them are edutainment titles like from The Learning Company. Maybe this category doesn't count, because it's all old stuff.

Telegram

I use Telegram for communicating with family. You have to make a stand somewhere, and apparently I was way too late with my self-hosted Matrix idea (which still isn't implemented). I'm sure the NSA or FSB or whoever my threat model is can read anything I choose to send anyways.

Upcoming project: fix-alttab

This quick-and-dirty project is taking longer than I had hoped, and I have to get this post out! I am working on a little shell script daemon that will re-run alttab when I change inputs on one of my monitors, so alttab is always visible on the screen.

This is a preview of the fully-fleshed-out project (Makefile, xdg autostart entry, etc.). Here is file fix-alttab

#!/bin/sh
# File: fix-alttab
# Location: /usr/libexec/fix-alttab
# Author: bgstack15
# Startdate: 2023-07-10-2 08:10
# Title: fix-alttab
# History:
# Usage:
#    called by fix-alttab-daemon
# Reference:
#    https://unix.stackexchange.com/questions/537529/how-do-i-get-xdg-config-home-from-a-shell-script-command-line
# Improve:
# Dependencies:
#    xrandr, awk
# Documentation: README.md
# load settings
test -f /etc/bgscripts/fix-alttab.conf && . /etc/bgscripts/fix-alttab.conf
test -f "${XDG_CONFIG_HOME:-~/.config}/fix-alttab" && . "${XDG_CONFIG_HOME:-~/.config}/fix-alttab"
# just in case no settings defined there
test -z "${ALTTAB_COMMON}" && ALTTAB_COMMON="alttab -w 1 -theme numix-circle"
test -z "${ALTTAB_LEFTYES_RIGHTYES}" && ALTTAB_LEFTYES_RIGHTYES=""
test -z "${ALTTAB_LEFTNO_RIGHTYES}" && ALTTAB_LEFTNO_RIGHTYES="-vp 1920x1080+1920+0"
test -z "${ALTTAB_LEFTYES_RIGHTNO}" && ALTTAB_LEFTYES_RIGHTNO="" #unsupported
test -z "${ALTTAB_LEFTNO_RIGHTNO}" && ALTTAB_LEFTNO_RIGHTNO="" # unsupported
# for better security, strip out any semicolons from these values because we evaluate without quotes
_strip() {
   printf '%s' "${@}" | sed -r -e 's/;.*$//;'
}
ALTTAB_COMMON="$( _strip "${ALTTAB_COMMON}" )"
ALTTAB_LEFTYES_RIGHTYES="$( _strip "${ALTTAB_LEFTYES_RIGHTYES}" )"
ALTTAB_LEFTNO_RIGHTYES="$( _strip "${ALTTAB_LEFTNO_RIGHTYES}" )"
ALTTABLEFTYES_RIGHTNO="$( _strip "${ALTTABLEFTYES_RIGHTNO}" )"
ALTTAB_LEFTNO_RIGHTNO="$( _strip "${ALTTAB_LEFTNO_RIGHTNO}" )"
# always evaluate this
_ALTTAB_COMMAND_TO_KILL="$( echo "${ALTTAB_COMMON}" | awk '{print $1}' )"
# main
# set environment variables "left" and "right" to yes or no, to indicate if monitor is there.
unset right left
eval $( xrandr | awk 'BEGIN{a[0]="no";a[1]="yes"}/HDMI-1/{if($2~/\<connected/){r=1;}else{r=0};} /HDMI-0/{if($2~/\<connected/){l=1;}else{l=0;};} END{print "right="a[r];print "left="a[l];}' )
unset _ALTTAB_PARAMS
case "${left}${right}" in
   yesyes)
      printf '%s\n' "Using left yes, right yes"
      _ALTTAB_PARAMS="${ALTTAB_LEFTYES_RIGHTYES}"
      ;;
   noyes)
      printf '%s\n' "Using left no, right yes"
      _ALTTAB_PARAMS="${ALTTAB_LEFTNO_RIGHTYES}"
      ;;
   yesno)
      printf '%s\n' "Using left no, right yes"
      _ALTTAB_PARAMS="${ALTTAB_LEFTNO_RIGHTYES}"
      ;;
   nono)
      printf '%s\n' "Using left no, right no: UNSUPPORTED!"
      _ALTTAB_PARAMS="${ALTTAB_LEFTNO_RIGHTNO}"
      ;;
   *)
      printf '%s\n' "Unknown config: leftright \"${left}${right}\". Aborted." 1>&2 ; exit 1
      ;;
esac
test -n "${APPLY}" && {
   test -n "${DEBUG}" && set -x
   killall "${_ALTTAB_COMMAND_TO_KILL}"
   # unquoted here:
   ${ALTTAB_COMMON} ${_ALTTAB_PARAMS} &
   set +x
}

File fix-alttab-daemon which gets called by ~/.fluxbox/startup or with an entry in directory /etc/xdg/autostart/.

#!/bin/sh
# File: fix-alttab-daemon
# Location: /usr/bin
# Author: bgstack15
# Startdate: 2023-07-10-2 08:55
# SPDX-License-Identifier: GPL-3.0
# Title: fix-alttab-daemon
# Project: fix-alttab
# Purpose: Detect changes to connected HDMI monitors
# History:
# Usage:
#    run in ~/.fluxbox/startup in the background
# Reference:
# Improve:
# Dependencies:
#    plecho
#    fix-alttab
# Documentation: README.md
# load settings
test -f /etc/bgscripts/fix-alttab.conf && . /etc/bgscripts/fix-alttab.conf
test -f "${XDG_CONFIG_HOME:-~/.config}/fix-alttab" && . "${XDG_CONFIG_HOME:-~/.config}/fix-alttab"
command -v plecho 1>/dev/null && _use_plecho=1
unset _laststatus
while ! test -f /tmp/stop-fix-alttab-daemon ;
do
   sleep "${ALTTAB_DAEMON_LOOP:-3}"
   _status="$( APPLY= ./fix-alttab )"
   test "${_status}" != "${_laststatus}" && {
      APPLY=1 ./fix-alttab | \
      {
         test "${_use_plecho}" = "1" && timeout 2 plecho || cat
      }
   } | tee -a "${ALTTAB_DAEMON_LOGFILE:-fix-alttab-daemon.log}"
   _laststatus="${_status}"
done

And here is the config file. The above scripts read /etc/bgscripts/fix-alttab.conf first, and then ~/.config/fix-alttab.

# Example fix-alttab.conf
# Usage: dot-sourced by fix-alttab and its daemon
# Locations:
#    /etc/bgscripts/fix-alttab.conf
#    ~/.config/fix-alttab
ALTTAB_DAEMON_LOOP=3
ALTTAB_DAEMON_LOGFILE="${XDG_RUNTIME_DIR:~}/fix-alttab-daemon.log"
# First word here is the binary to run. This will be killalled if it needs to be run.
ALTTAB_COMMON="alttab -w 1 -theme numix-circle"
ALTTAB_LEFTYES_RIGHTYES=""
ALTTAB_LEFTNO_RIGHTYES="-vp 1920x1080+1920+0"
ALTTAB_LEFTYES_RIGHTNO="" # unsupported
ALTTAB_LEFTNO_RIGHTNO="" # unsupported

Yes, I'm in love with sleep loops. I aspire to be a Unix Master Foo.

Poor man's html image gallery

For a personal website for which I hand-roll the html code, I wrote a small shell image link+thumbnail generator script. I shouldn't be proud, but I am.

#!/bin/sh
# Startdate: 2020-11-27 15:09
INDIR=/mnt/public/www/images
DIRPREFIX="images"
THUMBNAIL_SIZE=300
USE_THUMBNAILS=1
if test -n "${USE_THUMBNAILS}" && test "${USE_THUMBNAILS}" = 1 ;
then
   mkdir -p "${INDIR}/.thumbnails"
fi
cd "${INDIR}"
tfs="$( find "${INDIR}" -mindepth 1 -maxdepth 1 ! -type d ! -name '.*.swp' ! -name '*.sh' ! -name '*.html' -printf '%T@ %f\n' | sort -n | awk '{print $NF}' )"
for word in ${tfs} ;
do
   tf="${DIRPREFIX}/${word}"
   tf_thumb_web="${tf}"
   if test -n "${USE_THUMBNAILS}" && test "${USE_THUMBNAILS}" = 1 ;
   then
      tf_in="${INDIR}/${word}"
      tf_thumb="${INDIR}/.thumbnails/${word%%.???}.jpg"
      tf_thumb_web="${DIRPREFIX}/.thumbnails/${word%%.???}.jpg"
      convert -filter Lanczos -resize "${THUMBNAIL_SIZE}x${THUMBNAIL_SIZE}" "${tf_in}" "${tf_thumb}"
   fi
   echo "<a href=\"${tf}\"><img src=\"${tf_thumb_web}\" class=\"thumb\"></a>"
done

The find | sort | awk line is my preferred method for sorting by timestamp. It is not safe for files with spaces in the name, but who does that anyways?! (Ahem. [Well, not here obviously!]). Of course there are ways to make that happen, but it would be even less clean looking, and avoiding spaces is easy enough.

So like all good unix scripts, this just prints to stdout. I then copy-paste the contents and update my html file.

I haven't done extensive testing with non-square images. I'm pretty sure most images on my gallery are non-square, but everything seems to line up adequately.

And what gives away the fact that I'm a vim user?

And of course, some applicable css:

.thumb {
   max-width: 300px;
   max-height: 300px;
}

Newspipe with ldap auth

I recently wrote about how I wrote a docker container for newspipe. Now, I have added ldap authentication and it has been accepted by upstream!

Now you can add a bunch of config options to your configuration .py file:

# Ldap, optional
LDAP_ENABLED = False
# LDAP_URI will automatically try the _ldap._tcp lookups like for a
# kerberos domain but
# will fall back to this exact domain (server) name if such a TXT
# record is not found.
LDAP_URI = "ldaps://ipa.internal.com:636"
LDAP_USER_BASE = "cn=users,cn=accounts,dc=ipa,dc=internal,dc=com"
LDAP_GROUP_BASE = "cn=groups,cn=accounts,dc=ipa,dc=internal,dc=com"
LDAP_USER_MATCH_ATTRIB = "uid"
LDAP_USER_DISPLAY_ATTRIB = "uid"
LDAP_USER_ATTRIB_MEMBEROF = "memberof"
LDAP_GROUP_DISPLAY_ATTRIB = "cn"
LDAP_BIND_DN = "uid=sampleuser,cn=users,cn=accounts,dc=ipa,dc=internal,dc=com"
LDAP_BIND_PASSWORD = "examplepassword"
# Additional filter to restrict user lookup. If not equivalent to
# False (e.g., undefined), will be logical-anded to the
# user-match-attribute search filter.
LDAP_FILTER = "(memberOf=cn=newspipe-users,cn=groups,cn=accounts,dc=ipa,dc=internal,dc=com)"

I wrote my ldap logic back for a sample project 2 years ago.

I haven't yet tried adding kerberos authentication. I was too lazy to fetch a kerberos ticket for HTTP/examplserver.ipa.example.com and deal with that. It should be possible to implement that however, for a motivated individual.

Re: Re-evaluating RHEL (www.sacredheartsc.com)

This is my reply to Stonewall on his post titled Re-evaluating RHEL from 2023-06-25. This started as an email but got too long to be an email message.

I appreciate your write-up. I chose AlmaLinux for when I built my first and still only EL8-level host. I suppose the whole point should be that AlmaLinux is identical to Rocky Linux. I was classically trained in the selinux and (systemd) frameworks, and I miss the "good old days." RPM makes sense to me, although dpkg isn't so bad either.

I too depend on FreeIPA and I second your observations about FreeIPA server on non-EL OSes. I think the Debian FreeIPA maintainer switches off the server bits during rebuilds intermittently because the software just doesn't get enough attention on non-RHEL systems. I would never try to run a freeipa server on anything other than a RHEL-like system. At the time I was switching to Devuan GNU/Linux for desktop usage, and even some servers now, I learned that FreeIPA client was available only in the unstable release. By now, it's in the first stable release. In my experience, the freeipa client components are entirely stable and usable on Devuan Ceres, well, after the package I maintain in Devuan for this: systemctl-service-shim, and my various scripts of course. I should do a writeup of my current Devuan ipa-client-install process.

With the recently announced change that RHEL free developer licenses are increased to 240, do you think that will satisfy your server needs? I still wouldn't do it, because of the licensing required.

Let me pontificate for a minute on my ideal distro of GNU/Linux:

  • Dnf with dpkg as currently provided with versioning conventions and package building processes. The only layer of Devuan's packaging that could be objectively improved with EL technology is the network package manager itself. Dnf transactions (and display output) are more palatable to me than the sprawling and inconsistent output of apt. The dpkg naming conventions as they relate to major package versions are really nice: gcc-9 and gcc-11 are both available at the same time! I'm also used to debuild which has its quirks, but is still useful and easy enough that I don't miss rpmbuild that much.
  • SELinux. AppArmor just feels... insufficient. I've seen SELinux cranked way up, and cranked way down, and disabled.
  • Systemd not mandatory. Ideally support for sysvinit. Ironically I like systemd as a service manager, but it's a shame it won't stay in its corner.

Because multiple of these desires are entirely imaginary, I use both a mixture of RHEL-like (CentOS 7, AlmaLinux 8) and Debian-like (Devuan Ceres) for my real network.

p.s. It sounds like you read a lot of the same technical content on the Internet that I do! You have extremely good taste, my friend!

Newspipe docker image

I found a new project to tickle my fancy: newspipe! This is a python+flask rss feed aggregator. I had perused the awesome-selfhosted list for a feed manager written in a language I'm familiar with. I was inspired by a fellow netizen's FreeIPA auth plugin for his rss reader.

Because I was just experimenting with this, I didn't want to install it directly to a system. Nor did I want any of the dependencies, like npm or whatever "poetry" is. So I whipped up a docker image for it, which took a lot of trial-and-error. I found only one existing, outdated docker image, for architecture arm7 which doesn't apply to me.

So here is my project: newspipe-docker! I have a dockerfile and also a docker-compose.yml. And yes, I based it on the Devuan image! I wanted ease of development, and not minimum size. I should refactor for that now, though, shouldn't I? Feel free to undertake that for me and let me know.

Next time: I add ldap auth, so I'm not lagging behind the cool kids.

Shell alias: hide-song

I have a number of music albums in their entirety: I am a completionist! But that doesn't mean I like every song in the album. Rather than delete those bits, I set them aside. Gzipping alone isn't enough because vlc is smart enough to still play the audio. So I base64 them, and then gzip them (and then sink them into the swamp).

And then I wrote a shell function to do this for me, preserving timestamp. I like metadata; I don't like data.

hide-song() { _i="${1}" ; test ! -f "${_i}" && return 1 ; <"${_i}" base64 >"${_i%%.mp3}.b64" && touch --reference "${_i}" "${_i%%.mp3}.b64" && gzip "${_i%%.mp3}.b64" && \rm "${_i}" ; }
unhide-song() { _i="${1}" ; gunzip "${_i}" && <"${_i%%.gz}" base64 -d > "${_i%%.b64.gz}.mp3" && touch --reference "${_i%%.gz}" "${_i%%.b64.gz}.mp3" && \rm "${_i%%.gz}" ; }

Because I need more word-count, let's explain these.

hide-song() {
    _i="${1}"
    test ! -f "${_i}" && return 1
    <"${_i}" base64 >"${_i%%.mp3}.b64" && \
        touch --reference "${_i}" "${_i%%.mp3}.b64" && \
        gzip "${_i%%.mp3}.b64" && \
        \rm "${_i}"
}

Line 2 sets a variable, and notably not a bash local variable, but I like to use a single underscore to help avoid collisions. I like to avoid bashisms, never mind this is in my bash profile.

Line 3 fails quietly but with an error code. Maybe it should throw a warning. I suppose the -f makes sure it is a real file, and not a symlink and not a directory so it's pulling double duty.

Line 4 base64s the file to file.b64. And yes, I'm anticipating the file has a .mp3 extension. Adapt to your own needs, detail-oriented people!

Line 5 sets the timestamp of the new file to match the original file. (Yes, I don't like bashisms, but GNUisms are perfectly acceptable.)

Line 6 gzips file.b64, which notably preserves the timestamp in my GNU userland. I hardly use gzip without tar, but I guess I do here.

Line 7 deletes the original file. Note the leading slash to avoid any aliases and just run rm. And of course note that the rm happens only if the preceding commands are all successful (&& being the logical and).

connmanctl set nameservers

When connmanctl borks up whatever symlink /etc/resolv.conf points to this week, particularly with ipv6 nonsense, you can easily set it back to the real nameservers:

# select config id of currently connected network
current="$( connmanctl services | awk '/^[^ ]/ && $1~/R/{print $3}' )"
connmanctl config "${current}" --nameservers 192.168.1.10 192.168.1.11

Or you could just clobber the resolv.conf with a real file and not a symlink to /etc/connman/resolv.conf. Crap like this makes me miss wicd although I'm sure it had its problems too.

Experimenting with Spacefm

I was asked to possibly step up and offer to maintain spacefm in Debian. I don't even use spacefm, and I don't plan on maintaining it, particularly if it is a dead project upstream. I use xfe which meets my needs (minus the CTRL+L bug in the current version, 1.45 and I cannot wait for the new one!).

So I decided to check out spacefm. The file manager itself is modest and probably suitable for me, but like I said xfe is already my main solution. When I discovered that spacefm has desktop management functions, I was intrigued! It parses the XDG Desktop path for .desktop files and displays them on the wallpaper! Previously in my fluxbox world I've had to use idesk, which has its own format for those. I much prefer .desktop, which is one of the only good specs to come out of that freedesktop.org group.

So, I spent some time investigating using a transparent wallpaper, and then loading up a compositor (which I also don't use) so my fbsetbg can work like it always does. The problem was my wallpaper image, which I want duplicated across both monitors, was not the same resolution as my display. It was smaller than my display! So, spacefm would struggle to, well, center it on a single monitor because of the 2 monitors, and also fail to scale it correctly.

The compositor picom messed up my window borders, and apparently has no configuration options for the borders other than radius size. So I gave up on the compositor, and just used convert to resize my wallpaper image, and then set spacefm --desktop-pref to use that scaled image, and now I have my cool ~/Desktop/*.desktop icons and my desired wallpaper.

So, if you use fluxbox and you want to use the standard .desktop icons, check out spacefm!

Auxiliary thoughts

I wonder if, because spacefm uses gtk2 (or gtk3, how nice that we get choices!), if it can send those dbus messages to tumblerd thumbnailer service to generate the thumbnails of images in a given directory. I bet it could use a plugin/extension/configuration to do such a thing.

Sharing youtube video to Metube on Android

I recently established my own local metube instance, which facilitates passing links to youtube-dl (er, yt-dlp). It works very well, particularly when I got the watchtower bit running so it reloads with an updated image, because yt-dlp is a fast-moving target.

So, in order to get this link as a sharing target in Android, I needed to set up a few things.

On the Android device in question, install Termux from F-Droid. No additional packages/plugins are needed.

In the terminal on the android device:

mkdir -p bin
cd bin
nano metube.sh

#!/bin/sh
#set -x
curl --insecure --request POST --url 'https://server3.ipa.example.com/metube/add' --data "{\"url\":\"${1}\", \"quality\":\"best\"}"
printf 'press enter to leave. '
read none

chmod +x metube.sh
ln -sf metube.sh termux-url-opener

The symlink termux-url-opener is defined in the official Termux docs.

Now, for any link I wish to send, select the "Share" button, and select Termux. I added a prompt so I can see the output of the request, in case it fails so I can investigate.

Alternatives

Use the bookmarklet in a desktop web browser:

javascript:(function(){xhr=new XMLHttpRequest();xhr.open("POST","https://server3.ipa.example.com/metube/add");xhr.send(JSON.stringify({"url":document.location.href,"quality":"best"}));xhr.onload=function(){if(xhr.status==200){alert("Sent to metube!")}else{alert("Send to metube failed. Check the javascript console for clues.")}}})();

One additional parameter metube uses:

curl --request POST --url 'http://localhost:8081/add' --data '{"url":"https://www.youtube.com/watch?v=XXXXXXX", "quality":"720p", "format":"mp4"}

Making a PWA with a sharing target

If Termux is unavailable, I could try making a progressive web app.

  1. https://chodounsky.com/2019/03/24/progressive-web-application-as-a-share-option-in-android/
  2. https://developer.chrome.com/articles/web-share-target/
  3. https://stackoverflow.com/questions/38189160/can-a-progressive-web-app-be-registered-as-a-share-option-in-android

References

  1. https://wiki.termux.com/wiki/Intents_and_Hooks
  2. https://github.com/alexta69/metube/issues/71
  3. https://github.com/alexta69/metube/issues/133