Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

Find approximate filenames

I have a movie file collection. In the past, I was less discriminating about filetypes, but now I have standardized on .mkv format, because it can include subtitles, additional audio tracks and even video tracks, etc. Over time, I have replaced some of the non-mkv files, but I never cleaned up all the old .avi or .divx files. So I might have a video in multiple file formats, one of which can just go away.

To find duplicates, I wrote a script that uses tre-agrep for fuzzy-searching.

My script find-approx.sh:

#!/bin/sh
# Startdate: 2022-09-25 17:07
# Goal: for each non-mkv video file, see if I have a fuzzy-match mkv file for it
# Dependencies:
#    tre-agrep
#    find . ! -type d ! -name '*.nfo' ! -name '*.jpg' ! -name '*.png' ! -name '*.srt' ! -name '*.pdf' ! -name '*.txt' ! -name '*.csv' ! -name '*.sh' > approx1
INFILE=approx1
_func() {
   _in="${1}"
   # we know to always strip file ending
   _match="$( basename "${_in}" | sed -r -e 's@\.....?$@@;' )"
   # should probably also strip (YYYY)
   _match="$( echo "${_match}" | sed -r -e 's@ ?\((19|20)[0-9]{2}\) ?$@@;' )"
   # and remove ", The" and similar for good measure
   _match="$( echo "${_match}" | sed -r -e 's@, (The|A|An)$@@;' )"
   #echo "Do something with \"${_match}\""
   tre-agrep -e "${_match}.*\.mkv" < "${INFILE}" | sed -r -e "s@^@FOUND \"${_in}\": @;"
}
grep -viE '\.mkv' < "${INFILE}" | tr '\r' '\0' | while IFS='\0' read foo ;
do
   _func "${foo}"
done

So not every printed line is an actual match. "The Godfather.avi" matched "Godfather, The, II (1974).mkv", so clearly I need to ignore some. But I was able to clean up a few old-format files that I no longer need!

Check set -x in dash

This works in dash, and bash in sh-compliant mode.

is_x_set="$( set -o | awk '/xtrace/{if($2=="on")print "Y"}' )"
sh ${is_x_set:+-x }/path/to/subscript.sh

This example here shows how to run a subscript with "set -x" if the parent script was called with that, or if it is currently set at this point in the parent script.

AoE2DE on Linux with Alsa

I recently reinstalled my gaming workstation, and of course I reinstalled Age of Empires 2: Definitive Edition. It's one of the few non-open source activities I allow myself.

Well, with this OS reinstall, I set up Devuan GNU+Linux which uses sysvinit. I had the choise to use pulseaudio, like my Fedora GNU/Linux installation used. I recall that you could use the pulseaudio graphical control utility to change the default audio sink, which immediately switched over running applications' default audio streams. However, I am going more traditional and less poettering-style, and using just alsa for audio these days (and fluxbox for my window manager, as previously discussed).

I have had this setup for about a month, and it has gone well, including audio, until the past few days. Somehow, my default audio device switched probably to my Nvidia graphics card. I tried adjusting my /etc/asound.conf which worked in general, such as for vlc.

defaults.pcm.card 1
defaults.ctl.card 1

I still needed to manually choose the correct card for the game, however. Reviewing the output of steam and/or the game (by running steam in a terminal window), I saw these logs:

ALSA lib confmisc.c:767:(parse_card) cannot find card '0'
ALSA lib conf.c:4568:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4568:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4568:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory

This could have been because I unplugged what was getting picked for the default device, a cheap webcam that has a microphone but no speakers, so it really should never show up as an audio sink anyways.

After reviewing my output of aplay -l (a snippet is below), I set an environment variable in the Steam settings for the game.

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 1: PCH [HDA Intel PCH], device 0: ALC221 Analog [ALC221 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

So in steam settings for AoE2DE, I now have it set up with:

PROTON_USE_DXVK=1 ALSA_CARD='PCH' %command%

Also, I am currently using Proton 7.0-4 for compatibility. I was previously using 5.13-6.

References

Weblinks

  1. Advanced Linux Sound Architecture - ArchWiki
  2. Using apulse for alsa audio in games launched via proton? :: Steam for Linux Steam Play
  3. [SOLVED] Steam/proton issues with some games, related to ALSA - Page 2

Internet searches

  1. alsa lib cannot find card 0

Powershell find user password expiration date

edited 2023-02-09

This started as a direct duplicate of https://powershell-guru.com/powershell-tip-38-find-the-user-password-expiration-date/ but I improved it.

For a nice powershell function that shows a human-readable date for when the password expires on an account:

function Get-ADUserPasswordExpiration
{
    Param
    (
        [string]$Identity
        ,[Parameter (Mandatory=$False)][string]$Server = "ipa.example.com"
        ,[Parameter (Mandatory=$False)][System.Management.Automation.PSCredential]$Credential = [System.Management.Automation.PSCredential]::Empty
    )
    $Params = @{
        Identity = $Identity
        Server = $Server
        Properties = 'msDS-UserPasswordExpiryTimeComputed'
    }
    If ($Credential.UserName){$Params["Credential"]=$Credential}
    [DateTime]::FromFileTime($((Get-ADUser @Params).'msDS-UserPasswordExpiryTimeComputed'))
}

My value-add includes the optioanl -Server and -Credential parameters.

Also from that source

To list all the Active Directory constructed attributes:

Get-ADObject -SearchBase (Get-ADRootDSE).SchemaNamingContext -LDAPFilter "(&(systemFlags:1.2.840.113556.1.4.803:=4)(ObjectClass=attributeSchema))"

Jellyfin Chromecast selection never appears

I had a problem with my production Jellyfin server a few days ago.

User "jellyfin" was unable to pull up the menu of available Chromecast targets, so Chromecast was entirely unavailable. Also, new logins for the user would not always go through. The server logs showed:

[2022-09-19 09:09:10.415 +00:00] [INF] [43] Emby.Server.Implementations.HttpServer.WebSocketManager: WS "258.1.15.15" request
[2022-09-19 09:09:10.526 +00:00] [ERR] [13] Microsoft.EntityFrameworkCore.Query: An exception occurred while iterating over the results of a query for context type '"Jellyfin.Server.Implementations.JellyfinDb"'."
""System.InvalidOperationException: The data is NULL at ordinal 5. This method can't be called on NULL values. Check using IsDBNull before calling.

A search of the Internet and more specifically github jellyfin showed a promising suggestion:

sbstp commented on Aug 7, 2021 @sweisgerber I actually found a better fix, this issue came back for me. The fix was to go into jellyfin.db with sqlite3 (command line) or sqlitebrowser (gui) and delete the rows of CustomItemDisplayPreferences where value is null. Make sure to backup the db before doing that just in case.

DELETE FROM CustomItemDisplayPreferences WHERE Value IS NULL;

Somehow the settings in the database table got corrupted. This is probably a failure of the application logic and not something done by a user. But it's something I can live with! Thankfully there are smarter people on the Internet who experienced this before I did.

My mute notification for Fluxbox and volumeicon

I use fluxbox, with volumeicon-alsa, and notification-daemon. I could use the bog-standard notifications that volumeicon provides, but the little icons in the notification are really small. So I wrote my own.

In ~/.fluxbox/keys, I react to the volume mute button:

#121 :Exec amixer sset Master,0 toggle
121 :Exec /usr/local/bin/audio-toggle.sh

Instead of just toggling the mute with fluxbox, I call my special script at /usr/local/bin/audio-toggle.sh.

#!/bin/sh
# Startdate: 2022-09-12
# For fluxbox
# handle the audio being muted and send cute notification
tmpfile=/run/user/${UID}/audio-toggle-notify-id
AUDIO="muted"
ICON=/usr/share/icons/Adwaita/64x64/status/audio-volume-muted-symbolic.symbolic.png
amixer sset Master,0 toggle 2>&1 | grep -iqE '\[on\]' 2>/dev/null && {
   AUDIO="unmuted"
   ICON=/usr/share/icons/Adwaita/64x64/status/audio-volume-high-symbolic.symbolic.png
}
nid="$( cat "${tmpfile}" 1>/dev/null 2>&1 )"
notify-send --transient --expire-time 2000 ${nid:+--replace-id=${nid} }--icon "${ICON}" "Audio ${AUDIO}" --print-id | tee "${tmpfile}" 1>/dev/null 2>&1

I wanted the size 64 icons so I could see them! Also, I wanted the black icons because my default theme in notification-daemon is apparently the nice, boring gray that hides white icons.

Flashing window on taskbar

I got very interested, for probably no real reason, in knowing how to make my window titles blink or flash in the taskbar (iconbar in Fluxbox). I wanted to know if Fluxbox supports it, and it does. It's called urgency or _NET_WM_STATE_DEMANDS_ATTENTION.

A great little tool named xurgent exists and is trivial to compile. You tell it what window id (from xwininfo) to make urgent. You can accomplish the same with xdotool (upstream website) as well:

xdotool set_window --urgency 1 $( xwininfo | awk '/ id/{print $4}' )

This command lets the user use the crosshairs cursor to select a window, which will then set its window hint for urgency. Flubox and other reasonable window managers then interpret this as the need to flash the taskbar entry for that window.

If you want to have a very simple python program that can set its own urgency:

#!/usr/bin/env python3
# Reference:
#    logout-manager for basic python3 gtk3 app
#    https://stackoverflow.com/questions/3433615/python-window-focus/3433968#3433968
#    timeout_add https://www.programcreek.com/python/?code=berarma%2Foversteer%2Foversteer-master%2Foversteer%2Fgtk_ui.py
import gi, time
gi.require_version("Gtk", "3.0")
from gi.repository import Gtk, GLib
class MainWindow(Gtk.Window):
   def __init__(self):
      Gtk.Window.__init__(self, title="Normal program")
      self.connect("notify::is-active", self.is_active_changed)
      self.grid = Gtk.Grid()
      self.add(self.grid)
      button0 = Gtk.Button(label="_Make urgent")
      button0.connect("button-press-event", self.on_button0_press_event)
      button0.connect("activate", self.on_button0_press_event) # activate covers ALT action if used and spacebar when selected
      button0.set_use_underline(True)
      self.grid.add(button0)
   def is_active_changed(self, window, param):
      if self.props.is_active:
         self.set_urgent(False)
   def on_button0_press_event(self, *args):
      print("counting to 3")
      GLib.timeout_add(3000, self.set_urgent, True)
   def set_urgent(self, *args):
      # should call with True or False
      if len(args) >= 1 and args[0]:
         self.set_title("Urgent notification!")
         self.set_urgency_hint(True)
      else:
         self.set_title("Normal program")
         self.set_urgency_hint(False)
# MAIN LOOP
win = MainWindow()
win.connect("destroy", Gtk.main_quit)
win.show_all()
Gtk.main()

My dvd ripping solution, 2022 edition

Last year, I wrote about my dvd-ripping solution. Of course, over time my process has improved and evolved.

Ripping the DVDs

I now use makemkv again, because Handbrake itself suggests that other tools are better for ripipng from removable media. Handbrake is for transcoding.

When I stick discs in my (3) drives, I then run a command inside GNU screen.

time APPLY=1 INPUT=/dev/sr0:/dev/sr1:/dev/sr2 JOBNAME=s3d6 sh -x makemkv.sh

The jobname is arbitrary and helps keep job outputs distinct when I let them pile up. Here is the script makemkv.sh.

#!/bin/sh
# Startdate: 2021-10-02 19:21
# Purpose: rip disc automatically
# History:
#    This is the second version of makemkv.sh (first one was renamed to makemkv-2019.sh)
#    2022-08-03 13:11 add unshare to keep makemkv from accessing the network
# Reference:
#    /mnt/public/Support/Programs/DVDs/handbrake-internal.sh
# Flow: use makemkv to make the original rip, and then handbrake to convert to the preferred size/format.
test -z "${INPUT}" && INPUT="/dev/sr0" # colon-delimited
test -z "${OUTPUTDIR}" && OUTPUTDIR=/mnt/public/Video/temp/makemkv
test -z "${JOBNAME}" && JOBNAME="$( < /dev/urandom tr -dc 'A-Z0-9' | head -c6 )"
test -z "${step1file}" && step1file="${OUTPUTDIR}/1-${JOBNAME}-run-makemkv.sh"
test -z "${step2file}" && step2file="${OUTPUTDIR}/2-${JOBNAME}-run-handbrake.sh"
printf '' > "${step1file}"
printf '' > "${step2file}"
for line in $( echo ":${INPUT}:" | tr ':' '\n' | grep -viE '^\s*$' ) ;
do
   bline="$( basename "${line}" )"
   echo "process ${line}"
   mminfo="$( unshare -r -n makemkvcon info dev:"${line}" )"
   #echo "${mminfo}" >> "${TMPFILE}"
   count="$( echo "${mminfo}" | sed -r -e '1,/Total/d' | grep -c Title )"
   raw="$( HandBrakeCLI --markers --format mkv --min-duration 125 --scan --input "${line}" --output "${OUTPUT}"  --encoder x264 --rate 30 --native-language eng --title 0 2>&1 )"
   disctitle="$( echo "${raw}" | sed -n -r -e '/DVD Title:/{s/.*DVD Title: //;p}' )"
   x=0
   while test $x -lt $count ;
   do
      OUTDIR="${OUTPUTDIR}/${JOBNAME:+${JOBNAME}_}${bline}/${disctitle}_${x}" ; OUTDIR="$( echo "${OUTDIR}" | tr ' ' '_' )"
      #mkdir -p "${OUTDIR}"
      #echo makemkvcon mkv "dev:${line}" "${x}" "${OUTDIR}"
      echo "mkdir -p \"${OUTDIR}\" ; unshare -r -n makemkvcon mkv \"dev:${line}\" \"${x}\" \"${OUTDIR}\"" >> "${step1file}"
      newfile="$( find "${OUTDIR}" -mindepth 1 -maxdepth 1 ! -type d -print -quit 2>/dev/null )"
      test -z "${newfile}" && newfile="${OUTDIR}/*.mkv"
      outfilename="$( echo "${OUTPUTDIR}/${JOBNAME:+${JOBNAME}_}${bline}_${disctitle}v${x}.mkv" | tr ' ' '_' )"
      echo "HandBrakeCLI --markers --format mkv --input "${OUTDIR}"/*.mkv --output \"${outfilename}\" --encoder x264 --rate 30 --native-language eng --mixdown 6ch --aencoder ffaac --audio 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15, --subtitle 0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,scan" >> "${step2file}"
      x=$((x+1))
   done
done
# So after all the prep work is done, we need to run step 1 and then step 2
echo "BEGIN STEP 1"
if test -n "${APPLY}" ;
then
   sh -x "${step1file}"
else
   cat "${step1file}"
fi
echo "####################################################################"
echo "####################################################################"
echo "You may now eject the disc(s)."
echo "####################################################################"
echo "####################################################################"
echo "BEGIN STEP 2"
if test -n "${APPLY}" ;
then
   sh -x "${step2file}"
else
   cat "${step2file}"
fi
echo "DONE"
date

Once that is done, the raw ripped files are stored at glob /mnt/public/Video/temp/makemkv/JOBNAME_sr*/*/*.mkv, and the handbrake-transcoded files are at glob /mnt/public/Video/temp/makemkv/JOBNAME_sr*_namefromdisc.mkv.

Now I manually inspect each video file and rename them to the correct episode name, or extra file name (which probably won't be updated in the next step).

For show episodes, I run a script which I described in that previous post. I have an input csv which I hand-curate from the Wikipedia pages usually, that includes columns s, sep, ep, airdate, filename.

have,s,ep,sep,title,airdate,filename
1,1,1,01-e02,Encounter at Farpoint,1987-09-28,s01e01-e02 - Encounter at Farpoint
1,1,3,3,The Naked Now,1987-10-05,s01e03 - The Naked Now
1,1,4,4,Code of Honor,1987-10-12,s01e04 - Code of Honor
1,1,5,5,The Last Outpost,1987-10-19,s01e05 - The Last Outpost
1,1,6,6,Where No One Has Gone Before,1987-10-26,s01e06 - Where No One Has Gone Before

I describe my csv generation process a little more in that previous post.

Then I run the next script:

time tv-mkv-helper.py --inputcsv "/mnt/public/Video/TV/Star Trek The Next Generation (1987)/STTNG.csv" -d /mnt/public/Video/temp/

This script is still exactly the same as in the previous post. I like the metadata of the original airdate on the episode, as well as of course video title. And I have never found a use for the "next file" tags that mkvtoolnix can show you, but I populate them because I can. Maybe it'll be useful someday.

And then I manually move the files to their intended destinations.

Linux: Fluxbox on Nvidia driver: Fonts are huge

I installed Devuan GNU+Linux on a desktop computer with a Nvidia graphics card. Once I used the nvidia-driver dpkg and rebooted, everything worked correctly. Fluxbox had a problem where its size 8 fonts were gigantic! Investigating revealed that the Nvidia settings utility shows a DPI of 304x305. An Internet search for nvidia fluxbox large font showed a very useful solution from the FreeBSD forum

Inside the generated xorg.conf (which I placed in /etc/X11/xorg.conf.d/hostname.conf), add to the "Monitor" section:

Option         "UseEditDpi" "false"
Option         "DPI" "96 x 96"

Also, I learned that lspci -v will show you what kernel module is in use for a given PCI device. Here is my graphics card with the proprietary Nvidia driver.

01:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev a1) (prog-if 00 [VGA controller]) Subsystem: Gigabyte Technology Co., Ltd GP107 [GeForce GTX 1050 Ti] Flags: bus master, fast devsel, latency 0, IRQ 30, IOMMU group 1 Memory at f6000000 (32-bit, non-prefetchable) [size=16M] Memory at e0000000 (64-bit, prefetchable) [size=256M] Memory at f0000000 (64-bit, prefetchable) [size=32M] I/O ports at e000 [size=128] Expansion ROM at 000c0000 [virtual] [disabled] [size=128K] Capabilities: Kernel driver in use: nvidia Kernel modules: nvidia

References

Internet searches

  1. nvidia fluxbox large font

Weblinks

  1. https://forums.freebsd.org/threads/fluxbox-fonts-are-huge.22849/
update 2024-04-13

While you might be tempted to place a file, /etc/X11/Xresources/dpi with contents:

! bgstack15 2023-07-16-1 17:29
Xft.dpi: 96

Which tends to help most of the time, it doesn't fix the "hardware" dpi that this post helps with. See also https://old.reddit.com/r/archlinux/comments/gsq0ix/nvidia_drivers_ruins_my_resolution_and_dpi/ which indicates to place this in a file, e.g., /etc/X11/xorg.conf.d/hostname.conf:

Section "OutputClass"
    Identifier "nvidia dpi settings"
    MatchDriver "nvidia-drm"
    Option "UseEdidDpi" "False"
    Option "DPI" "96 x 96"
EndSection

Linux: Check Publix sale paper in command line

I have simplified how I check the sale paper for a store in my area, Publix, by scraping the website and parsing the useful output myself. Check out my project aptly named coupons. It supports two stores from the SouthernSavers.com website, although a store is really just a uuid so it'd be trivial to add the other ones the site supports.

To use coupons.py, you specify a store and optionally a search string in lowercase to filter the sale items.

$ ./coupons.py --store publix --search candy | jq
INFO(get_cached_contents): using cache /home/bgstack15/.cache/coupons/publix_2022-09-01.json
{
  "Publix Ad: 8/31-9/6 or 9/1-9/7": {
    "Buy One Get Ones": [
      "Hershey's Chocolate Candy Bars or Reese's Peanut Butter Cups or Big Cup or Kit Kat, 6 ct, at $6.99 <small>($3.49)</small>"
    ]
  },
  "Extra Savings Flyer: 8/27-9/9": {
    "Grocery": [
      "Candy or Cookie Pop Popcorn, 5.25 oz, $3.50"
    ]
  }
}

I'm going to put this in a cron job now and then I'll never have to read the salepaper again; I'll just scan it automatically for the things I want!