Knowledge Base (Posts about metadata)

Trying owntracks to track my mobile devices

bgstack15 — Tue, 11 Nov 2025 13:40:00 GMT

I was reading some discussion on Hacker News and the fine folks there mentioned Dawarich, a self-hostable server for managing your location data. I got interested in this sort of thing. I've never really used stuff like this before for myself, even from the big names. I use Google Maps for navigation, but not for showing myself my location history. I'm sure the setting for "do not record my location" merely turns off my ability to look it up; I'm sure Google still knows exactly where my mobile device has been (that is not the same as where I have been) across all of time.

But F-Droid didn't have the Dawarich app, but it had the app for owntracks, a similar tool.

Stackrpms installation of owntracks

This application is for tracking my location history on my own infrastructure.

Goals

I want to be able to view my location history. In the past, I used Google Maps location history to find where I had stayed on a particular vacation. I want to be able to do this sort of auditing without depending on external vendors.

Setup

This application is installed in docker on server4.

sudo useradd owntracks
sudo usermod -a -G docker owntracks
sudo su - owntracks
mkdir -p compose ; cd compose

I established file docker-compose.yml and config.js. I just picked random ports available, including server4:8083 for recorder, and server4:8085 for frontend.

On server3, set up custom apache httpd config owntracks.cnf which is an adaptation of [Reference #5][5]. Adapt local_mirror.conf to include this for VirtualHosts 443 (internal https) and 444 (external https).

Establish new password repository. Use password scheme 3j.

# as root@server3
htpasswd -c .owntracks device5
htpasswd -c .owntracks device4

Added a port in the firewall on server4.

sudo firewall-cmd --add-port=8083/tcp
sudo firewall-cmd --add-port=8085/tcp
sudo firewall-cmd --add-port=1883/tcp
sudo firewall-cmd --add-port=8883/tcp

I established the container mosquitto, which uses its own TLS cert. The openssl req needed to be run on a newer system where parameter -addext can be used. I ran the ipa cert-request from server4 which owns the service we add here.

openssl genpkey -algorithm RSA -out https-server4.ipa.internal.com.key
openssl req -new -key https-server4.ipa.internal.com.key -subj "/O=IPA.INTERNAL.COM/CN=server4.ipa.internal.com" -addext "subjectAltName = DNS:server4.ipa.internal.com,DNS:www.example.com" -out https-server4.ipa.internal.com.csr
ipa service-add --force HTTP/server4.ipa.internal.com
ipa cert-request --principal=HTTP/server4.ipa.internal.com https-server4.ipa.internal.com.csr --certificate-out=https-server4.ipa.internal.com.pem

I configured the main router to forward port 8883/tcp to server4:8883. There might be a way to use an apache httpd reverse proxy for websockets to mqtt, but I have not investigated that.

To establish the mosquitto password database, use -c in the command, inside the container.

mosquitto_passwd -c /mosquitto/config/password_file recorder

The rest of the commands will omit -c.

The mosquitto config is in file mosquitto-conf/mosquitto.conf, and uses TLS.

Establishing useful contact info with cards

I want to be able to see pretty names/faces for "friends". https://owntracks.org/booklet/features/card/

I Used https://github.com/owntracks/recorder/raw/refs/heads/master/contrib/faces/image2card.sh from https://github.com/owntracks/recorder/blob/master/contrib/faces/image2card.sh to do this:

sh image2card.sh user1.jpg "user1" "na" > device5-device5.json
sh image2card.sh user2.png "user2" "nn" > device4-device4.json
sh image2card.sh user3.png "user3" "eo" > device2-device2.json

I prepared these so they are accessible for container mosquitto in /mosquitto/config/cards/, and then entered the container and ran these commands.

docker-compose exec mosquitto /bin/sh
mosquitto_pub -L 'mqtt://recorder:KEEPASS@localhost:1883/owntracks/device5/device5/info' -f /mosquitto/config/cards/device5-device5.json -r -q 2 -d

To remove/delete a card:

mosquitto_pub -L 'mqtt://recorder:KEEPASS@localhost:1883/owntracks/device5/device5/info' -n -r -d

Associated files

On server server4.

/home/owntracks/compose
- config.js, taken from official example https://github.com/owntracks/frontend/blob/main/public/config/config.example.js for frontend.
- docker-compose.yml, very simple combination of official compose example files for recorder (backend) and frontend.
- mosquitto-conf/password_file
- mosquitto-conf/mosquitto.conf
- mosquitto-conf/https-server4.ipa.internal.com.key
- mosquitto-conf/https-server4.ipa.internal.com.pem
- mosquitto-conf/ca-ipa.internal.com.pem
- mosquitto-conf/cards directory of custom .json files for "cards" that describe the various devices, and the script that loads them
  
  !/bin/sh
  
  cd "${1}" for word in *.json ; do echo "${word%%.json}" | while IFS='-' read a b ; do mosquitto_pub -L "mqtt://recorder:KEEPASS@localhost:1883/owntracks/${a}/${b}/info" -f "${word}" -r -q 2 -d ; done ; done

On server server3, the main web server.

/etc/httpd/conf.d/owntracks.cnf
/etc/httpd/conf.d/local_mirror.conf includes owntracks.cnf in virtualhosts 443 and 444.
/etc/httpd/.owntracks (or similar) for the credential.

Associated urls

http://server4.ipa.internal.com:8083 recorder. The main url for the apps includes virtual path /pub.
http://server4.ipa.internal.com:8085 frontend
https://www.example.com/owntracks reverse proxy to recorder.

Operations

Restarting the containers

Connect to owntracks@server4.

docker-compose pull ; docker-compose down ; docker-compose up -d

Adding a new device

Adding a device for MQTT auth

Connect to the mosquitto container and then modify the existing password file. The username will probably be the device name.

docker-compose exec mosquitto /bin/sh
mosquitto_passwd /mosquitto/config/password_file <username>
<password>

I am not sure if container mosquitto needs to be restarted to take effect.

Configure the client settings Connection.

Mode=MQTT
Host=www.example.com
Port=8883
Client ID=<device name>
Use Websockets=True
Device ID=<device name>
Tracker ID=?????
Username=<from password_file>
Password=<from password_file>
TLS=True

Make sure the android device trusts the CA cert, which is available at https://server3/internal/certs/ca-ipa.internal.com.crt

Adding a device for http auth

WARNING I think this might work, but it also might not work anymore now with MQTT usage.

I have decided to use a separate http credential for each device. Visit server3 and set up a new credential in the password file. This example shows adding device4.

sudo htpasswd -c /etc/httpd/.owntracks device4
<enter a password, probably "KEEPASS">

It is not necessary to reload the web server. Then configure OwnTracks on the new device to use this http endpoint.

URL=https://www.example.com/owntracks/pub
Device ID=device4
Tracker ID=<anything will do fine>
Username=device4
Password=<same as from htpasswd command>

Removing data about an old user/device combo.

To remove user "user" device "device5", run this command:

# on server4
sudo rm -rf /home/owntracks/compose/owntracks-recorder/store/last/user/device5

I am not sure if this removes info in the frontend for this user+device.

Viewing server app logs

docker-compose logs -f otrecorder mosquitto

Viewing location history

Visit http://server4:8085/ which is container frontend.

Re-adding the cards

A card is the pretty info about a device, such as user name and picture. These are hand-curated with a useful script (search image2card.sh in the install section), and after a container restart, you need to reload these.

docker-compose exec mosquitto /mosquitto/config/cards/import-cards /mosquitto/config/cards

References

Alternate reading

In case I want to experiment with using httpd to reverse-proxy MQTT or websockets.

https://www.rushworth.us/lisa/?p=358

proxy - Mosquitto + Apache - Super User

<VirtualHost *:1883>
        ProxyRequests Off
        ProxyPreserveHost On
        ProxyPass /mqtt ws://$Broker-IP:$Broker-Port
        ProxyPassReverse /mqtt ws://$Broker-IP:$Broker-Port
</VirtualHost>

n8n apache2 reverse proxy fix websockets – 🔐 Karlo Luiten

<IfModule mod_ssl.c>
<VirtualHost *:443>
    SSLEngine off
    ServerAdmin webmaster@karloluiten.nl
    ServerName n8n.karloluiten.nl
    DocumentRoot "/var/www/html/"
    ErrorLog "${APACHE_LOG_DIR}/error_n8n.servar_nl.log"
    CustomLog ${APACHE_LOG_DIR}/access_n8n.servar_nl.log combined

    RequestHeader set X-Forwarded-Proto https
    RemoteIPHeader X-Forwarded-For
    RequestHeader set X-Forwarded-Host "%{SERVER_NAME}e"

    ProxyPreserveHost Off
    ProxyPass        / http://10.0.40.54:5678/
    ProxyPassReverse / http://10.0.40.54:5678/

    RewriteEngine On
    RewriteCond %{HTTP:Upgrade} =websocket [NC]
    RewriteRule /(.*)           ws://10.0.40.54:5678/$1 [P,L]
    RewriteCond %{HTTP:Upgrade} !=websocket [NC]
    RewriteRule /(.*)           http://10.0.40.54:5678/$1 [P,L]
    ProxyPassReverse /          https://n8n.karloluiten.nl
    SSLCertificateFile /etc/letsencrypt/live/n8n.karloluiten.nl/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/n8n.karloluiten.nl/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

Obviously fill in your own details/domains/ip.

Also for env vars:

N8N_BASIC_AUTH_ACTIVE=true
N8N_BASIC_AUTH_USER=admin
N8N_BASIC_AUTH_PASSWORD=xxx
N8N_SECURE_COOKIE=false
WEBHOOK_URL=https://n8n.karloluiten.nl/
N8N_PROXY_HOPS=1

Reverse Proxying WebSockets through mod_proxy — HTTP Failback – Lisa's Home Page

I’ve been successfully reverse proxying MQTT over WebSockets via Apache HTTPD for quite some time now. The last few weeks, my phone isn’t able to connect. There’s no good rational presented (and manually clicking the “send data” button on my client successfully connects). It was time to upgrade the server anyway. Once I got the latest Linux distro on the server, I couldn’t connect at all to my MQTT server. The error log showed AH00898: Error reading from remote server returned by /mqtt

Evidently, httpd 2.4.47 added functionality to upgrade and tunnel in accordance with RFC 7230. And that doesn’t work at all in my scenario. Haven’t dug in to the why of it yet, but adding ProxyWebsocketFallbackToProxyHttp Off to the reverse proxy config allowed me to successfully communicate with the MQTT server through the reverse proxy.

add chapter titles to mkv automatically

bgstack15 — Wed, 26 Jun 2024 12:55:00 GMT

Here's an old one I haven't used in years apparently, because the shebang was /bin/sh but it requires bash, so from my Fedora desktop days.

files/2024/listings/chapter-titles.sh (Source)

#!/bin/bash
# utility to simplify entering chapter titles into MKVToolNix GUI
# startdate: 2019-04-07 20:26
# dependencies:
#    INFILE needs one line per chapter. Each line is the exact contents of the chapter title.
#    xdotool

test -z "${INFILE}" && INFILE=/mnt/public/Video/chapters.txt
test -n "${1}" && INFILE="${1}"

echo "Using INFILE=${INFILE} with chapter titles:"
head -n3 "${INFILE}"
printf "%s\n" "..."
tail -n3 "${INFILE}"
echo "Press enter to start the 5-second countdown, and then move the cursor to the MKVToolNix window and Name field."
read foo
sleep 5

x=0
while read -u10 line ;
do
   # in case you want to count them
   x=$(( x + 1 ))
   echo "line=${line}"
   xdotool type --delay 12ms "${line}"
   xdotool key --delay 12ms "Return" "Return"
done 10< "${INFILE}"

echo "done!"

So while it's kind of a oneliner, you have to prepare your input file with the chapter names. That is left as an exercise for the reader.

To use this, you have to have the input file prepared, and you have to be running mkvtoolnix-gui already. (I never learned how to do with with mkvtoolnix cli which probably renders this whole script moot.) Open your mkv file on the chapter tab. Then in a nearby terminal, run:

chapter-titles.sh ~/chapters.txt

It will tell you to press enter, and then you have 5 seconds to position the cursor at the chapter 1 Name field.

Ironic that I go to all this effort, but only one of my video players (vlc, and not mpv or Jellyfin) seems to show chapter titles so why do I bother? Maybe that's why I haven't touched this script in 5 years. I got the itch to add all the metadata things to a file and got carried away, and was pleasantly surprised to learn I needed to describe it here!

Fix old git commit author info

bgstack15 — Fri, 17 Feb 2023 14:00:40 GMT

To learn which repos need the info corrected:

cd ~/dev/sync-git/
for word in * ; do ( cd "${word}" ; git log --pretty=format:'%an,%ae' | sort -u | sed -r -e "s/^/${word},/;" ; ) ; done | grep 'localhost|example' ~/fix3

I inspected ~/fix3 and found two projects that had incorrect email addresses.

Now, prepare a script, ~/fix-git-commits.sh

	`#!/bin/sh`
	`# Startdate: 2023-01-30-2 14:00`
	`# ripped from https://www.git-tower.com/learn/git/faq/change-author-name-email/`
	`git filter-branch --env-filter '`
	`WRONG_EMAILS="localhost"`
	`NEW_NAME="B. Stack"`
	`NEW_EMAIL="bgstack15@gmail.com"`
	`if echo "${GIT_COMMITTER_EMAIL}" \| grep -qE "${WRONG_EMAILS}" ;`
	`then`
	`export GIT_COMMITTER_NAME="${NEW_NAME}"`
	`export GIT_COMMITTER_EMAIL="${NEW_EMAIL}"`
	`fi`
	`if echo "${GIT_AUTHOR_EMAIL}" \| grep -qE "${WRONG_EMAILS}" ;`
	`then`
	`export GIT_AUTHOR_NAME="${NEW_NAME}"`
	`export GIT_AUTHOR_EMAIL="${NEW_EMAIL}"`
	`fi`
	`' --tag-name-filter cat -- --branches --tags`

In the two repositories that needed to have info corrected, run this script.

cd ~/dev/project1
sh ~/fix-git-commits.sh

Then, you can just force-push each branch to each remote.

I also had to manually re-tag each tag. I just matched exact commit time and manually ran:

git tag --force v2.1 330b3e91f357f54668c54d5e543a7f3138b77ad7

And then I force-pushed the tags to each remote.

git push --tags local --force
git push --tags cloud --force

Future reading

Gitlab has some extra steps I should bother to do, but I haven't yet: https://docs.gitlab.com/ee/user/project/repository/reducing_the_repo_size_using_git.html

References

Slightly modified from https://www.git-tower.com/learn/git/faq/change-author-name-email/

Split single mp3 of whole CD

bgstack15 — Sat, 31 Dec 2022 13:35:19 GMT

I had a single mp3 recording of an entire CD that I wished to split into the individual tracks. Thankfully, people smarter than me have already solved this problem on the Internet.

The first order of business was to find the track times. Thankfully, musicbrainz.org delivered TOC info. It looks like this but I wasn't able to figure how to download a cddb-style TOC file, but that's OK. I used Firefox's CTRL+click-and-drag functionality to select just a column of the start times of the tracks, and pasted them in a file.

I ended up having to subtract lots of seconds, because my recording was somehow skewed. I also remember that ffmpeg can only be so precise with seconds (due to how compression works?) so that affected things to. So my track times ended up a little different than the TOC I used.

File ./times

00:02:21
00:04:32
00:08:32
00:12:25
00:15:12
00:19:14
00:22:53
00:25:52
00:29:58
00:35:03
00:39:27
00:42:48
00:46:54
00:48:54
00:53:33
00:58:17
01:02:44

Notice how I had to add the end time of the last track! Armed with these times, I used a small script from Unix.SE, of course.

split.sh (Source)

#!/bin/bash
# Adapted from https://unix.stackexchange.com/questions/659789/automating-the-splitting-of-a-large-mp3-file-with-ffmpeg-into-multiple-files-in
x="00:00:00"
z=0
filename=$(basename -- "$2")
ext="${filename##*.}"
filename="${filename%.*}"
initcmd="ffmpeg  -nostdin -hide_banner -loglevel error -i $2"
while read y ; do
   initcmd+=" -ss $x -to $y -c copy $filename$z.$ext"
   let "z=z+1"
   x=$y 
done < $1
${initcmd}

Then I ran the script:

./split.sh ./times album.mp3

I had to subtract lots of seconds, smeared across the whole track. I suspect the whole recording was the tiniest bit fast, but I also remember that ffmpeg plays fast and loose with relative time specifications.

But now, I had an mp3 file for each track! Then I used puddletag and its "Tag Sources" window to load the musicbrainz information for these songs.

My dvd ripping solution, 2022 edition

bgstack15 — Sat, 10 Sep 2022 13:16:48 GMT

Last year, I wrote about my dvd-ripping solution. Of course, over time my process has improved and evolved.

Ripping the DVDs

I now use makemkv again, because Handbrake itself suggests that other tools are better for ripipng from removable media. Handbrake is for transcoding.

When I stick discs in my (3) drives, I then run a command inside GNU screen.

time APPLY=1 INPUT=/dev/sr0:/dev/sr1:/dev/sr2 JOBNAME=s3d6 sh -x makemkv.sh

The jobname is arbitrary and helps keep job outputs distinct when I let them pile up. Here is the script makemkv.sh.

	`#!/bin/sh`
	`# Startdate: 2021-10-02 19:21`
	`# Purpose: rip disc automatically`
	`# History:`
	`# This is the second version of makemkv.sh (first one was renamed to makemkv-2019.sh)`
	`# 2022-08-03 13:11 add unshare to keep makemkv from accessing the network`
	`# Reference:`
	`# /mnt/public/Support/Programs/DVDs/handbrake-internal.sh`

	`# Flow: use makemkv to make the original rip, and then handbrake to convert to the preferred size/format.`

	`test -z "${INPUT}" && INPUT="/dev/sr0" # colon-delimited`
	`test -z "${OUTPUTDIR}" && OUTPUTDIR=/mnt/public/Video/temp/makemkv`
	`test -z "${JOBNAME}" && JOBNAME="$( < /dev/urandom tr -dc 'A-Z0-9' \| head -c6 )"`
	`test -z "${step1file}" && step1file="${OUTPUTDIR}/1-${JOBNAME}-run-makemkv.sh"`
	`test -z "${step2file}" && step2file="${OUTPUTDIR}/2-${JOBNAME}-run-handbrake.sh"`
	`printf '' > "${step1file}"`
	`printf '' > "${step2file}"`

	`for line in $( echo ":${INPUT}:" \| tr ':' '\n' \| grep -viE '^\s*$' ) ;`
	`do`
	`bline="$( basename "${line}" )"`
	`echo "process ${line}"`
	`mminfo="$( unshare -r -n makemkvcon info dev:"${line}" )"`
	`#echo "${mminfo}" >> "${TMPFILE}"`
	`count="$( echo "${mminfo}" \| sed -r -e '1,/Total/d' \| grep -c Title )"`
	`raw="$( HandBrakeCLI --markers --format mkv --min-duration 125 --scan --input "${line}" --output "${OUTPUT}" --encoder x264 --rate 30 --native-language eng --title 0 2>&1 )"`
	`disctitle="$( echo "${raw}" \| sed -n -r -e '/DVD Title:/{s/.*DVD Title: //;p}' )"`
	`x=0`
	`while test $x -lt $count ;`
	`do`
	`OUTDIR="${OUTPUTDIR}/${JOBNAME:+${JOBNAME}_}${bline}/${disctitle}_${x}" ; OUTDIR="$( echo "${OUTDIR}" \| tr ' ' '_' )"`
	`#mkdir -p "${OUTDIR}"`
	`#echo makemkvcon mkv "dev:${line}" "${x}" "${OUTDIR}"`
	`echo "mkdir -p \"${OUTDIR}\" ; unshare -r -n makemkvcon mkv \"dev:${line}\" \"${x}\" \"${OUTDIR}\"" >> "${step1file}"`
	`newfile="$( find "${OUTDIR}" -mindepth 1 -maxdepth 1 ! -type d -print -quit 2>/dev/null )"`
	`test -z "${newfile}" && newfile="${OUTDIR}/*.mkv"`
	`outfilename="$( echo "${OUTPUTDIR}/${JOBNAME:+${JOBNAME}_}${bline}_${disctitle}v${x}.mkv" \| tr ' ' '_' )"`
	`echo "HandBrakeCLI --markers --format mkv --input "${OUTDIR}"/*.mkv --output \"${outfilename}\" --encoder x264 --rate 30 --native-language eng --mixdown 6ch --aencoder ffaac --audio 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15, --subtitle 0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,scan" >> "${step2file}"`
	`x=$((x+1))`
	`done`
	`done`

	`# So after all the prep work is done, we need to run step 1 and then step 2`
	`echo "BEGIN STEP 1"`
	`if test -n "${APPLY}" ;`
	`then`
	`sh -x "${step1file}"`
	`else`
	`cat "${step1file}"`
	`fi`
	`echo "####################################################################"`
	`echo "####################################################################"`
	`echo "You may now eject the disc(s)."`
	`echo "####################################################################"`
	`echo "####################################################################"`
	`echo "BEGIN STEP 2"`
	`if test -n "${APPLY}" ;`
	`then`
	`sh -x "${step2file}"`
	`else`
	`cat "${step2file}"`
	`fi`
	`echo "DONE"`
	`date`

Once that is done, the raw ripped files are stored at glob /mnt/public/Video/temp/makemkv/JOBNAME_sr*/*/*.mkv, and the handbrake-transcoded files are at glob /mnt/public/Video/temp/makemkv/JOBNAME_sr*_namefromdisc.mkv.

Now I manually inspect each video file and rename them to the correct episode name, or extra file name (which probably won't be updated in the next step).

For show episodes, I run a script which I described in that previous post. I have an input csv which I hand-curate from the Wikipedia pages usually, that includes columns s, sep, ep, airdate, filename.

have,s,ep,sep,title,airdate,filename
1,1,1,01-e02,Encounter at Farpoint,1987-09-28,s01e01-e02 - Encounter at Farpoint
1,1,3,3,The Naked Now,1987-10-05,s01e03 - The Naked Now
1,1,4,4,Code of Honor,1987-10-12,s01e04 - Code of Honor
1,1,5,5,The Last Outpost,1987-10-19,s01e05 - The Last Outpost
1,1,6,6,Where No One Has Gone Before,1987-10-26,s01e06 - Where No One Has Gone Before

I describe my csv generation process a little more in that previous post.

Then I run the next script:

time tv-mkv-helper.py --inputcsv "/mnt/public/Video/TV/Star Trek The Next Generation (1987)/STTNG.csv" -d /mnt/public/Video/temp/

This script is still exactly the same as in the previous post. I like the metadata of the original airdate on the episode, as well as of course video title. And I have never found a use for the "next file" tags that mkvtoolnix can show you, but I populate them because I can. Maybe it'll be useful someday.

And then I manually move the files to their intended destinations.

My DVD-ripping solution

bgstack15 — Thu, 12 Aug 2021 13:14:52 GMT

This is a work in progress, but its current status is as follows.

Ripping the DVDs

To rip my DVD collection, use my handbrake wrapper script. My ripping machine uses two disc drives. Run the script once per drive, and save all output to a temporary file.

{ INPUT=/dev/sr0 handbrake.sh ; INPUT=/dev/sr1 handbrake.sh ; } > ~/handbrake1

Inspect the script for correctness. Mine looks like:

HandBrakeCLI --markers --format mkv --min-duration 125 --input /dev/sr0 --output /mnt/public/Video/temp/TNGS2D3_V1.mkv --encoder x264 --rate 30 --native-language eng --title 1 --audio 1,2, --subtitle 1,2,scan --native-language eng --mixdown 6ch --aencoder ffaac
HandBrakeCLI --markers --format mkv --min-duration 125 --input /dev/sr0 --output /mnt/public/Video/temp/TNGS2D3_V2.mkv --encoder x264 --rate 30 --native-language eng --title 2 --audio 1,2, --subtitle 1,2,scan --native-language eng --mixdown 6ch --aencoder ffaac

The handbrake.sh script can be found at the bottom of this post and is explained further there. Run the temp file's commands. I like to do this in a GNU screen session.

time sh -x ~/handbrake1

Adding TV show metadata

Prepare a csv with s, sep, ep, airdate, title, and filename columns.

have,s,ep,sep,title,airdate,filename
1,1,1,01-e02,Encounter at Farpoint,1987-09-28,s01e01-e02 - Encounter at Farpoint
1,1,3,3,The Naked Now,1987-10-05,s01e03 - The Naked Now
1,1,4,4,Code of Honor,1987-10-12,s01e04 - Code of Honor
1,1,5,5,The Last Outpost,1987-10-19,s01e05 - The Last Outpost
1,1,6,6,Where No One Has Gone Before,1987-10-26,s01e06 - Where No One Has Gone Before

This filename output is supposed to be what Jellyfin wants for TV shows. I used the information from Wikipedia and some LibreOffice Calc functions to make this csv file, and some hand-munging for the double-long episodes.

="s" & TEXT(A2,"\00") & "e" & TEXT(C2,"00") & " - " & REGEX($D2,"[,!']","")

I'm a bit obsessive about removing commas, exclamation marks, question marks, and quote marks from my filenames. Inspect the saved files and rename each one to the filename string from the csv that corresponds to the correct episode. To add the mkv metadata including airdate, and next and previous episodes, run the next script on the directory. This does not currently recurse into subdirectories.

time tv-mkv-helper.py --inputcsv "/mnt/public/Video/TV/Star Trek The Next Generation (1987)/STTNG.csv" -d /mnt/public/Video/temp/

Alternatives

Automatic Ripping Machine is just too fully-featured for me.

References

Jellyfin TV show naming scheme

Scripts

handbrake.sh

My handbrake.sh wrapper reads the disc contents to learn the disc title and the number of video tracks longer than 125 seconds.

	`#!/bin/sh`
	`# startdate: 2021-03-10 11:36`
	`# Alternatives:`
	`# handbrake preset: /mnt/public/Support/Programs/DVDs/1920x1080.json`

	`test -z "${INPUT}" && INPUT=/dev/sr0 ; export INPUT`
	`test -z "${OUTPUT}" && OUTPUT=/mnt/public/Video/temp ; export OUTPUT`

	`raw="$( HandBrakeCLI --markers --format mkv --min-duration 125 --scan --input "${INPUT}" --output "${OUTPUT}" --encoder x264 --rate 30 --native-language eng --title 0 2>&1 )"`
	`disctitle="$( echo "${raw}" \| sed -n -r -e '/DVD Title:/{s/.*DVD Title: //;p}' )"`
	`array="$( echo "${raw}" \| grep -E '^\s*\+' \| awk 'BEGIN{x=0} /title [0-9]/{gsub(":","",$NF);x++;title[x]=$NF} /kbps/{audio[x]=audio[x]""$2} !/subtitle tracks:/ && !/\+ [0-9]+,/ {us=0} /subtitle tracks:/{us=1} us == 1 && $2 ~ /[0-9]+,/{subtitle[x]=subtitle[x]""$2} END {for(i in title) {{print title[i],audio[i],subtitle[i]}}}' )"`
	`commands="$( echo "${array}" \| awk -v "disctitle=${disctitle}" -v "input=${INPUT}" -v "output=${OUTPUT}" '{a=$1;b=$2;c=$3;print "HandBrakeCLI --markers --format mkv --min-duration 125 --input "input" --output "output"/"disctitle"_V"a".mkv --encoder x264 --rate 30 --native-language eng --title "a" --audio "b" --subtitle "c"scan --native-language eng --mixdown 6ch --aencoder ffaac"}' )"`
	`echo "${commands}"`

My hard-coded values include outputting to mkv, and x264 video encoding, and 30 bps, with English language. I also tell it to use the specific audio encoder ffaac which was the first one I tried when trying to get my 6-channel mixdown (that's 5.1 surround sound) option to work. I'm not a media bitrate snob; just trying to preserve some semblance of fancier file value than I can even consume, should I ever upgrade my viewing equipment in the next 40 years.

tv-mkv-helper.py

	`#!/usr/bin/env python3`
	`# Startdate: 2021-08-07`
	`# Purpose: given INCSV="/mnt/public/Video/TV/Star Trek The Next Generation 1987)/STTNG.csv"`
	`# with columns s, ep, sep, title, airdate, filename,`
	`# update files in provided directory, whose name matches the filename column, with the mkv properties`
	`# Usage:`
	`# time tv-mkv-helper.py --inputcsv "/mnt/public/Video/TV/Star Trek The Next Generation (1987)/STTNG.csv" -d /mnt/public/Video/temp/`
	`# References:`
	`# /mnt/public/Support/Programs/DVDs/handbrake-internal.sh`
	`# Dependencies:`
	`# STTNG.csv sorted ascending, with columns s, sep, title, airdate, filename`
	`# mkvpropedit`

	`import csv, os, subprocess`

	`###############################################################`
	`def load_file_to_array(inputcsv):`
	`episode_array = []`
	`with open(inputcsv,"r") as o:`
	`csv_reader = csv.reader(o)`
	`headers = next(csv_reader)`
	`error_string = ""`

	`# sanity check for the columns we want`
	`if "s" not in headers:`
	`error_string = "Inputcsv needs column named 's' for season number."`
	`#elif "ep" not in headers:`
	`# error_string = "Inputcsv needs column named 'ep' for episode number."`
	`elif "sep" not in headers:`
	`error_string = "Inputcsv needs column named 'sep' for season episode number."`
	`elif "airdate" not in headers:`
	`error_string = "Inputcsv needs column named 'airdate'."`
	`elif "filename" not in headers:`
	`error_string = "Inputcsv needs column named 'filename'."`
	`if error_string != "":`
	`print(error_string)`
	`return -1`

	`x = 0`
	`for row in csv_reader:`
	`x = x + 1`
	`ep = {key: value for key, value in zip(headers,row)}`
	`episode_array.append(ep)`
	`return episode_array`

	`############################################################`
	`def fix_directory_contents(directory, inputcsv):`
	`os.chdir(directory)`
	`episodes = load_file_to_array(inputcsv)`
	`if episodes == -1:`
	`print("Aborting.")`
	`return -1`
	`# simple one-level down method. This might be insufficient someday.`
	`for filename in os.listdir(directory):`
	`if filename.endswith(".mkv"):`
	`print("----------------------")`
	`print(f"Checking file {filename}")`
	`filename_sans_ext, _ = os.path.splitext(filename)`
	`# loop through episodes to find this file`
	`# need enumerate, per https://stackoverflow.com/questions/1011938/loop-that-also-accesses-previous-and-next-values/1011962#1011962`
	`l = len(episodes)`
	`for index, ep in enumerate(episodes):`
	`previous = next_ = None`
	`if ep['filename'] == filename_sans_ext:`
	`if index > 0:`
	`previous = episodes[index-1]['filename']`
	`if index < ( l - 1 ):`
	`next_ = episodes[index+1]['filename']`
	`fix_file(`
	`filename=filename,`
	`title=ep['filename'],`
	`season=ep['s'],`
	`seasonep=ep['sep'],`
	`next_name=next_,`
	`prev_name=previous,`
	`airdate=ep['airdate']`
	`)`

	`############################################`
	`def fix_file(filename,title,season,seasonep,airdate,next_name, prev_name):`
	`print(f"Please fix {filename} to have")`
	`print(f"title: {title}")`
	`print(f"season: {season}")`
	`print(f"seasonep: {seasonep}")`
	`print(f"airdate: {airdate}")`
	`print(f"next_name: {next_name}")`
	`print(f"prev_name: {prev_name}")`
	`if next_name is not None and not next_name.endswith(".mkv"):`
	`next_name = next_name + ".mkv"`
	`if prev_name is not None and not prev_name.endswith(".mkv"):`
	`prev_name = prev_name + ".mkv"`
	`date_string = str(airdate) + "T12:00:00Z" # just set it to noon UTC, so the EST/EDT time will still be the same day.`
	`#cmd_string = f"mkvpropedit --set 'title={title}' --set 'date={date_string}' --set 'segment-filename={filename}.mkv' --set 'prev-filename={prev_name}' --set 'next-filename={next_name}'"`
	`cmd_array = ["mkvpropedit","--set",f'title={title}',"--set",f'date={date_string}',"--set",f'segment-filename={filename}',"--set",f'prev-filename={prev_name}',"--set",f'next-filename={next_name}',filename]`
	`result = subprocess.run(cmd_array)`
	`print(result)`
	`aux_cmd_array=["mkvpropedit",filename,"--delete"]`
	`if next_name is None or next_name == "":`
	`aux_cmd_array.append("next-filename")`
	`if prev_name is None or prev_name == "":`
	`aux_cmd_array.append("prev-filename")`
	`# if we added anything to the auxiliary command array, then run it`
	`if aux_cmd_array[-1] != "--delete":`
	`result = subprocess.run(aux_cmd_array)`
	`print(result)`

	`# argparse stuff`
	`import argparse`
	`parser = argparse.ArgumentParser()`
	`parser.add_argument("-i","--inputcsv",required=True,help="Episode CSV, with columns s, sep, title, filename, airdate")`
	`parser.add_argument("-d","--directory",required=True,help="Path to directory where files should be given metadata")`
	`args = parser.parse_args()`

	`inputcsv = args.inputcsv`
	`directory = args.directory`

	`if __name__ == "__main__":`
	`print(f"running with inputcsv=\"{inputcsv}\", directory={directory}")`
	`fix_directory_contents(directory,inputcsv)`

In a departure from my normal style where I put all the functions in a library and call it, I wanted a self-contained script with the argument parsing. It was quick and dirty. And all I'm doing is adding the 3 or 4 mkv attributes to the mkv files whose filename matches an entry in the csv. I do all this using a custom csv, because I'm too lazy to learn how to use some fancy TheTVDB api or whatever. I'm obsessive enough to be fine with manually scraping metadata from Wikipedia tv show episode lists.

Building an apt repo on CentOS that supports apt-file operations

bgstack15 — Thu, 05 Nov 2020 13:38:53 GMT

This is a newer version of Building an apt repository on CentOS

Overview

My network infrastructure consists of CentOS 7 systems and Devuan client systems. I maintain mirrors of Devuan Ceres locally so I only need one outbound operation. In addition to the official repos (albeit stored in an unofficial manner), I also maintain my own collections of packages, as the first link in this post describes. One thing I noticed though, is that apt-file does not operate on the packages in my own repositories. So I used the wonderful free and open source nature of all the great tools that make up Devuan's apt software, and wrote my own set of tools that add the support for apt-file. Adding the apt-file support requires modifications to the server, as well as the clients. This is acceptable because I of course am the admin on my own systems and can add /etc contents at will.

Configuring the repo server

On the CentOS 7 server, you need to generate gpg keys (see Weblink 1). I have elected to store the passphrase in plaintext (bottom of Weblink 1) as file /root/.gnupg/passphrasefile

Repo update script

Write the wrapper script. Looking back I realize I should have set this up as a config file and then I just need to invoke the lib script as the main script, with parameters, but I'll save that for a future refactoring.

#!/bin/sh
# Filename: update-exampledeb.sh
# Location: /mnt/public/www/example/repo/deb
# License: CC-BY-SA 4.0
# Author: bgstack15
# Startdate: 2017-07-23
# Title: Script that updates apt repo "exampledeb"
# Purpose: automate rebuilding the repo
# History:
#    2020-10-23 adding apt-file compatibility.
# Usage:
#    just call this after adding a new package to the repository
# Reference:
# Improve:
# Documentation:
# Dependencies:

# Set variables
export repodir=/mnt/public/www/example/repo/deb/
export ownership="apache:admins"
export filetypes="deb"
export gpgkey_passphrase_file="/root/.gnupg/passphrasefile"
test -z "${SKIP_SCAN}" && export SKIP_SCAN=0
test -z "${SKIP_CONTENTS}" && export SKIP_CONTENTS=0

# load library, which validates the above variables
. /mnt/public/www/example/repo/deb/scripts/apt-repo-lib.sh

# Prepare directory and files
fix_owner_and_mode

# Prepare repo for apt
make_apt_repo

# create the Release and InRelease files
make_release_files

Update repo library

All of the logic is in the dot-sourced library script.

#/bin/sh
# File: /mnt/public/www/example/repo/deb/scripts/apt-repo-lib.sh
# Location: server1
# License: CC-BY-SA 4.0
# Author: bgstack15
# Startdate: 2020-10-23
# Title: Library for apt repo update scripts
# Purpose: provide common functions for each apt repo to reduce code duplication
# History:
#    2020-10-22 the make_contents functions were started 
#    2020-10-23 this library started and took basically all logic out of the update script itself, because it is all boilerplate minus a few variables.
# Usage: only dot-source from an upate-*.sh repo update script.
# Reference:
#    /mnt/public/www/example/repo/deb/update-exampledeb.sh
# Improve:
#    actually bother to build content files per architecture.
# Documentation:
#    designed to run on CentOS 7 where not all apt tools exist.
# Dependencies:
#    gzip, gpg2

# Functions
_mc_inner() {
   word="${1}"
      name="$( dpkg-deb --show "${word}" | awk '{print $1}' )"
      # list dpkg contents and remove directories from listing
      dpkg-deb --contents "${word}" | awk '!/\/$/ {$1="";$2="";$3="";$4="";$5="";print}' | sed -r -e 's/^\s+//' | \
         # change listings to be relative paths, like reference Contents file, and append package name
         # this appears to be incomplete compared to reference Contents file because we do not include the "section" information, e.g., "contrib/admin/zfs-test"
         sed -r -e 's:^\./::' -e "s:$:\t${name}:;" ;
      # reset var just in case
      echo "DONE: ${word}" 1>&2
}

make_contents() {
   # call: make_contents "/path/to/dir" "reponame" "file"
   _path="${1:-.}"
   _reponame="${2}"
   test "." != "${_path}" && pushd "${_path}" 1>/dev/null 2>&1
   # use the . for path so total count of parameter characters is shorter
   for word in $( find . -name "*.deb" ! -name '*teamviewer*' ) ; do
      _mc_inner "${word}" &
   done | \
      # only show new entries, which hopefully will make sort faster. It is possible this is not useful.
      awk '!x[$0]++' | \
      # sort output and show only unique lines. This is required in my repos because I leave multiple versions of a single package around.
      sort | uniq
   popd 1>/dev/null 2>&1
}

fail() {
   _ec="${1}" ; shift 1 ;
   echo "${@}" ;
   return "${_ec}"
}

fix_owner_and_mode() {
   find "${repodir}" -exec chown "${ownership}" {} + 1>/dev/null 2>&1
   find "${repodir}" -type f ! -name '*.sh' -exec chmod "0664" {} + 1>/dev/null 2>&1
   find "${repodir}" -type f -name '*.sh'   -exec chmod "0754" {} + 1>/dev/null 2>&1
   find "${repodir}" -type d -exec chmod "0775" {} + 1>/dev/null 2>&1
   restorecon -RF "${repodir}"
}

make_apt_repo() {
   pushd "${repodir}" 1>/dev/null 2>&1
   if ! test "${SKIP_SCAN}" = "1" ;
   then
      dpkg-scanpackages -m . > Packages # this takes a long time to run
      gzip -9c Packages > Packages.gz
   fi
   if ! test "${SKIP_CONTENTS}" = "1" ;
   then
      make_contents . "" "file" > "${repodir}/Contents"
      wait
      cd "${repodir}"
      # apt-file needs -amd64.gz and -i386.gz files, based on default /etc/apt/apt.conf.d/50apt-file.conf
      for word in amd64 i386 all ; do cp -pf Contents "Contents-${word}" ; done
      for word in Contents* ; do ! echo "${word}" | grep -qE '\.gz$' && gzip -9c "${word}" > "${word}.gz" ; done
   fi
   popd 1>/dev/null 2>&1
}

make_release_files() {
   pushd "${repodir}" 1>/dev/null 2>&1
   # create the Release and InRelease files
   md5s="$( for word in Packages* Contents* ; do printf "%s %s\n" "$( md5sum "${word}" | cut -d" " -f1 )" "$( wc -c "${word}" )" ; done | column -t | sed -r -e 's/^/ /;' )"
   sha1s="$( for word in Packages* Contents* ; do printf "%s %s\n" "$( sha1sum "${word}" | cut -d" " -f1 )" "$( wc -c "${word}" )" ; done | column -t | sed -r -e 's/^/ /;' )"
   sha2s="$( for word in Packages* Contents* ; do printf "%s %s\n" "$( sha256sum "${word}" | cut -d" " -f1 )" "$( wc -c "${word}" )" ; done | column -t | sed -r -e 's/^/ /;' )"
   cat <<EOF > Release
Architectures: all
Date: $(date -u '+%a, %d %b %Y %T %Z')
MD5Sum:
${md5s}
SHA1:
${sha1s}
SHA256:
${sha2s}
EOF
   gpg --batch --yes --passphrase-file "${gpgkey_passphrase_file}" --pinentry-mode loopback -abs -o Release.gpg Release
   gpg --batch --yes --passphrase-file "${gpgkey_passphrase_file}" --pinentry-mode loopback --clear-sign -o InRelease Release
   popd 1>/dev/null 2>&1
}

# When dot-sourcing the library, validate all input parameters
test -z "${repodir}" && { fail 1 "Fatal! Need \"repodir\" defined." || exit 1 ; }
test -z "${ownership}" && { fail 1 "Fatal! Need \"ownership\" defined, nominally \"apache:admins\"." || exit 1 ; }
test -z "${filetypes}" && { fail 1 "Fatal! Need \"filetypes\" defined, nominally \"deb\"." || exit 1 ; }
test -z "${gpgkey_passphrase_file}" && { fail 1 "Fatal! Need \"gpgkey_passphrase_file\" defined." || exit 1 ; }

Putting it all together on the server

When I have a new package underneath /mnt/public/www/example/repo/deb/, I just need to run sudo /mnt/public/www/example/repo/deb/update-exampledeb.sh. Previously my separate repositories were using duplicates of a single script. But now, they can all reference this one library and the in-tree update script is basically an executable config file.

Configuring clients

My architecture of the server, with the flat repository style (i.e., without dists/ceres/main/contrib directories) made it difficult to configure clients to always fetch the Contents files which matter. The clients need the apt repo definitions adjusted, and a custom apt-file conf.d added. The apt repo, nominally file /etc/apt/sources.list.d/exampledeb.list, needs to look like:

deb [target-=Contents-deb target+=Contents-stackrpms] http://www.example.com/example/repo/deb/ /

The target commands tell apt to use the custom indexing definition from this next file. Configure apt with file /etc/apt/apt.preferences.d/52apt-file- stackrpms.conf

# File: /etc/apt/apt.preferences.d/52apt-file-stackrpms.conf
# Part of support devuan scripts
# This enables the flat apt repos in example to be supported by apt-file
Acquire::IndexTargets {
    deb::Contents-stackrpms {
        MetaKey "Contents-$(ARCHITECTURE)";
        ShortDescription "Contents-$(ARCHITECTURE)";
        Description "$(RELEASE) $(ARCHITECTURE) Contents (deb)";

        flatMetaKey "Contents-$(ARCHITECTURE)";
        flatDescription "$(RELEASE) Contents (deb)";
        PDiffs "true";
        KeepCompressed "true";
        DefaultEnabled "false";
        Identifier "Contents-deb";
    };
};

Looking back, I probably should have just learned how to use the built-in "Contents-deb-legacy" but why make things too simple? I accomplish the client configuration on my own network with a script that does a couple of extra things, but I will not cover everything that it does here.

#!/bin/sh
# File: /mnt/public/Support/Platforms/devuan/set-my-repos.sh
# Location:
# Author: bgstack15
# Startdate: 2019-08-10 16:02
# Title: Script that Establishes the repos needed for Devuan
# Purpose: Set up the 3 repos I always need on devuan clients
# History:
#    2020-02-01 customize clients for devuan-archive
#    2020-10-23 add apt-file compatibility
# Usage:
#    sudo set-my-repos.sh
# Reference:
#    /mnt/public/Support/Platforms/devuan/devuan.txt
# Improve:
#    need to control the sources.list file itself to have the main, contrib, etc., for ceres.
# Documentation:

test -z "${ALLREPOSGLOB}" && ALLREPOSGLOB="/etc/apt/sources.list /etc/apt/sources.list.d/*"
test -z "${REPOSBASE}" && REPOSBASE="/etc/apt/sources.list.d"
test -z "${PREFSBASE}" && PREFSBASE="/etc/apt/preferences.d"
test -z "${ADDLCONFBASE}" && ADDLCONFBASE="/etc/apt/apt.conf.d"

# confirm key
confirm_key() {
   # call: confirm_key "${PRETTYNAME}" "${SEARCHPHRASE}" "${URL_OF_KEY}"
   ___ck_repo="${1}"
   ___ck_sp="${2}"
   ___ck_url="${3}"
   if apt-key list 2>/dev/null | grep -qe "${___ck_sp}" ;
   then
      :
   else
      # not found so please add it
      echo "Adding key for ${___ck_repo}" 1>&2
      wget -O- "${___ck_url}" | sudo apt-key add -
   fi
}

# confirm repo
confirm_repo() {
   # call: confirm_repo "${PRETTYNAME}" "${SEARCHPHRASE}" "${SEARCHGLOB}" "${FULLSTRING}" "${PREFERRED_FILENAME}" "${OVERWRITE}"
   ___cr_repo="${1}"
   ___cr_sp="${2}"
   ___cr_sf="${3}"
   ___cr_full="${4}"
   ___cr_pref="${5}"
   ___cr_overwrite="${6}"
   if ! grep -E -qe "${___cr_sp}" ${___cr_sf} ;
   then
      # not found so please add it to preferred file
      echo "Adding repo ${___cr_repo}" 1>&2
      if test "${___cr_overwrite}" = "true" ;
      then
         # overwrite, instead of append
         echo "${___cr_full}" > "${REPOSBASE}/${___cr_pref:-99_misc.list}"
      else
         echo "${___cr_full}" >> "${REPOSBASE}/${___cr_pref:-99_misc.list}"
      fi
   fi
}

confirm_preferences() {
   # call: confirm_preferences "${PRETTYNAME}" "${FILENAME}" "{PACKAGE}" "${PIN_EXPRESSION}" "{PRIORITY}"
   ___cp_prettyname="${1}"
   ___cp_pref="${2}"
   ___cp_package="${3}"
   ___cp_pin_expression="${4}"
   ___cp_priority="${5}"

   ___cp_tempfile="$( mktemp )"
   {
      echo "Package: ${___cp_package}"
      echo "Pin: ${___cp_pin_expression}"
      echo "Pin-Priority: ${___cp_priority}"
   } > "${___cp_tempfile}"

   diff "${PREFSBASE}/${___cp_pref}" "${___cp_tempfile}" 1>/dev/null 2>&1 || {
      echo "Setting preferences for ${___cp_prettyname}"
      touch "${PREFSBASE}/${___cp_pref}" ; chmod 0644 "${PREFSBASE}/${___cp_pref}"
      cat "${___cp_tempfile}" > "${PREFSBASE}/${___cp_pref}"
   }

   rm -f "${___cp_tempfile:-NOTHINGTODEL}" 1>/dev/null 2>&1
}

# REPO 1: local exampledeb
confirm_key "exampledeb" "bgstack15.*example\.example\.com" "http://www.example.com/example/repo/deb/exampledeb.gpg"
confirm_repo "exampledeb" "target.*example\/repo\/deb" "${ALLREPOSGLOB}" "deb [target-=Contents-deb target+=Contents-stackrpms] http://www.example.com/example/repo/deb/ /" "exampledeb.list" "true"

# REPO 2: local devuan-deb
confirm_key "devuan-deb" "bgstack15.*example\.example\.com" "http://www.example.com/example/repo/deb/exampledeb.gpg"
confirm_repo "devuan-deb" "target.*example\/repo\/devuan-deb" "${ALLREPOSGLOB}" "deb [target-=Contents-deb target+=Contents-stackrpms] http://www.example.com/example/repo/devuan-deb/ /" "devuan-deb.list"

# REPO 3: local obs
#confirm_key "OBS bgstack15" "bgstack15@build\.opensuse\.org" "https://download.opensuse.org/repositories/home:bgstack15/Debian_Unstable/Release.key"
#confirm_repo "OBS bgstack15" "repositories\/home:\/bgstack15\/Debian_Unstable" "${ALLREPOSGLOB}" "deb http://download.opensuse.org/repositories/home:/bgstack15/Debian_Unstable/ /" "home:bgstack15.list"
confirm_key "OBS bgstack15" "bgstack15@build\.opensuse\.org" "http://www.example.com/mirror/obs/Release.key"
confirm_repo "OBS bgstack15" "mirror\/obs" "${ALLREPOSGLOB}" "deb http://www.example.com/mirror/obs/ /" "home:bgstack15.list"

# REPO 4: local devuan-archive
confirm_key "devuan-archive" "bgstack15.*example\.example\.com" "http://www.example.com/example/repo/deb/exampledeb.gpg"
confirm_repo "devuan-archive" "target.*server1((\.ipa)?\.example\.com)?(:180)?.*example\/repo\/devuan-archive" "${ALLREPOSGLOB}" "deb [target-=Contents-deb target+=Contents-stackrpms] http://server.ipa.example.com:180/example/repo/devuan-archive/ /" "devuan-archive.list"
confirm_preferences "devuan-archive" "puddletag" "*" "origin server1.ipa.example.com" "700"

# ADDITIONAL APT PREFS
# important for the [target] stuff to work on repos so apt-file can work
cp -p "$( dirname "$( readlink -f "${0}" )")/input/52apt-file-stackrpms.conf" "${ADDLCONFBASE}/"

Backstory

I want to be able to view the contents of my packages without having to install them first. Apt supports this, but it took half a day to discover how to generate the Contents file, get it listed in the Release/InRelease file, actually generate an InRelease file (the gpg-signed Release file), and get clients to pull down the Contents files. Coincidentally, apt clients store the Contents files compressed with lz4, and not gzip like apt repos tend to provide. Go figure. I found nothing on the Internet for using apt-file with flat apt repositories, so I am assuming this is an original concept. I'm guessing all the shops that bother with custom repos use one of those [DebianRepository Setup tools. I assume all those Debian-focused apt repo tools already take all these steps, but my limitation here was CentOS. It makes for a fun challenge, that is solvable within my skillset and thanks to the open source nature of these great tools.

References

Weblinks

Building an apt repository on CentOS gpg key instructions Package for CentOS 7: gnupg2-2.2.18-2.el7 newer version of gpg so the apt tools work in CentOS

Local files

/usr/share/doc/apt-file/README.md.gz

Manpages

apt-file(1)

Touch files to change modification time relative to original modification time

bgstack15 — Mon, 31 Dec 2018 14:20:01 GMT

If you are trying to correct an offset your digital camera has, or perhaps the filesystem has, you can fix it per file!

for word in *.jpg ;
do
   echo "${word}" ; touch -r "${word}" -d "+5 hours" "${word}"
done

References

Weblinks

Ripped off from https://superuser.com/questions/122863/linux-shell-change-a-files-modify-timestamp-relatively/122896#122896

Internet search

linux change modify time relative to modify time