Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

Linux: Mount android phone directories and open them in tabs

Last time I described the stolen script for opening multiple directories as tabs in Thunar. And now we will use that, to open the two useful directories from an mtpfs-mounted Android mobile phone.

At present, my solution depends on two shortcuts in the main application menu. Eventually I hope to write a udev rule (that works) to detect an mtp device connected, and automount and open the useful directories.

  • mount-mtp.desktop
  • umount-mtp.desktop

These files are pretty basic.

File /usr/share/applications/mount-mtp.desktop

[Desktop Entry]
Name=Connect phone and open photo dirs
Comment=After setting phone to "transfer files"
Exec=/usr/local/bin/mount-mtp.sh
Icon=phone
Type=Application
Terminal=false
Categories=System;Utility;

File /usr/share/applications/umount-mtp.desktop.

[Desktop Entry]
Name=Disconnect phone
Comment=Before unplugging phone
Exec=/usr/local/bin/umount-mtp.sh
Icon=phone
Type=Application
Terminal=false
Categories=System;Utility;

And obviously the script from last time, thunartab, which is described in the link at the top of this post.

Script /usr/local/bin/mount-mtp.sh

#!/bin/sh
# File: /usr/local/bin/mount-mtp.sh
# Author: bgstack15
# Startdate: 2022-08-21 16:22
# SPDX-License-Identifier: GPL-3.0
# Title: Mount and open phone camera directories
# Purpose: Make it easy to mount
# History:
# Usage:
#    run this from the .desktop file
# References:
#    https://wiki.archlinux.org/title/Udev#udev_rule_example
#    https://forums.linuxmint.com/viewtopic.php?t=244076
#    udevadm monitor --udev --environment
# Improve:
#    Add udev rule to run this on detection of an android or mtp device?
#    Add yad message if mtpfs is not mounted at the end
# Dependencies:
#   /usr/local/bin/thunartab
/usr/local/bin/umount-mtp.sh
if test "${USER}" != "user1" ;
then
    su user1 - mtpfs /home/user1/PHONE
else
    mtpfs /home/user1/PHONE
fi
:
if test "${USER}" = "user1" ;
then
    sleep 1 ; mount | grep -i PHONE && {
        thunar "/home/user1/PHONE/Internal shared storage/DCIM/Camera/" &
        sleep 10 ;
        thunartab "/home/user1/PHONE/SanDisk SD card/DCIM/Camera/" &
    }
fi

Script /usr/local/bin/umount-mtp.sh

#!/bin/sh
# Usage:
#    run this from the .desktop file
if test "${USER}" != "user1" ;
then
    su user1 -c 'fusermount -u /home/user1/PHONE'
else
    fusermount -u /home/user1/PHONE
fi
:

Open directory in new tab in Thunar

This is a direct rip-off of [Tutorial] Improving Thunar in XFCE by opening new tabs instead of windows with Bash. - Linux Mint Forums. A very enterprising individual named happysadhu has produced a fantastic script that will make future blog posts easier (ahem).

If you want to open multiple directories in Thunar, in tabs and not separate windows, there's not a programmatic way to do that natively. But if you set up a script named /usr/local/bin/thunartab with the following contents, you can then run the following commands.

#!/bin/bash
# Script to open new file manager tab instead of window (e.g., Thunar) in XFCE.
# Version 1. 2017-04-18
# Lock script to process multiple instances consecutively when "Open All" on Desktop.
if [[ $1 == "file:///home/$USER/Desktop/"* ]]; then
    ME=`basename "$0"`;
    LCK="/tmp/${ME}.LCK";
    exec 8>$LCK;
    flock -x 8;  # lock $LCK
    trap 'rm -f $LCK' exit # remove $LCK regardless of exit status
    sleep .1 # hack to give time for each script to execute properly--weird!
fi
# Convert desktop file urls to thunar-friendly local paths.
# Accommodates special characters in directory names (!@#$, etc).
fileurl=$1
filepath="$(urlencode -d ${fileurl#file://})/"
# Check for running instances of $app on current desktop/workspace.
app=thunar
wid=( $(xdotool search --desktop $(xdotool get_desktop) --class $app) )
lastwid=${wid[*]: -1} # Get PID of newest active thunar window.
# If $wid is null launch app with filepath.
if [ -z $wid ]; then
    $app "$filepath"
# If app is already running, activate it and use shortcuts to paste filepath into path bar.
else  
    xdotool windowactivate --sync $lastwid key ctrl+t ctrl+l # Activate pathbar in thunar
    xdotool type -delay 0 "$filepath" # "--delay 0" removes default 12ms between each keystroke
    xdotool key Return
fi
exit 0
# Easy Installation:
# cd ~/Downloads && tar -zxvf thunartab.tar.gz && sudo chmod 755 thunartab && sudo cp thunartab /usr/local/bin/
# Install dependencies: sudo apt install gridsite-clients xdotool # gridsite-clients contains urlencode
# Reboot # if you want to run "thunartab" as a command in terminal
# Select script in the XFCE's GUI "Preferred Applications"
#
# For alternatives for script location run "echo $PATH" (without quotes) in terminal
# Troubleshooting: If more than one thunar window opens with "Open All", try increasing sleep value on line 12
#
# For other file-managers (e.g., Nemo, Caja, Nautilus), change app name on line 21
# You may also have to modified  the xdotool shortcuts on line 31.
# For example for Nemo, change line 31 to something like
# xdotool windowactivate --sync $lastwid key ctrl+t ctrl+l ctrl+v Return ctrl+l Escape
# Manually test shortcuts in file manager first.
#
# Main sources for learning bash for this script include:
# http://ryanstutorials.net/bash-scripting-tutorial/ "Bash Scripting Tutorial" (Very good for beginners like myself"
# http://jdimpson.livejournal.com/5685.html "Using flock to protect critical sections in shell scripts"
# https://askubuntu.com/questions/55656/open-nautilus-as-new-tab-in-existing-window (the starting idea for this script)
#
# Bug reports or suggestions, please email me, Sam, at sampub@islandroots.ca

The commands to open multiple directories would then be:

thunar /path/to/first
thunartab /path/to/second
thunartab /here/is/another

Setting up remote server, bgstack15-style

I have previously described some of these tasks in an old post, but this is a single section of steps, updated!

When I set up a remote system I want to have a connection to it so I can control and administer it. I set up two paths to it:

  1. autossh from $NEWSERVER back to $OLDSITE
  2. wireguard vpn connection

Install wireguard and autossh. Additionally I used resolvconf because it makes wireguard control dns better. That might resemble:

sudo apt-get install wireguard autossh resolvconf

Establish autossh

Create a user for this purpose and generate an ssh key.

sudo useradd --create-home --shell /bin/bash autossh
sudo passwd autossh
sudo su autossh -c 'ssh-keygen'
sudo su autossh
# as user autossh:
ssh-copy-id -p 2022 autossh@www.example.com

Make a new system service with either an init file or unit file.

Restart the system service!

Establish wireguard

And for wireguard, establish the settings to connect my relevant nodes. Select an available IP address from "IP space map - Internal.ods" file. Establish file /etc/wireguard/wg0.conf like below.

[Interface]
Address = 10.222.0.102/24
ListenPort = 51820
# from `wg genkey`
PrivateKey = SCRUBBED
# this system  public key
# from `echo $PrivateKey | wg pubkey`
# SCRUBBED
# If I need dns servers and search domains
DNS = 192.168.1.10,192.168.1.11, ipa.internal.com, vm.internal.com, internal.com
[Peer]
# first main peer
PublicKey = KOQVWMYb+TMzkMrCSsG7DJm29wQFovEV1LfKrptfAjw=
AllowedIPs = 192.168.1.10/32, 192.168.1.11/32, 192.168.1.12, 192.168.1.15/32, 10.222.0.0/24
PersistentKeepalive = 25
Endpoint = www.example.com:51820
[Peer]
# second main peer
PublicKey = aReyDUOGHqhhnqyUJQltfuWw+JoG+KES8DzD1k3CNWE=
AllowedIPs = 10.222.0.3/32
PersistentKeepalive = 25
Endpoint = secondary.ddns.net:51820

And of course, add this new peer to both the primary and secondary wireguard nodes.

[Peer]
# new system comment
PublicKey = +gJ2m3vJmIQzR7AfmBNq6+8+y9gWlISeCsuCgEGvPTM=
AllowedIPs = 10.222.0.102/32
# If needed:
PersistentKeepalive = 25
Endpoint = location.remote.example.com:51820

Start wireguard. If on a non-systemd distro, use a wireguard init script.

sudo update-rc.d wireguard defaults
sudo service wireguard start
# for systemd:
sudo systemctl enable wg-quick@wg0.service
sudo systemctl start wg-quick@wg0.service

Optionally, set up new A record under remote.example.com on server1 with:

updatezone remote.example.com

Infcloud: auto login my way

I have now added to InfCloud the ability to use the browser localStorage feature to store username, password, and chosen locale. To be more precise, it can load these values if present. InfCloud does not set them yet. I haven't bothered to write that part.

To use this feature, you need to run a few commands in the javascript console of your web browser. Not all of my browsers work correctly, even though localStorage object exists.

localStorage.setItem("user","bgstack15");
localStorage.setItem("pass","plaintexthere");
localStorage.setItem("locale","English (24h)");

The locale value is the contents of the text box, and not the shortcode such as en_BS.

If these values are set, when you refresh the page it should auto-login right away.

See the commit that includes the functionality.

Infcloud with event import feature

I've previously discussed my self-hosted calendar solution, and I have an update for it!

I have now added a working upload feature! See commit 81431ce6cce507535d871020fcb4c9b3844ffd4a. I spent a long time to add these very short lines.

Now, in the new-event form, you can drag-and-drop a .ics file onto the "Import" button, and it will add the new event for you! I was not aware of any previous import/upload feature in InfCloud, so I wrote my own.

I started with duplicates of multiple, large internal functions. In the end though, it was a lot shorter than I thought it would be.

Thinkpad X230 Tablet: use headset microphone

I investigated using my combined headset (headphones and microphone) jack on my Lneovo Thinkpad X230 Tablet. The headphones would work but not the microphone input. Ultimately, what I got to work was rather simple!

File /usr/local/bin/mutemic sends a little graphical popup to indicate the microphone status.

#!/bin/sh
# File: /usr/local/bin/mutemic
# Location: LTB-019
# Author: bender, bgstack15
# Startdate: 2022-08-07
# Title: Mutemic script
# Purpose: Handle mute-mic button on Thinkpad X230 Tablet
# Project: X230T-headset
# History:
# Usage:
#    Called from /etc/acpi/events/mutemic
#    Learn device name with `amixer scontrols`
# Related:
#    /etc/acpi/events/mutemic
# Reference:
#    https://forums.linuxmint.com/viewtopic.php?t=299427
#    https://forums.linuxmint.com/viewtopic.php?t=293804
#    https://www.kernel.org/doc/html/latest/sound/hd-audio/models.html
#    https://askubuntu.com/questions/125367
# Alternatives:
#    /etc/modprobe.d/alsa-base.conf: options_snd_hda_intel model=dual-codecs
# Improve:
# Dependencies:
#    apt-get install acpid notification-daemon libnotify-bin
#    ~/.fluxbox/startup has notification-daemon &
INPUT_DEVICE="Capture"
YOUR_USERNAME="$( loginctl list-sessions | awk '/seat/{print $3}' )"
date "+%F $0 $@" >> ~"${YOUR_USERNAME}"/mutemic.log
if amixer sget $INPUT_DEVICE,0 | grep '\[on\]' ; then
    amixer sset $INPUT_DEVICE,0 toggle
    #echo "0 blink" > /proc/acpi/ibm/led
    su $YOUR_USERNAME -c 'DISPLAY=":0.0" notify-send -t 1000 \
            -i microphone-sensitivity-muted-symbolic "Mic MUTED"'
else
    amixer sset $INPUT_DEVICE,0 toggle                       
    #echo "0 on" > /proc/acpi/ibm/led
    su $YOUR_USERNAME -c 'DISPLAY=":0.0" notify-send -t 1000 \
            -i microphone-sensitivity-high-symbolic "Mic ON"'
fi

The Thinkpad automatically recognizes that the correct input was muted/unmuted (although it's not exactly a mute; it's a type of input-selection or availability?) and enables the orange lamp on the mute-mic button. The /proc/acpi/ibm/led special device does not have an integer available for the mute-mic LED lamp (unless you compile a kernel module yourself?) anyways.

The above script depends on an acpi event definition, which I placed in /etc/acpi/events/mutemic:

# File: /etc/acpi/events/mutemic
# Location: LTB-019
# Author: bgstack15
# Startdate: 2022-08-07
# Title: Mic-mute operation
# Purpose: React to acpi event for mute-mic button
# History:
# Usage:
#    determine events by running acpi_listen and pressing buttons
# Related:
#    /usr/local/bin/mutemic
# Reference:
# Improve:
#    get more specific with event?
# Project: X230T-headset
# Dependencies:
#    apt-get install acpid
# Documentation:
#    In file /usr/local/bin/mutemic
event=button/micmute
action=/usr/local/bin/mutemic

After modifying this file, restart acpid service.

Extra research

I had investigated if I needed to change the driver (settings, or which driver was used?), by modifying /etc/modprobe.d/mic-input.conf:

options snd_hda_intel model=dual-codecs

There is a whole list of audio "codecs" to use in that model value. Ultimately though, I learned the original (unspecified, so "auto") settings worked, and I just needed to unmute the mic input.

References

Weblinks

  1. Microphone from headset not working - Linux Mint Forums
  2. <SOLVED> Line Out Built-in Audio & Headphones combined - Linux Mint Forums
  3. HD-Audio Codec-Specific Models — The Linux Kernel documentation
  4. shortcut keys - Enabling Mic Mute button and light on Lenovo Thinkpads - Ask Ubuntu

Run program without network access

If you want to run a program with a special restriction, such as without network access, you can do that with the unshare(1) utility.

unshare -r -n makemkv

I had noticed that makemkv makes some unwanted network traffic, and I was too lazy to insert my network proxy from the past. So a quick Internet search later, and I learned about unshare.

Unshare -n usually needs -r so the application will run. Read the man page for details.

Rsync errors in my main Fedora mirror script

So for just over a month I have noticed some errors in my logs for my main mirror sync scripts.

[2022-07-23T12:00:02Z]root@server3.ipa.internal.com: BEGIN errors
@ERROR: max connections (150) reached -- try again later
rsync error: error starting client-server protocol (code 5) at main.c(1661) [Receiver=3.1.3]
@ERROR: Unknown module 'fedora-enchilada'
rsync error: error starting client-server protocol (code 5) at main.c(1661) [Receiver=3.1.3]
@ERROR: max connections (150) reached -- try again later
rsync error: error starting client-server protocol (code 5) at main.c(1661) [Receiver=3.1.3]
[2022-07-23T12:00:40Z]root@server3.ipa.internal.com: END errors

I didn't think much of it, but I have now learned that each of these "max connection" errors means that my entire connection to the upstream mirror failed. Two of these were kernel.org's mirrors. Apparently now they're so busy/popular at the time I connect to them that the server refuses the connection.

So my on-prem Fedora mirrors were out of date. This problem frustrated me, so I had to find a new mirror. So here is the Fedora releases mirror list and here is EPEL mirror list.

So now that I've picked a new one, my connections work and my local mirror is up-to-date!

You see, nowadays I have only one Fedora workstation. Everything else is now running Devuan GNU+Linux to avoid systemd. If I ever reinstall my Fedora machine, it will get Devuan too. I love my SELinux, but I don't actually care about it for desktop systems. And I'd like to avoid the bloatware on my systems when possible.

Shell snippets for CI jobs

In some CI (continuous integration; i.e., Gitlab runners, Jenkins jobs, et al) shell jobs that I wrote, I finally turned some useful things into parameters. With code being checked into source control, I would hate to have to update the scm just to re-enable pipefail. But updating the invocation of a shell script in the CI is easy! So now I have this:

echo " $@ " | grep -qE " -v | --verbose " && set -x || :
echo " $@ " | grep -qE " --pipefail " && set -oe pipefail || :

It is important to wrap the $@ (all parameters excluding $0 which is the name of the invoked command) with spaces, so that the checks work correctly. Also, the logical or || with a shell no-op : is important for those pipefail-enabled situations!

Share any small snippets that you use to make your life easier in your CI jobs!

Thoughts on self-hosted youtube alternatives

The whole topic of self-hosted youtube alternatives is rather scant on the Internet, which surprised me.

I found a nice web-based yt-dlp frontend! I have a single Docker server for other purposes, so it was nice to play around with this metube. Ultimately it wasn't better than running yt-dlp in a terminal and dragging links into that command to append the url to the command.

And for the viewing frontend, I tried https://git.mills.io/prologic/tube which sounded so promising! The video upload operations never completed for me. I think it was trying to encode the videos (transcode?) but it never actually populated the web page with contents even after ffmpeg stopped running.

So I then tried MediaCMS which I really, really wanted to like. It's a Django (python) app, which I have no experience with. I was unable to get this reverse-proxied behind Apache httpd under a virtual path (or prefix, i.e., https://example.com/media/). I gave up and then used a different port number, so all the traffic on port 1285 went to the application. This one worked with uploaded videos. I had to change a few settings in the deploy/docker/local_settings.py:

MINIMUM_RESOLUTIONS_TO_ENCODE = [240, 360, 1080]
USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

The last two are the bog-standard Django reverse-proxy instructions that deal with https. This is needed for any proxying tasks. I removed my manipulated STATIC_URL, MEDIA_URL, and FORCE_SCRIPT_NAME components which never actually completely worked.

The MediaCMS web presentation looks slick. With the 1080 minimum resolution (which ended up producing a file twice as large as the 1080p files I uploaded), it was acceptable. There is even a little button over the video for "Cast video." From a mobile browser, the cast failed to send to my Chromecast. From a desktop's chromium, it did work to cast to a Chromecast. It started as screen mirroring, and then it asked me if I wanted to send just the video.

So the usability was missing a little bit.

And none of this was actually better than Jellyfin, and also just visiting the files over nfs, except the awesome youtube interface. The quadruple files per video was a little excessive but storage is cheap these days, right?