Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

TheMovieDb: Generate my Custom CSV for a show

I've discussed before how I use a CSV to help apply metadata to media files. One of my problems with just manually munging the Wikipedia list of TV show episodes is that TheMovieDB organizes episodes a little differently. Sometimes a two-parter is listed as a single episode, and so on. All minor things, but my episode number can get off which affects my filenames and metadata.

So I decided to investigate using TheMovieDb to generate my CSV/spreadsheet. I found a great python library for tmdbv3api. I think there's multiple python libraries for this, but they all get to the same thing: the contents of the show details.

Here's my little wrapper script: files/2023/04/listings/eplib.py (Source)

#!/usr/bin/env python3
# File: eplib.py
# Location: /mnt/public/Support/Programs/themoviedb
# Author: bgstack15
# Startdate: 2023-04-19-4 13:51
# Title: Episode CSV Generator
# Purpose: facilitate my csv list of episodes of a TV show
# History:
# Usage:
#    source ~/venv1/bin/activate ; cd /mnt/public/Support/Programs/themoviedb/
#    python3
#    >>> import importlib, eplib
# References:
#    1. https://pypi.org/project/tmdbv3api/
#    2. https://stackoverflow.com/questions/517923/what-is-the-best-way-to-remove-accents-normalize-in-a-python-unicode-string
# Improve:
# Dependencies:
#    One time:
#       python3 -m venv ~/venv1
#       source ~/venv1/bin/activate
#       pip3 install tmdbv3api
from tmdbv3api import TMDb, TV, Season
import os
tmdb = TMDb()
tmdb.language = "en"
tmdb.debug = True
tv = TV()
def set_api_key(filename = None):
   """ Given a filename (or use hardcoded default file if None), load api key from file. """
   if filename is None:
      filename = "/mnt/public/Support/Programs/themoviedb/apikey"
   try:
      with open(filename,"r") as o:
         tmdb.api_key = o.read().rstrip('\n')
   except:
      return -1
   return 0
### Ripped from ref 2
from unicodedata import combining, normalize
LATIN = "ä  æ  ǽ  đ ð ƒ ħ ı ł ø ǿ ö  œ  ß  ŧ ü "
ASCII = "ae ae ae d d f h i l o o oe oe ss t ue"
def remove_diacritics(s, outliers=str.maketrans(dict(zip(LATIN.split(), ASCII.split())))):
    return "".join(c for c in normalize("NFD", s.translate(outliers)) if not combining(c))
### end ref 2
# bgstack15 function
def filename_safifier(s):
   """ Cruddy detox for myself. Should consider using my customized detoxrc """
   return remove_diacritics(s.replace("'","").replace('"',"").replace("...","").replace(",","").replace(":","_").replace("!","").replace(".","").replace("?","").replace("&","and").replace("/","_"))
def get_tv_details(show_id):
   """ Uses web connection to get details of the item, either id number or show name """
   if show_id.isdigit():
      details = tv.details(show_id)
   else:
      show_temp = get_tv_search(show_id)
      try:
         details = tv.details(show_temp[0]["id"])
      except:
         print("Invalid TV show name. Found these names though:")
         for i in show_temp:
            print(f"id {i['id']}: {i['name']}")
         return -1
   return details
def get_tv_search(show_id):
   """ Uses web connection to search for item, probably the name of the show """
   return tv.search(show_id)
def get_episodes_for_show(show_id = "", include_specials = False):
   """
   Given a show id (can be show_id integer, show name string, or a TV().details object), return useful list of episodes. If given show_id is a TV().details object, use that to avoid all the web calls. Example:
      a = TV().details(1855)
      get_episodes_for_show(a)
      OR
      a = get_tv_details("Star Trek Deep Space Nine")
      get_episodes_for_show(a)
   If include_specials, then also list episode titles and airdates for these special features but they still are not counted as absolute episode numbers.
   """
   if type(show_id).__name__ == "AsObj":
   #if show_details is not None:
      show = show_id
   else:
      if show_id.isdigit():
      # IMPROVE: accept show name?
         show = get_tv_details(show_id)
      else:
         show_temp = get_tv_search(show_id)
         try:
            show = get_tv_details(show_temp[0]["id"])
         except:
            print("Invalid TV show name. Found these names though:")
            for i in show_temp:
               print(f"id {i['id']}: {i['name']}")
            return -1
   #print(show)
   name = f"{show['name']} ({show['first_air_date'][0:4]})"
   season = Season()
   seasons = show["seasons"]
   print(f"Name: {name}")
   show_id = show["id"]
   abs_epnum = 0
   abs_epnum_including_specials = 0
   for s in show["seasons"]:
      snum = s["season_number"]
      season_eps = season.details(show_id, snum).episodes
      for e in season_eps:
         abs_epnum_including_specials += 1
         if snum != 0:
            abs_epnum += 1
         if include_specials or snum != 0:
            sep = e["episode_number"]
            epname = e["name"].replace("(1)","Part 1").replace("(2)","Part 2").replace("(3)","Part 3")
            if '"' in epname:
               epname = "'" + epname + "'"
            if ',' in epname and not '"' in epname:
               epname = '"' + epname + '"'
            airdate = e["air_date"]
            filename = filename_safifier(f"s{snum:02d}e{sep:02d} - {epname}")
            #print(f"{snum},{abs_epnum},{sep},{epname},{airdate},")
            if abs_epnum_including_specials == 1:
               print("s,ep,sep,epname,airdate,filename,")
            print(f"{snum:02d},{abs_epnum:03d},{sep:02d},{epname},{airdate},{filename},")
            #print("%02d,%03d,%02d,%s,%s," % (snum, abs_epnum, sep, epname, airdate))
# Run this regardless of __main__
set_api_key(os.environ.get("TMDB_APIKEY"))

So now I can get the output I want!

Infcloud improvement: keyboard shortcuts dialog

I added a dialog to my fork of Infcloud! This application originally had zero keyboard navigation, and I've slowly been adding some to make it more usable to me.

Right now the contents of the help dialog are hard-coded, but perhaps someday I will attempt to have the section that adds the keyboard shortcuts add their own text description to a variable that is used to populate this field.

All the work this time is in one merge commit.

Libreoffice Impress extension: PhotoAlbum

LibreOffice Impress has an extension, PhotoAlbum, that lets you load a set of images into a presentation. That site doesn't have the downloadable asset, but extensions.openoffice.org does! You want exact file https://extensions.openoffice.org/en/download/4216.

I also mirror this extension in case this upstream site ever takes it down: portablephotoalbum-1.0.3.oxt

This is a very useful extension if you use other software to generate a presentation and export it as a set of images.

Jellyfin: casting to my TV without Chromecast

Previously, I have used Chromecast and the Play Store release of the Jellyfin android client. This arrangement depends on the Internet connectivity, and of course a previously-established Chromecast.

On a whim, I set up the GNU/Linux desktop client of Jellyfin on my htpc, and was very pleased with the performance of video playback. The web browser on that device seems to struggle with seamless playback from the Jellyfin web page. I initially thought this native client was just an Electron app, but it appears otherwise!

And, if I sign in to this desktop app with the same user account as on my mobile device, I can use the mobile device to connect to this client and control it! Menu navigation happens on the mobile device, and when I select play on a media selection, it runs it on the big tv. This type of remote control is really nice! Now I don't have to rely on Chromecast, or dig out my living room keyboard which could be anywhere (but the living room, apparently): I can just use a mobile device, not really with any screen casting tech, but then Chromecast wasn't either.

Jellyfin is a great piece of FLOSS tech. I know I had investigated kodi but Jellyfin fit my use case way better.

Firefox 111.0: fix puzzle piece in toolbar

Starting in Firefox 109.0, we could use a line in the prefs.js (or about:config) to remove the annoying puzzle piece.

user_pref("extensions.unifiedExtensions.enable", false);

However, starting in Firefox 111.0 the Mozilla people started ignoring this pref. The brilliant folks over at askvg.com solved this one though!

You have take a few steps for this solution.

  1. Set about:config value toolkit.legacyUserProfileCustomizations.stylesheets to true.
  2. Visit the profile directory (available from about:support, button "Open Directory") and make sub-directory chrome.
  3. Make file chrome/userChrome.css with contents:
#unified-extensions-button, #unified-extensions-button > .toolbarbutton-icon{
width: 0px !important;
padding: 0px !important;
}
  1. Restart Firefox.

I don't plan on ever using Chrome, but wow, the Mozilla people make me want to look for something else.

This is the first time I've ever needed to modify userChrome.css. Ever since 2006, I have never needed it!

Mbbmlib improvements because of firefox changes

A while back I wrote about my project that exports Firefox bookmarks.

Due to changes between LibreWolf 109.0 and 110.0, I have now updated the source to work with the latest schema for bookmarks and how icons are connected to them. I also removed Newmoon support, because I don't use that browser for primary usage where I save bookmarks that need to be shared/bupped.

I actually use specific SELECT statements now, instead of SELECT *, so I grab only the columns that matter, and the results are predicable and the code is therefore more readable.

Monthly todos

Every month, I have a series of tasks I want to perform. I like to be organized (have you noticed the cadence of posts to this site?).

My monthly task items include, in no particular order:

  1. Backing up bookmarks with my own cruddy bookmark exporter, mbbmlib. The mbbmlib library is broken at the moment, at least for actually storing icons of the sites. Mozilla changed the schema for the favicon_ids and I need to reverse-engineer it or find the official schema. Drat those guys.
  2. Check backups. I have a number of backup processes, only partially described under tags backup and backups. I should check them more often, but if I don't, I do it at least monthly.
  3. OS updates. You'll see updates on my blog on occasion if I ran into an interesting problem after monthly updates.
  4. Backup entire browser profile. I guess I generate lots of backups. Maybe I should actually test a restore someday.
  5. Make a new tab in my budget spreadsheet. I use LibreOffice Calc, btw.

LibreWolf and Jitsi Meet

Jitsi Meet is a great piece of self-hosted software. I recently had to update my instance to get the mobile device support back. At various points in its history, it has required the use of Chromium-based web browsers. For a while now though, it has supported Firefox.

Apparently it uses the useragent string to determine which browser you're using, which is quite ordinary. I'm not that interested in Google's attempts to manipulate this on the World Wide Web and use "capabilities" or something else which will need to be spoofed to accomplish what we want with non-Google browsers.

By default, LibreWolf uses its own browser name in the user agent string, so Jitsi Meet will tell you to use Firefox or Chrome. You can set privacy.resistFingerprinting = true but this affects my calendar solution by using UTC timezone and messing up the display of my calendar events. Well, just recently I poked around in about:config and saw a very useful setting:

general.useragent.compatMode.firefox = true

By using true here, you can use the useragent string that solves the Jitsi Meet problem, as well as a few other sites' lack of awareness of LibreWolf as a firefox-based browser. Well, you just ignore their lack of awareness, because they are now only aware of your Firefox browser. I've filed bug reports, but I guess webmasters don't care these days.

I don't mind advertising that I use LibreWolf instead of Firefox. But I do want sites to work, so if the firefox user agent string gets me what I want, then I will use it.

Scraping Pharaoh walkthroughs

Every so often I intend to play through the old Pharaoh computer game: no Cleopatra! I have the original disc and don't need some mangled version.

But I don't get very far before I lose interest in the game. Sometimes I don't even get all the way to Giza and the big, famous pyramids. So I need guidance to play the game, and for that I use the classic walkthroughs on Heavengames.

Just in case this site ever disappears, I cloned the pages locally for myself.

wget --limit-rate=50k -E -k -r -N -p -l 7 https://pharaoh.heavengames.com/walkthroughs/ --reject-regex '.*caesar3.*|.*archives.*|.*strategy.*|.*downloads.*|.*faqs.*|.*buildings.*|.*links.*|.*comps/|.*egypt/|.*interviews/|.*gallery/|.*sitemap/|.*lympis/|.*\<e3/|.*cgi-bin/.*newsfiles/'

I built that exclusion regex after a bunch of trial-and-error. I only wanted the sources necessary to display the ~21 level walkthroughs with images and css.