Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

My newer-in-debian script

I wanted to use the really cool repology.org site to help track the packages I maintain in Devuan, so I don't get too behind. LeePen feels obligated to tell me when I'm behind, and that shouldn't be his job; I should be doing a better job.

So I wrote this script I can run whenever I feel like it.

files/2024/listings/newer-in-debian.sh (Source)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
#!/bin/sh
# File: newer-in-debian.sh
# Location: /mnt/public/Support/Programs/repology.org/
# Author: bgstack15
# Startdate: 2023-12-28-5 17:42
# SPDX-License-Identifier: GPL-3.0-only
# Title: Newer in Debian script
# Purpose: List newer packages in debian than devuan that I maintain
# History:
# Usage: ./newer-in-debian.sh
# Reference:
#    https://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux
#    https://repology.org/projects/?maintainer=bgstack15@gmail.com&inrepo=devuan_unstable&newest=1
# Improve:
# Dependencies:
#    req-devuan: jq, curl, devscripts
#    req-raw: jq, curl, rmadison
# Documentation: read repology

# CONFIG
test -z "${CACHE_DIR}" && CACHE_DIR="${XDG_CACHE_DIR:-${HOME}/.cache}"
test -z "${CACHE_FILE}" && CACHE_FILE="${CACHE_DIR}/newer-in-debian.cache"
test -z "${EMAIL}" && EMAIL=bgstack15@gmail.com
test -z "${RMADISON_URL_MAP_DEVUAN}" && export RMADISON_URL_MAP_DEVUAN=https://api.pkginfo.devuan.org/madison
BAD_COLOR='\033[0;31m' # red
GOOD_COLOR='\033[0;32m' # green
NEUTRAL='\033[0m' # no color

# CACHE OPERATIONS
# invalidate cache older than 23 hours
_cache_is_too_old="$( find "${CACHE_FILE}" -maxdepth 0 -mmin +$(( 60 * 23 )) -print -quit 2>/dev/null )"
test -n "${_cache_is_too_old}" && rm "${CACHE_FILE:-NOTHINGTODEL}"
test -n "${CLEAR_CACHE}" && rm "${CACHE_FILE:-NOTHINGTODEL}"

# test and rebuild cache if necessary
_cache_contents="$( cat "${CACHE_FILE}" 2>/dev/null )"
test -z "${_cache_contents}" && {
   curl --silent --header 'User-Agent: bgstack15-cli' "https://repology.org/api/v1/projects/?maintainer=${EMAIL}&inrepo=devuan_unstable" > "${CACHE_FILE}"
   _cache_contents="$( cat "${CACHE_FILE}" 2>/dev/null )"
} || {
   printf '%s\n' "Using cached package list." 1>&2
}

# if cache is still empty, then something broke with curl statement.
test -z "${_cache_contents}" && {
   printf '%s\n' "Fatal! Cache is still empty. Re-run with debugging. Aborted." 1>&2
   exit 1
}

# FUNCTIONS
_check_status() {
   package="${1}"
   # this only uses amd64. If I ever change personal architectures that I care about, I would need to update this.
   # The purpose is to avoid the "source" listing, which might not be the same version as a built architecture, due to binary repacks, e.g., lightdm_1.32.0-4+b1 in debian.
   # --source-and-binary converts "freeipa" source into "freeipa-client" amd64, and a few others but since the versions are all the same we just take the first one (head -n1)
   _deb="$( rmadison              --source-and-binary --architecture=amd64 --suite unstable "${package}" | awk -F'|' '{print $2}' | head -n1 | xargs | sed -r -e 's/\+b[1-9]$//;' )" ;
   # only do anything if the package exists in Debian.
   if test -n "${_deb}" ;
   then
      _dev="$( rmadison --url devuan --source-and-binary --architecture=amd64 --suite unstable "${package}" | awk -F'|' '{print $2}' | head -n1 | xargs | sed -r -e 's/\+b[1-9]$//;' )" ;
      _result="${BAD_COLOR}BAD${NEUTRAL}"
      echo "${_dev}" | grep -qE "${_deb}\+?devuan-?[0-9]" && {
         _result="${GOOD_COLOR}good${NEUTRAL}"
      }
      printf '%4b\t%s\t%s\t%s\n' "${_result}" "${package}" "${_deb}" "${_dev}"
   else
      printf '%4s\t%s\t\n' "none" "${package}" 1>&2
   fi
}

# MAIN
_package_list="$( <"${CACHE_FILE}" jq --raw-output 'keys[]' )"
# do not double-quote variable in next line with the for statement.
for package in ${_package_list} ;
do
   if test -n "${PARALLEL}" ;
   then
      _check_status "${package}" &
   else
      _check_status "${package}"
   fi
done
test -n "${PARALLEL}" && {
   wait
}

I use cache invalidation (which I hear is hard; I hope I did it right!). The web url is https://repology.org/projects/?maintainer=bgstack15@gmail.com&inrepo=devuan_unstable&newest=1. Some of these are original packages and don't have an upstream from Debian.

$ ./newer-in-debian.sh 
good    freeipa 4.10.2-2    4.10.2-2devuan1
good    lightdm 1.32.0-4    1.32.0-4devuan1
good    oddjob  0.34.7-2    0.34.7-2devuan1
none    pam-mkhomedir   
none    sgm 
none    systemctl-service-shim

I even added colors, and parallelization (but not really necessary now that we only run rmadison for Devuan if the rmadison for Debian returned anything), just for this blog post. You're welcome, future self.

Comments