Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

updating set-my-repos.sh for deb822 format

bgstack15

Comments

I've previously described set-my-repos.sh in a multitude of posts:

I've updated my process to work with deb822 format, which ironically is not fully documented there, and you have to read manpages.debian.org to understand all the fields.

A major improvement to this whole process is that I no longer need a custom /etc/apt/apt.conf.d/52apt-file-stackrpms.conf file at all. Because you can specify Components, or just omit that, apt will then figure out the Contents sections correctly!

The downside is that because I'm not storing the original upstream url for the OBS repositories in, for example, a separate file, and because I resign the repository (so I can include the older versions of dpkgs in the repo), the gpg keys won't match even if I use the upstream url. I should probably bother to set up a "Enabled: no" version of "OBS bgstack15-upstream" repo files, but I haven't bothered to do that yet.

I don't think one-line-style format is going away anytime soon, but I liked the idea of using the newer option, and it had that awesome bonus of not needing the custom apt-file config (which was flaky).

You can see the old version (from my 2024-05-05 post linked earlier) here

files/2024/listings/set-my-repos-2024-09.sh (Source)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
 
#!/bin/sh
# File: /mnt/public/Support/Platforms/devuan/set-my-repos.sh
# Location:
# Author: bgstack15
# Startdate: 2019-08-10 16:02
# Title: Script that Establishes the repos needed for Devuan
# Purpose: Set up the 3 repos I always need on devuan clients
# History:
#    2020-02-01 customize clients for devuan-archive
#    2020-10-23 add apt-file compatibility
#    2021-01-27 disable devuan-archive
#    2024-01-03-4 14:53 add pref for zenity with gtk3
#    2024-09-12-5 13:52 use deb822 format sources now
# Usage:
#    sudo set-my-repos.sh
# Reference:
#    /mnt/public/Support/Platforms/devuan/devuan.txt
# Improve:
#    need to control the sources.list file itself to have the main, contrib, etc., for ceres.
# Documentation:

test -z "${ALLREPOSGLOB}" && ALLREPOSGLOB="/etc/apt/sources.list /etc/apt/sources.list.d/*"
test -z "${REPOSBASE}" && REPOSBASE="/etc/apt/sources.list.d"
test -z "${PREFSBASE}" && PREFSBASE="/etc/apt/preferences.d"
test -z "${ADDLCONFBASE}" && ADDLCONFBASE="/etc/apt/apt.conf.d"

# confirm key
confirm_key() {
   # call: confirm_key "${PRETTYNAME}" "${SEARCHPHRASE}" "${URL_OF_KEY}"
   ___ck_repo="${1}"
   ___ck_sp="${2}"
   ___ck_url="${3}"
   if apt-key list 2>/dev/null | grep -qe "${___ck_sp}" ;
   then
      :
   else
      # not found so please add it
      echo "Adding key for ${___ck_repo}" 1>&2
      #wget -O- "${___ck_url}" | sudo apt-key add -
      ___ck_keyfile="/etc/apt/trusted.gpg.d/$( echo "${___ck_repo}" | tr '[: ]' '_' ).gpg"
      wget -O- --quiet "${___ck_url}" | gpg --dearmor | sudo tee "${___ck_keyfile}" 1>/dev/null
   fi
}

# confirm repo
confirm_repo() {
   # call: confirm_repo "${PRETTYNAME}" "${SEARCHPHRASE}" "${SEARCHGLOB}" "${FULLSTRING}" "${PREFERRED_FILENAME}" "${OVERWRITE}"
   ___cr_repo="${1}"
   ___cr_sp="${2}"
   ___cr_sf="${3}"
   ___cr_full="${4}"
   ___cr_pref="${5}"
   ___cr_overwrite="${6}"
   if ! grep -E -qe "${___cr_sp}" ${___cr_sf} ;
   then
      # not found so please add it to preferred file
      echo "Adding repo ${___cr_repo}" 1>&2
      if test "${___cr_overwrite}" = "true" ;
      then
         # overwrite, instead of append
         echo "${___cr_full}" > "${REPOSBASE}/${___cr_pref:-99_misc.list}"
      else
         echo "${___cr_full}" >> "${REPOSBASE}/${___cr_pref:-99_misc.list}"
      fi
   fi
}

confirm_sources() {
   # call: confirm_sources "${PRETTYNAME}" "${URIS}" "${SUITES}" "${SIGNEDBY}" "${FILENAME}" "${OVERWRITE}" "${REMOVE_LISTFILE}"
   __cs_prettyname="${1}"
   __cs_uris="${2}"
   __cs_suites="${3}" # probably will be "/" for my kind of repos
   __cs_signedby="${4}"
   __cs_filename="${5}"
   __cs_overwrite="${6}"
   __cs_remove_listfile="${7}"
   # determine if cs_filename is short, if so, prepend /etc/apt/sources.list.d
   if test "${__cs_filename}" = "$( basename "${__cs_filename}" )" ;
   then
      __cs_filename="/etc/apt/sources.list.d/${__cs_filename}"
   fi
   __cs_listfile="${__cs_filename%%.sources}.list"
   if test "${__cs_listfile}" = "$( basename "${__cs_listfile}" )" ;
   then
      __cs_listfile="/etc/apt/sources.list.d/${__cs_listfile}"
   fi
   # determine of cs_signedby gpg key is short, if so, prepend /etc/apt/trusted.gpg.d
   if test -n "${__cs_signedby}" && test "${__cs_signedby}" = "$( basename "${__cs_signedby}" )" ;
   then
      __cs_signedby="/etc/apt/trusted.gpg.d/${__cs_signedby}"
   fi
   if test ! -r "${__cs_filename}" || test "{__cs_overwrite}" = "true";
   then
      {
         echo "Enabled: yes"
         echo "Types: deb"
         echo "URIs: ${__cs_uris}"
         echo "Suites: ${__cs_suites}"
         test -n "${__cs_prettyname}" && echo "X-Repolib-Name: ${__cs_prettyname}" || :
         test -n "${__cs_signedby}" && echo "Signed-By: ${__cs_signedby}"
      } > "${__cs_filename}"
   fi
   test "${__cs_remove_listfile}" = "true" && test -f "${__cs_listfile}" && echo "${__cs_listfile}" | grep -qE "\/etc\/apt\/sources\.list\.d\/.+\.list$" 1>/dev/null 2>&1 && rm -f "${__cs_listfile}"
}

confirm_preferences() {
   # call: confirm_preferences "${PRETTYNAME}" "${FILENAME}" "{PACKAGE}" "${PIN_EXPRESSION}" "{PRIORITY}"
   ___cp_prettyname="${1}"
   ___cp_pref="${2}"
   ___cp_package="${3}"
   ___cp_pin_expression="${4}"
   ___cp_priority="${5}"
   ___cp_version="${6}"

   ___cp_tempfile="$( mktemp )"
   {
      echo "Package: ${___cp_package}"
      test -n "${___cp_version}" && echo "Version: ${___cp_version}"
      echo "Pin: ${___cp_pin_expression}"
      echo "Pin-Priority: ${___cp_priority}"
   } > "${___cp_tempfile}"

   diff "${PREFSBASE}/${___cp_pref}" "${___cp_tempfile}" 1>/dev/null 2>&1 || {
      echo "Setting preferences for ${___cp_prettyname}"
      touch "${PREFSBASE}/${___cp_pref}" ; chmod 0644 "${PREFSBASE}/${___cp_pref}"
      cat "${___cp_tempfile}" > "${PREFSBASE}/${___cp_pref}"
   }

   rm -f "${___cp_tempfile:-NOTHINGTODEL}" 1>/dev/null 2>&1
}

# REPO 1: local internaldeb
confirm_key "internaldeb" "bgstack15.*www\.no-ip\.biz" "http://server3/internal/repo/deb/internaldeb.gpg"
confirm_sources "Internal Dpkg repo" "http://server3/internal/repo/deb" "/" "internaldeb.gpg" "internaldeb.sources" "true" "true"

# REPO 2: local devuan-deb
confirm_key "devuan-deb" "bgstack15.*www\.no-ip\.biz" "http://server3/internal/repo/deb/internaldeb.gpg"
confirm_sources "Internal Devuan dpkgs" "http://server3/internal/repo/devuan-deb" "/" "internaldeb.gpg" "devuan-deb.sources" "true" "true"

# REPO 3: local obs
# Thankfully I re-sign this with my own key.
#confirm_key "OBS bgstack15" "bgstack15@build\.opensuse\.org" "https://download.opensuse.org/repositories/home:bgstack15/Debian_Unstable/Release.key"
confirm_key "OBS bgstack15" "bgstack15.*www\.no-ip\.biz" "http://server3/mirror/obs/Release.key"
confirm_sources "OBS bgstack15" "http://server3/mirror/obs" "/" "OBS_bgstack15.gpg" "home:bgstack15.sources" "true" "true"

# REPO 4: local devuan-archive
# deprecated circa 2021-05
# enabled again 2023-08-22 for discord/gconf
confirm_key "devuan-archive" "bgstack15.*www\.no-ip\.biz" "http://server3/internal/repo/deb/internaldeb.gpg"
confirm_sources "Internal Devuan archive" "http://server3.ipa.internal.com/internal/repo/devuan-archive" "/" "internaldeb.gpg" "devuan-archive.sources" "true" "true"
confirm_preferences "devuan-archive" "puddletag" "*" "origin server3.ipa.internal.com" "700"

# REPO 5: local obs-aftermozilla key for non-local aftermozilla repo
# just the key
#confirm_key "OBS bgstack15 aftermozilla" "bgstack15@build\.opensuse\.org" "https://download.opensuse.org/repositories/home:bgstack15:aftermozilla/Debian_Unstable/Release.key"

# REPO 5: local obs-AfterMozilla
#confirm_key "OBS bgstack15" "bgstack15@build\.opensuse\.org" "http://server3/mirror/obs/Release.key"
confirm_key "OBS bgstack15" "bgstack15.*www\.no-ip\.biz" "http://server3/mirror/obs/Release.key"
confirm_sources "OBS AfterMozilla" "http://server3/mirror/obs-AfterMozilla" "/" "OBS_bgstack15.gpg" "home:bgstack15:AfterMozilla.sources" "true" "true"

# REPO 6: local obs-gtk3-classic
confirm_key "OBS bgstack15" "bgstack15.*www\.no-ip\.biz" "http://server3/mirror/obs/Release.key"
confirm_sources "OBS gtk3-classic" "http://server3/mirror/obs-gtk3-classic" "/" "OBS_bgstack15.gpg" "home:bgstack15:gtk3-classic.sources" "true" "true"

# ADDITIONAL APT PREFS
# important for the [target] stuff to work on repos so apt-file can work
#cp -p "$( dirname "$( readlink -f "${0}" )")/input/52apt-file-stackrpms.conf" "${ADDLCONFBASE}/"
rm -f "${ADDLCONFBASE}/52apt-file-stackrpms.conf" 2>/dev/null || :
# 2023-10-27-6 08:47 use apt-preferences to hold this exact app version because newer versions remove the system tray icon.
confirm_preferences "all" "krb5-auth-dialog" "krb5-auth-dialog" "release" "1000" "3.26.1-4"
# zenity with gtk3 is stored in devuan-deb
confirm_preferences "all" "zenity" "zenity" "release" "600" "3.44.2-1"
confirm_preferences "all" "zenity-common" "zenity-common" "release" "600" "3.44.2-1"

Basically, a new style repo looks like:

Enabled: yes
Types: deb
URIs: http://server3/internal/repo/deb
Suites: /
X-Repolib-Name: Internal Dpkg repo
Signed-By: /etc/apt/trusted.gpg.d/internaldeb.gpg

Which is easier for some parsing compared to:

deb [target-=Contents-deb target+=Contents-stackrpms] http://server3/internal/repo/deb/ /

Which is believable.

Comments