Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

Setup Yum Repository with Security Metadata

bgstack15

Comments

Define repository

Prepare the repo file on the server, so clients can download it.

cd /var/www/html/yum
cat <<'EOF' > hosting.repo
[hosting]
name=Hosting Delivery
baseurl=http://yum5.ipa.example.com/yum/hosting/
enabled=0
gpgcheck=0
EOF

Make or update repository

Use createrepo tool to make the repository. A wrapper script for creating or updating the existing repository is shown here.

tf=/usr/local/bin/updaterepo.sh
cat <<'EOF' > "${tf}"
#!/bin/sh
# reference:
#    https://gitlab.com/bgstack15/mirror/blob/master/usr/share/mirror/examples/rpm/update-examplerpm.sh

# Prepare directory and files
test -z "${UR_REPODIR}" && UR_REPODIR=/var/www/html/yum/hosting
test -z "${UR_BASEURL}" && UR_BASEURL=http://yum5.ipa.example.com/yum/hosting
test -z "${UR_OWNERSHIP}" && UR_OWNERSHIP="root.root"
test -z "${UR_FILETYPES}" && UR_FILETYPES="rpm"

find "${UR_REPODIR}" -exec chown "${UR_OWNERSHIP}" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type f -exec chmod "0664" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type d -exec chmod "0775" {} + 1>/dev/null 2>&1
chmod 0754 "$0"
restorecon -RF "${UR_REPODIR}"

# Prepare repo for rpm
cd "${UR_REPODIR}"
createrepo -v -u "${UR_BASEURL}" --basedir "${UR_REPODIR}" --simple-md-filenames --no-database --update --pretty .
EOF

Run this script.

/usr/local/bin/updaterepo.sh

Manually make the security metadata

The security metadata that yum interprets is stored in updateinfo.xml.gz. To make this file and include it in repomd.xml, you need to prepare it and learn some information about it. This is a trim example of updateinfo.xml. Please see the epel metadata for a full example. I do not have an automatic process for generating this file yet.

tf=updateinfo.xml
cat <<'EOF' > "${tf}"
<?xml version="1.0" encoding="UTF-8"?>
<updates>
  <update status="final" type="security" version="1" from="bgstack15@gmail.com">
    <id>HELP-210217</id>
    <title>bgscripts-core update</title>
    <release>Enterprise Linux 7</release>
    <issued date="2018-04-02"/>
    <rights>CC-BY-SA 4.0</rights>
    <description>bgscripts-core
[1.3-8]
- latest version from upstream
</description>
    <solution>This update is internal to the company.</solution>
    <references>
      <reference href="https://gitlab.com/bgstack15/bgscripts" type="self" title="bgscripts-core" />
    </references>
    <pkglist>
      <collection short="bgscripts">
        <name>bgscripts suite</name>
        <package name="bgscripts-core" version="1.3-8" release="" epoch="0" arch="noarch">
          <filename>bgscripts-core-1.3-8.noarch.rpm</filename>
          <sum type="md5">eaa20075720bf12d6e837a4f546241ab</sum>
        </package>
     </collection>
    </pkglist>
  </update>
</updates>
EOF

Update the repo metadata to include updateinfo.xml

A yum repository includes metadata of the package metadata, and stores this meta-metadata in repomd.xml. Insert the metadata for this new file, updateinfo.xml in the repomd file. This script is an update version of updaterepo.sh, which was listed earlier in this document.

tf=/usr/local/bin/updaterepo.sh
        cat <<'EOF' > "${tf}"
#!/bin/sh
# reference:
#    https://gitlab.com/bgstack15/mirror/blob/master/usr/share/mirror/examples/rpm/update-examplerpm.sh

# Prepare directory and files
test -z "${UR_REPODIR}" && UR_REPODIR=/var/www/html/yum/hosting
test -z "${UR_BASEURL}" && UR_BASEURL=http://yum5.ipa.example.com/yum/hosting
test -z "${UR_OWNERSHIP}" && UR_OWNERSHIP="root.root"
test -z "${UR_FILETYPES}" && UR_FILETYPES="rpm"
test -z "${UR_UPDATEINFO_INPUT}" && UR_UPDATEINFO_INPUT=/var/www/html/yum/build-hosting-repo/updateinfo.xml

find "${UR_REPODIR}" -exec chown "${UR_OWNERSHIP}" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type f -exec chmod "0664" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type d -exec chmod "0775" {} + 1>/dev/null 2>&1
chmod 0754 "$0"
restorecon -RF "${UR_REPODIR}"

# Prepare basic repo
cd "${UR_REPODIR}"
createrepo -v -u "${UR_BASEURL}" --basedir "${UR_REPODIR}" --simple-md-filenames --no-database --update --pretty .

# Inject custom updateinfo
# this task assumes the repomd file does not include node <data type="updateinfo"> yet.
UR_repomd="${UR_REPODIR}/repodata/repomd.xml"
UR_updateinfo_gz_short="repodata/updateinfo.xml.gz"
UR_updateinfo_gz="${UR_REPODIR}/${UR_updateinfo_gz_short}"

if ! test -e "${UR_UPDATEINFO_INPUT}" ;
then
   # file is absent, so decide how to fail.
   :
else
   # file exists, so continue with custom injection

   # learn open-size and open-checksum
   UR_updateinfo_opensize="$( /usr/bin/stat -c "%s" "${UR_UPDATEINFO_INPUT}" )"
   UR_updateinfo_openchecksum="$( /usr/bin/sha256sum "${UR_UPDATEINFO_INPUT}" | awk '{print $1}' )"

   # compress file and learn size and checksum
   /usr/bin/gzip < "${UR_UPDATEINFO_INPUT}" > "${UR_updateinfo_gz}"
   UR_updateinfo_size="$( /usr/bin/stat -c "%s" "${UR_updateinfo_gz}" )"
   UR_updateinfo_checksum="$( /usr/bin/sha256sum "${UR_updateinfo_gz}" | awk '{print $1}' )"
   UR_updateinfo_timestamp="$( /usr/bin/stat -c "%Y" "${UR_updateinfo_gz}" )"

   # insert information into repomd
   this_string="<data type=\"updateinfo\">
  <checksum type=\"sha256\">${UR_updateinfo_checksum}</checksum>
  <open-checksum type=\"sha256\">${UR_updateinfo_openchecksum}</open-checksum>
  <location xml:base=\"${UR_BASEURL}\" href=\"${UR_updateinfo_gz_short}\"/>
  <timestamp>${UR_updateinfo_timestamp}</timestamp>
  <size>${UR_updateinfo_size}</size>
  <open-size>${UR_updateinfo_opensize}</open-size>
</data>"

   {
      sed -r -e '/<\/repomd>/d' "${UR_repomd}"
      printf "%s\n%s\n" "${this_string}" "</repomd>"
   } > "${UR_repomd}.$$"
   /bin/touch --reference "${UR_repomd}" "${UR_repomd}.$$"
   /bin/mv -f "${UR_repomd}.$$" "${UR_repomd}"
fi
EOF

Summary

Using bash to modify xml files is obviously not ideal. However, this xml file is simple enough so this ugly mechanism suffices. For teams that know how to manage custom yum repositories and also want to just use yum update --security, this process should be a good basis or even complete solution!

Appendices

Appendix A: http proxy

If you use an http proxy for your yum traffic, the proxy might cache old versions of the metadata or package files. A quick and dirty way to clean up a squid proxy of the metadata file follows.

time squidclient -h localhost -r -p 3128 -m PURGE http://yum5.ipa.example.com/yum/hosting/repodata/updateinfo.xml.gz

Squid unfortunately does not allow recursive purging, so you will have to loop over all the metadata files and any package files you want to ensure get cleared.

References

Local file /var/cache/yum/x86_64/7Server/epel/69b82df00108c0ac8ac82fafbd0f3b89cc98d8dfe4fa350af7a23331a878fea2-updateinfo.xml.bz2

Comments