Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

reposync fails: "No more mirrors to try."

bgstack15

Comments

Symptom

The following message occurs, when running a yum reposync.

# time /usr/bin/reposync --source --repoid=EL7 -m --download-metadata --download_path="/var/www/html/yum/EL/EL7_Mirror"
Repository hosting is listed more than once in the configuration
firefox-60.6.0-3.0.1.el7_6.src FAILED
firefox-60.6.0-3.0.1.el7_6.src: [Errno 256] No more mirrors to try.                            ]  0.0 B/s |    0 B  --:--:-- ETA<[pre>

This could occur during an ansible play.

TASK [sync preprod EL7 to prod] ******************************************************************************************************************
fatal: [yumserver01]: FAILED! => {"changed": true, "cmd": "/usr/bin/reposync --source --repoid=EL7 -m --download-metadata --download_path=\"/var/www/html/yum/EL/EL7_Mirror\"", "delta": "0:04:35.805969", "end": "2019-04-17 08:27:22.119744", "msg": "non-zero return code", "rc": 1, "start": "2019-04-17 08:22:46.313775", "stderr": "", "stderr_lines": [], "stdout": "Repository hosting is listed more than once in the configuration\n\rfirefox-60.6.1-1.0.1.el7_6.src FAILED                                          \n\r(1/34): firefox-60.6.0-3.0 0% [                 ]  0.0 B/s |    0 B   --:-- ETA \r\rfirefox-60.6.0-3.0.1.el7_6.src FAILED                                          \n\r(1/34): firefox-60.6.0-3.0 0% [                 ]  0.0 B/s |    0 B   --:-- ETA \r\r(1/34): freerdp-1.0.2-15.e 0% [                 ]  0.0 B/s | 2.5 MB   --:-- ETA \r\rfreerdp-1.0.2-15.el7_6.1.src.r FAILED                                          \n\r(1/34): freerdp-1.0.2-15.e 0% [                 ]  0.0 B/s | 2.5 MB   --:-- ETA \r\rfreerdp-1.0.2-15.el7_6.1.src.r FAILED                                          \n\r(1/34): freerdp-1.0.2-15.e 0% [                 ]  0.0 B/s | 2.5 MB   --:-- ETA \r\rfreerdp-1.0.2-15.el7_6.1.src.r FAILED                                          \n\r(1/34): freerdp-1.0.2-15.e 0% [                 ]  0.0 B/s | 2.5 MB   --:-- ETA \r\rfreerdp-1
.............................. TRUNCATED ..............................
 0.0 B/s | 2.5 MB   --:-- ETA ", "", "tzdata-java-2019a-1.el7.noarch FAILED                                          ", "", "(1/34): tzdata-java-2019a- 0% [                 ]  0.0 B/s | 2.5 MB   --:-- ETA ", "libwsman1-2.6.3-6.git4391e5c.el7_6.i686: [Errno 256] No more mirrors to try.", "openwsman-client-2.6.3-6.git4391e5c.el7_6.i686: [Errno 256] No more mirrors to try.", "tuned-utils-2.10.0-6.0.1.el7_6.3.noarch: [Errno 256] No more mirrors to try.", "openwsman-server-2.6.3-6.git4391e5c.el7_6.x86_64: [Errno 256] No more mirrors to try.", "sbd-1.3.1-8.2.el7_6.1.x86_64: [Errno 256] No more mirrors to try.", "libwsman1-2.6.3-6.git4391e5c.el7_6.x86_64: [Errno 256] No more mirrors to try.", "sbd-1.3.1-8.2.el7_6.1.src: [Errno 256] No more mirrors to try.", "freerdp-1.0.2-15.el7_6.1.x86_64: [Errno 256] No more mirrors to try.", "freerdp-plugins-1.0.2-15.el7_6.1.x86_64: [Errno 256] No more mirrors to try.", "openwsman-client-2.6.3-6.git4391e5c.el7_6.x86_64: [Errno 256] No more mirrors to try.", "ocfs2-tools-1.8.6-11.el7.x86_64: [Errno 256] No more mirrors to try.", "python-2.7.5-77.0.1.el7_6.src: [Errno 256] No more mirrors to try.", "tuned-2.10.0-6.0.1.el7_6.3.noarch: [Errno 256] No more mirrors to try.", "firefox-60.6.0-3.0.1.el7_6.src: [Errno 256] No more mirrors to try.", "freerdp-libs-1.0.2-15.el7_6.1.i686: [Errno 256] No more mirrors to try.", "freerdp-1.0.2-15.el7_6.1.src: [Errno 256] No more mirrors to try.", "freerdp-libs-1.0.2-15.el7_6.1.x86_64: [Errno 256] No more mirrors to try.", "ghostscript-9.07-31.el7_6.10.i686: [Errno 256] No more mirrors to try.", "ocfs2-tools-1.8.6-11.el7.src: [Errno 256] No more mirrors to try.", "tzdata-java-2019a-1.el7.noarch: [Errno 256] No more mirrors to try.", "ghostscript-9.07-31.el7_6.10.x86_64: [Errno 256] No more mirrors to try.", "openwsman-2.6.3-6.git4391e5c.el7_6.src: [Errno 256] No more mirrors to try.", "tuned-profiles-cpu-partitioning-2.10.0-6.0.1.el7_6.3.noarch: [Errno 256] No more mirrors to try.", "firefox-60.6.1-1.0.1.el7_6.src: [Errno 256] No more mirrors to try.", "thunderbird-60.6.1-1.0.1.el7_6.src: [Errno 256] No more mirrors to try.", "openwsman-server-2.6.3-6.git4391e5c.el7_6.i686: [Errno 256] No more mirrors to try.", "libssh2-1.4.3-12.el7_6.2.src: [Errno 256] No more mirrors to try.", "pcs-0.9.165-6.0.3.el7_6.1.x86_64: [Errno 256] No more mirrors to try.", "openwsman-python-2.6.3-6.git4391e5c.el7_6.x86_64: [Errno 256] No more mirrors to try.", "pcs-0.9.165-6.0.3.el7_6.1.src: [Errno 256] No more mirrors to try.", "tuned-2.10.0-6.0.1.el7_6.3.src: [Errno 256] No more mirrors to try.", "tzdata-2019a-1.el7.src: [Errno 256] No more mirrors to try.", "ghostscript-9.07-31.el7_6.10.src: [Errno 256] No more mirrors to try."]}
        to retry, use: --limit @/etc/ansible/retries/reposync-prod.retry
PLAY RECAP **********************************************************************************************************************************************
yumserver01               : ok=1    changed=0    unreachable=0    failed=1

Possible causes

The original rpm files could be malformed, so even though the reposync client is successfully downloading whatever the server provided, the rpm does not match the checksum in the yum metadata files.

Fix #1

  1. Run the command manually to learn the whole list. Save it down to a file. 
    # time /usr/bin/reposync --source --repoid=EL7 -m --download-metadata --download_path="/var/www/html/yum/EL/EL7_Mirror"

    Repository hosting is listed more than once in the configuration firefox-60.6.0-3.0.1.el7_6.src FAILED firefox-60.6.0-3.0.1.el7_6.src: [Errno 256] No more mirrors to try. ] 0.0 B/s | 0 B --:--:-- ETA

The file used in the documented example consisted of about 30 packages. file: bad.in

    firefox-60.6.0-3.0.1.el7_6.src
firefox-60.6.1-1.0.1.el7_6.src
freerdp-1.0.2-15.el7_6.1.src
freerdp-1.0.2-15.el7_6.1.x86_64
freerdp-libs-1.0.2-15.el7_6.1.i686
freerdp-libs-1.0.2-15.el7_6.1.x86_64
freerdp-plugins-1.0.2-15.el7_6.1.x86_64
ghostscript-9.07-31.el7_6.10.i686
ghostscript-9.07-31.el7_6.10.src

  1. On the reposync client delete or move to a temp directory the failed files. You could use a shell script similar to the following. 

    1
    		 
    #!/bin/sh

    INFILE=/root/bad.in INDIR=/var/www/html/yum/EL/EL7_Mirror/EL7 OUTDIR=/root/orig/ mkdir -p "${OUTDIR}" find $( for word in $( cat "${INFILE}" ) ; do find "${INDIR}" -name "${word}*rpm" done ) -print -exec mv {} "${OUTDIR}" \;

  2. On the repo server, manually download the files. You could modify the input file to have the upstream URLs. bad.in.url

    http://upstream.example.com/repo/EL/EL7/latest/x86_64/getPackage/tuned-utils-2.10.0-6.0.1.el7_6.3.noarch.rpm

    http://upstream.example.com/repo/EL/EL7/latest/x86_64/getPackage/tzdata-java-2019a-1.el7.noarch.rpm http://upstream.example.com/repo/EL/EL7/latest/x86_64/getPackageSource/firefox-60.6.0-3.0.1.el7_6.src.rpm http://upstream.example.com/repo/EL/EL7/latest/x86_64/getPackageSource/firefox-60.6.1-1.0.1.el7_6.src.rpm http://upstream.example.com/repo/EL/EL7/latest/x86_64/getPackageSource/freerdp-1.0.2-15.el7_6.1.src.rpm .... TRUNCATED .....

You could use a shell script similar to the following.

    #!/bin/sh
INFILE=/root/bad.in.url
INDIR=/var/www/html/yum/EL/EL7_Mirror/EL7/getPackage/
INDIRSRC=/var/www/html/yum/EL/EL7_Mirror/EL7/getPackageSource/

for word in $( cat "${INFILE}" ) ;
do
   if echo "${word}" | grep -q getPackageSource ;
   then
      cd "${INDIRSRC}"
   else
      cd "${INDIR}"
   fi
   #echo "$( pwd ) --> ${word}"
   wget "${word}"
done
  1. Rerun the reposync manually and make sure it works.

Comments