intermittent job failures due to failing to build bpftrace

[sdimitro]

Tail of log:

15:17:17 Err:7 http://apt.llvm.org/bionic llvm-toolchain-bionic/main Sources
15:17:17   File has unexpected size (2527 != 2528). Mirror sync in progress? [IP: 151.101.190.49 80]
15:17:17   Hashes of expected file:
15:17:17    - Filesize:2528 [weak]
15:17:17    - SHA256:1debfc554e873cb9a7551fd1339b571f49d33ba25a9712c84cf3f91d55dc974a
15:17:17    - SHA1:8dc32ec3ac58e6b816f85292bb48deda69162a37 [weak]
15:17:17    - MD5Sum:baed3593f037e02364b7cb4e9bb393a2 [weak]
15:17:17   Release file created at: Mon, 24 Dec 2018 22:26:14 +0000
15:17:17 Hit:8 http://ppa.launchpad.net/ansible/ansible/ubuntu bionic InRelease
15:17:17 Get:9 http://apt.llvm.org/bionic llvm-toolchain-bionic/main amd64 Packages [8166 B]
15:17:17 Get:10 http://apt.llvm.org/bionic llvm-toolchain-bionic-5.0/main Sources [2063 B]
15:17:17 Get:11 http://apt.llvm.org/bionic llvm-toolchain-bionic-5.0/main amd64 Packages [7178 B]
15:17:17 Fetched 25.9 kB in 0s (57.8 kB/s)
15:17:18 Reading package lists...
15:17:18 E: Failed to fetch http://apt.llvm.org/bionic/dists/llvm-toolchain-bionic/main/source/Sources.gz  File has unexpected size (2527 != 2528). Mirror sync in progress? [IP: 151.101.190.49 80]
15:17:18    Hashes of expected file:
15:17:18     - Filesize:2528 [weak]
15:17:18     - SHA256:1debfc554e873cb9a7551fd1339b571f49d33ba25a9712c84cf3f91d55dc974a
15:17:18     - SHA1:8dc32ec3ac58e6b816f85292bb48deda69162a37 [weak]
15:17:18     - MD5Sum:baed3593f037e02364b7cb4e9bb393a2 [weak]
15:17:18    Release file created at: Mon, 24 Dec 2018 22:26:14 +0000
15:17:18 E: Some index files failed to download. They have been ignored, or old ones used instead.
15:17:18 Error: failed command 'sudo apt-get update'
15:17:18 Error: failed command './buildpkg.sh bpftrace'
15:17:18 Error: failed command './buildall.sh'

links:

• http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/80/console
• http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/84/console
• http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/90/console

[pzakha]

Hopefully that will be fixed by https://jira.delphix.com/browse/LB-117

[gllghr]

http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/143/console

07:18:34 Get:8 http://apt.llvm.org/bionic llvm-toolchain-bionic/main Sources [2525 B]
07:18:34 Err:8 http://apt.llvm.org/bionic llvm-toolchain-bionic/main Sources
07:18:34   File has unexpected size (2535 != 2525). Mirror sync in progress? [IP: 151.101.190.49 80]
07:18:34   Hashes of expected file:
07:18:34    - Filesize:2525 [weak]
07:18:34    - SHA256:0ce938f092e166ffc0b46edd641e38af205ad0b0e2a388ba221a9324b27a6951
07:18:34    - SHA1:7ed461b011f81dc0194695e7bfa00376562e8bca [weak]
07:18:34    - MD5Sum:77499f7be07973bd7bf86487d1d0ab46 [weak]
07:18:34   Release file created at: Wed, 09 Jan 2019 14:23:15 +0000
07:18:34 Get:9 http://apt.llvm.org/bionic llvm-toolchain-bionic/main amd64 Packages [8163 B]
07:18:34 Get:10 http://apt.llvm.org/bionic llvm-toolchain-bionic-5.0/main Sources [2063 B]
07:18:34 Get:11 http://apt.llvm.org/bionic llvm-toolchain-bionic-5.0/main amd64 Packages [7178 B]
07:18:34 Fetched 25.9 kB in 1s (48.7 kB/s)
07:18:36 Reading package lists...
07:18:36 E: Failed to fetch http://apt.llvm.org/bionic/dists/llvm-toolchain-bionic/main/source/Sources.gz  File has unexpected size (2535 != 2525). Mirror sync in progress? [IP: 151.101.190.49 80]
07:18:36    Hashes of expected file:
07:18:36     - Filesize:2525 [weak]
07:18:36     - SHA256:0ce938f092e166ffc0b46edd641e38af205ad0b0e2a388ba221a9324b27a6951
07:18:36     - SHA1:7ed461b011f81dc0194695e7bfa00376562e8bca [weak]
07:18:36     - MD5Sum:77499f7be07973bd7bf86487d1d0ab46 [weak]
07:18:36    Release file created at: Wed, 09 Jan 2019 14:23:15 +0000
07:18:36 E: Some index files failed to download. They have been ignored, or old ones used instead.
07:18:36 Error: failed command 'sudo apt-get update'
07:18:36 Error: failed command './buildpkg.sh bpftrace'
07:18:36 Error: failed command './buildall.sh'

[gllghr]

http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/162/console

[prashks]

Another occurance :
http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/281/console

07:22:59   File has unexpected size (2533 != 2529). Mirror sync in progress? [IP: 151.101.42.49 80]
07:22:59   Hashes of expected file:
07:22:59    - Filesize:2529 [weak]
07:22:59    - SHA256:93a4bc95d3a020e0f7aedc5de542bfda7970c852acfbb2ae48518ae1321e9203
07:22:59    - SHA1:4cc50f732919ac1e75774e5a52b6ee39347a4369 [weak]
07:22:59    - MD5Sum:1ce55bb52948af20d9c745ef2f1ab020 [weak]
07:22:59   Release file created at: Sat, 16 Feb 2019 14:35:39 +0000
07:22:59 Hit:7 http://ppa.launchpad.net/ansible/ansible/ubuntu bionic InRelease
07:22:59 Get:8 http://apt.llvm.org/bionic llvm-toolchain-bionic/main amd64 Packages [8161 B]
07:22:59 Get:9 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
07:22:59 Get:10 http://apt.llvm.org/bionic llvm-toolchain-bionic-5.0/main Sources [2063 B]
07:22:59 Get:11 http://apt.llvm.org/bionic llvm-toolchain-bionic-5.0/main amd64 Packages [7178 B]
07:22:59 Fetched 115 kB in 1s (134 kB/s)
07:23:00 Reading package lists...
07:23:00 E: Failed to fetch http://apt.llvm.org/bionic/dists/llvm-toolchain-bionic/main/source/Sources.gz  File has unexpected size (2533 != 2529). Mirror sync in progress? [IP: 151.101.42.49 80]
07:23:00    Hashes of expected file:
07:23:00     - Filesize:2529 [weak]
07:23:00     - SHA256:93a4bc95d3a020e0f7aedc5de542bfda7970c852acfbb2ae48518ae1321e9203
07:23:00     - SHA1:4cc50f732919ac1e75774e5a52b6ee39347a4369 [weak]
07:23:00     - MD5Sum:1ce55bb52948af20d9c745ef2f1ab020 [weak]
07:23:00    Release file created at: Sat, 16 Feb 2019 14:35:39 +0000
07:23:00 E: Some index files failed to download. They have been ignored, or old ones used instead.
07:23:00 Error: failed command 'sudo apt-get update'
07:23:00 Error: failed command './buildpkg.sh bpftrace'
07:23:00 Error: failed command './buildall.sh'```

[prashks]

Another occurance:
http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/304/console

[pzakha]

Again: http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-build/job/master/job/post-push/315/ (That happened before #19 was integrated)

[jwk404]

Hit again: http://ops.jenkins.delphix.com/job/devops-gate/job/master/job/linux-pkg-update/job/master/job/userland/job/update/176/console

21:24:50     Hashes of expected file:
21:24:50      - Filesize:2599 [weak]
21:24:50      - SHA256:d5ec7ce7f19a45698cf4ebd12f4f7d95db7d4dee4a487d1a9be74a32b65b2344
21:24:50      - SHA1:1fb091eb6eaada370291fcf8dc114b7584f9c406 [weak]
21:24:50      - MD5Sum:3a5c85a13783f163eb99affc12553923 [weak]
21:24:50     Release file created at: Tue, 10 Sep 2019 03:04:56 +0000
21:24:50  E: Some index files failed to download. They have been ignored, or old ones used instead.
21:24:50  Error: failed command 'sudo apt-get update'
21:24:50  Error: Unexpected failure of buildpkg.sh for bpftrace
21:24:50  Error: Script has exited unexpectedly.
21:24:50  
21:24:50  Status Report: FAILURE
21:24:50  Error: failed command './updatelist.sh userland'