Caught on the CI, failing for 2 runs in a row, both amd64 and x86. Currently getting: 2020-02-03T16:08:33.4598611Z checking for openssl options with pkg-config... found 2020-02-03T16:08:33.4660307Z configure: pkg-config: SSL_LIBS: "-lssl -lcrypto" 2020-02-03T16:08:33.4665785Z configure: pkg-config: SSL_LDFLAGS: "" 2020-02-03T16:08:33.4666383Z configure: pkg-config: SSL_CPPFLAGS: "" 2020-02-03T16:08:33.5531924Z checking for HMAC_Update in -lcrypto... no 2020-02-03T16:08:33.5994851Z checking for HMAC_Init_ex in -lcrypto... no 2020-02-03T16:08:33.6379300Z checking OpenSSL linking with -ldl... no 2020-02-03T16:08:33.6770590Z checking OpenSSL linking with -ldl and -lpthread... no 2020-02-03T16:08:33.6800164Z configure: OPT_SSL: yes 2020-02-03T16:08:33.6805488Z configure: OPENSSL_ENABLED: 2020-02-03T16:08:33.6806239Z configure: error: --with-ssl was given but OpenSSL could not be detected 2020-02-03T16:08:33.7370622Z 2020-02-03T16:08:33.7372478Z !!! Please attach the following file when seeking support: 2020-02-03T16:08:33.7378185Z !!! /tmp/gentoo/var/tmp/portage/net-misc/curl-7.68.0/work/curl-7.68.0-abi_x86_64.amd64/config.log 2020-02-03T16:08:33.7412411Z * ERROR: net-misc/curl-7.68.0::gentoo failed (configure phase): Sunday 2 of Feb 2020 was the last successful build. The last successful build had this output in the same place: 2020-02-01T16:05:36.0159500Z checking for openssl options with pkg-config... found 2020-02-01T16:05:36.0228663Z configure: pkg-config: SSL_LIBS: "-lssl -lcrypto" 2020-02-01T16:05:36.0229290Z configure: pkg-config: SSL_LDFLAGS: "" 2020-02-01T16:05:36.0229742Z configure: pkg-config: SSL_CPPFLAGS: "" 2020-02-01T16:05:36.0967327Z checking for HMAC_Update in -lcrypto... yes 2020-02-01T16:05:36.1689677Z checking for SSL_connect in -lssl... yes 2020-02-01T16:05:36.2590864Z checking openssl/x509.h usability... yes 2020-02-01T16:05:36.3057855Z checking openssl/x509.h presence... yes 2020-02-01T16:05:36.3064738Z checking for openssl/x509.h... yes 2020-02-01T16:05:36.3738996Z checking openssl/rsa.h usability... yes 2020-02-01T16:05:36.4050716Z checking openssl/rsa.h presence... yes 2020-02-01T16:05:36.4055778Z checking for openssl/rsa.h... yes 2020-02-01T16:05:36.4610006Z checking openssl/crypto.h usability... yes 2020-02-01T16:05:36.4859529Z checking openssl/crypto.h presence... yes 2020-02-01T16:05:36.4862992Z checking for openssl/crypto.h... yes 2020-02-01T16:05:36.5894987Z checking openssl/pem.h usability... yes 2020-02-01T16:05:36.6391167Z checking openssl/pem.h presence... yes 2020-02-01T16:05:36.6391643Z checking for openssl/pem.h... yes 2020-02-01T16:05:36.7487359Z checking openssl/ssl.h usability... yes 2020-02-01T16:05:36.8140367Z checking openssl/ssl.h presence... yes 2020-02-01T16:05:36.8145326Z checking for openssl/ssl.h... yes 2020-02-01T16:05:36.8730943Z checking openssl/err.h usability... yes 2020-02-01T16:05:36.9028188Z checking openssl/err.h presence... yes 2020-02-01T16:05:36.9041997Z checking for openssl/err.h... yes 2020-02-01T16:05:36.9750548Z checking for RAND_egd... no 2020-02-01T16:05:37.0376147Z checking for SSLv2_client_method... no 2020-02-01T16:05:37.1098116Z checking for OpenSSL_version... yes 2020-02-01T16:05:37.1327040Z checking for BoringSSL... no 2020-02-01T16:05:37.1582760Z checking for libressl... no 2020-02-01T16:05:37.2196056Z checking for OpenSSL headers version... 1.1.1 - 0x1010104fL 2020-02-01T16:05:37.2898810Z checking for OpenSSL library version... 1.1.1 2020-02-01T16:05:37.2901645Z checking for OpenSSL headers and library versions matching... yes 2020-02-01T16:05:37.2948927Z checking for "/dev/urandom"... yes 2020-02-01T16:05:37.3662198Z checking for SRP_Calc_client_key in -lcrypto... yes 2020-02-01T16:05:37.3709789Z configure: built with one SSL backend Full log successful build: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1103/logs/41 Full log unsuccessful build (current build): https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1112/logs/41 Sorry, the logs are too big to attach, and I think the problem comes from some previous dependency so I thought it would be better this way. I don't see any change in net-misc/curl: https://packages.gentoo.org/packages/net-misc/curl And I haven't seen any commit in the prefix repo. If you want to find yourself in a docker image with that exact situation I'm reporting you can do: docker pull awesomebytes/gentoo_prefix_ci_stage3:1112 docker run -it awesomebytes/gentoo_prefix_ci_stage3:1112 /bin/bash (Bootstrap is happening in /tmp/gentoo) Reproducible: Always
I have inspected the docker image. The real bug is from binutils as recorded in stage3.log. 29533 >>> Source unpacked in /tmp/gentoo/var/tmp/portage/sys-devel/binutils-2.34/work 29534 * Prefixifying native library path ... 29535 [ ok ] 29536 * Prefixifying path to /etc/ld.so.conf ... 29537 sed: can't read /tmp/gentoo/var/tmp/portage/sys-devel/binutils-2.34/work/binutils-2.34/ld/emultempl/elf32.em: No such file or directory 29538 [ !! ] 29539 >>> Preparing source in /tmp/gentoo/var/tmp/portage/sys-devel/binutils-2.34/work/binutils-2.34 ...
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0e74313ccdf8a00d796f76583a3aece6cf4beb05 commit 0e74313ccdf8a00d796f76583a3aece6cf4beb05 Author: Benda Xu <heroxbd@gentoo.org> AuthorDate: 2020-02-12 12:13:09 +0000 Commit: Benda Xu <heroxbd@gentoo.org> CommitDate: 2020-02-12 12:15:51 +0000 p/f/prefix/s/profile.bashrc: new location of emultempl/elf32.em. From binutils-2.34, /etc/ld.so.conf is coded in ld/ldelf.c instead. Closes: https://bugs.gentoo.org/708184 Signed-off-by: Benda Xu <heroxbd@gentoo.org> profiles/features/prefix/standalone/profile.bashrc | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
Hello, Last night's CI show the same error still happening. This CI was triggered 30min after the commit mentioned in your previous reply. The error you were reporting: sed: can't read /tmp/gentoo/var/tmp/portage/sys-devel/binutils-2.34/work/binutils-2.34/ld/emultempl/elf32.em: No such file or directory Is still in the logs. Latest log from 10h ago from when I post this: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1175/logs/41 With latest docker image: docker pull awesomebytes/gentoo_prefix_ci_stage3:1175 docker run -it awesomebytes/gentoo_prefix_ci_stage3:1175 /bin/bash (In case it matters, the bootstrap is triggered with: LATEST_TREE_YES=1 TESTING_PV=latest ) I also ran in another machine a vanilla bootstrap (with the interactive mode) and it stopped at net-misc/openssh-8.1_p1-r2 with: configure: error: *** working libcrypto not found, check config.log !!! Please attach the following file when seeking support: !!! /tmp/gentoo/var/tmp/portage/net-misc/openssh-8.1_p1-r2/work/openssh-8.1p1/config.log * ERROR: net-misc/openssh-8.1_p1-r2::gentoo failed (configure phase): * econf failed * * Call stack: * ebuild.sh, line 125: Called src_configure * environment, line 2376: Called econf '--with-ldflags=-Wl,-O1 -Wl,--as-needed' '--disable-strip' '--with-pid-dir=/tmp/gentoo/run' '--sysconfdir=/tmp/gentoo/etc/ssh' '--libexecdir=/tmp/gentoo/usr/lib64/misc' '--datadir=/tmp/gentoo/usr/share/openssh' '--with-privsep-path=/tmp/gentoo/var/empty' '--with-privsep-user=sshd' '--without-audit' '--without-kerberos5' '--without-ldns' '--without-libedit' '--without-pam' '--with-pie' '--without-selinux' '--with-openssl' '--with-md5-passwords' '--with-ssl-engine' '--with-hardening' * phase-helpers.sh, line 681: Called __helpers_die 'econf failed' * isolated-functions.sh, line 112: Called die * The specific snippet of code: * die "$@"
I suspect the change had not propagated. How about now, after half a day?
I launched another CI run at the time of posting (and a manual bootstrap in another machine also just in case). In a couple of hours we'll know if it's a matter of propagation. I'll report then. But, that said, what kind of propagation must happen once a new commit hits the main Gentoo git repo? Thanks!
(In reply to Sammy Pfeiffer from comment #5) > I launched another CI run at the time of posting (and a manual bootstrap in > another machine also just in case). In a couple of hours we'll know if it's > a matter of propagation. I'll report then. > > But, that said, what kind of propagation must happen once a new commit hits > the main Gentoo git repo? Thanks! I don't really know. The Gentoo Infra team manage it. I can imagine there some primary servers for each region and some secondary servers and mirrors. Allow for at most 24 hours for them to catch up.
I confirm that CI is still failing due to curl (with LATEST_TREE_YES=1 TESTING_PV=latest). Will re-evaluate tomorrow given what I'm writing next. Also, vanilla manual interactive bootstrap, has succeeded again. But manual interactive bootstrap with LATEST_TREE_YES=1 TESTING_PV=latest fails on net-misc/openssh-8.1_p1-r2 (not net-misc/curl !). This is due to an already reported bug: https://bugs.gentoo.org/699384 net-misc/openssh is not emerged directly on bootstrap-prefix.sh as far as I can see (it must be a dependency of something else).
We've moved forward, this is not an issue anymore, although now a new issue appeared, I've reported it here: https://bugs.gentoo.org/709570
Hello, sorry to change the status of the ticket back. But the bug is back. net-misc/curl-7.68.0 fails in the same place than the initially reported: configure: pkg-config: SSL_LIBS: "-lssl -lcrypto" configure: pkg-config: SSL_LDFLAGS: "" configure: pkg-config: SSL_CPPFLAGS: "" checking for HMAC_Update in -lcrypto... no checking for HMAC_Init_ex in -lcrypto... no checking OpenSSL linking with -ldl... no checking OpenSSL linking with -ldl and -lpthread... no configure: OPT_SSL: yes configure: OPENSSL_ENABLED: configure: error: --with-ssl was given but OpenSSL could not be detected !!! Please attach the following file when seeking support: !!! /tmp/gentoo/var/tmp/portage/net-misc/curl-7.68.0/work/curl-7.68.0-abi_x86_64.amd64/config.log * ERROR: net-misc/curl-7.68.0::gentoo failed (configure phase): * econf failed The last successful build in the CI happened in 6th of March after a streak of 9 days of successful builds. Previous to that, there were 2 nightly builds that also failed with the same error (no changes upstream in Gentoo Prefix in all this 15 days period up to now) and a couple successful ones. For the sake of comparison, the logs of the last successful build: Stage 1: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1326/logs/18 Stage 2: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1326/logs/30 Stage 3: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1326/logs/41 Logs of the last failed build: Stage 1: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1332/logs/18 Stage 2: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1332/logs/30 Stage 3 (failing): https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1332/logs/41 Comparing Stage 3 logs, I found that dev-libs/openssl-1.1.1d-r3 has a difference: In the failing logs the patch: * Applying openssl-1.1.1d-config-Drop-linux-alpha-gcc-bwx.patch ... [ ok ] Is NOT being applied. Other than that... both logs for emerging dev-libs/openssl-1.1.1d-r3 look more or less the same (some ordering of lines due to -j2 changes, but that's about it). You can find these logs separated here (too big to attach): openssl emerge on the successful curl bootstrap: https://gist.github.com/awesomebytes/d851cffe6c37d5c47f0d2c8a0e5e3d49 openssl emerge on the failing curl bootstrap: https://gist.github.com/awesomebytes/0521e94de9e8957c91480a62f915fa45 Grepping for libcrypto in both logs gives nothing. Grepping for HMAC_Update gives nothing. If anyone else could take a look into it, that would be great. I'm lost. In order to get into a machine with the latest successful stage3 bootstrap you can do: docker pull awesomebytes/gentoo_prefix_ci_stage3:1326 docker run -it awesomebytes/gentoo_prefix_ci_stage3:1326 /bin/bash To get yourself into the latest failing stage3 bootstrap: docker pull awesomebytes/gentoo_prefix_ci_stage3:1353 docker run -it awesomebytes/gentoo_prefix_ci_stage3:1353 /bin/bash
I just noticed a build on x86 managed to build curl-7.69.0. In case it helps.
Can't update on this issue as now: https://bugs.gentoo.org/712020 Stops the bootstrap earlier than when this bug happened. (But I can confirm that curl 7.69.0 does build).
Gentoo Prefix is bootstrapping again (2 successful daily builds worked in both amd64 and x86). The bug is gone, closing.
Yay!
*** Bug 721268 has been marked as a duplicate of this bug. ***
This may be an issue of the CI platform... but weirdly on x86 the bootstrap is stuck in stage 3 at this point: 2020-05-12T02:56:28.8791249Z >>> Emerging (2 of 2) sys-apps/coreutils-8.32-r1::gentoo 2020-05-12T02:56:29.2757868Z * coreutils-8.32.tar.xz BLAKE2B SHA512 size ;-) ... [ ok ] 2020-05-12T02:56:29.2758899Z * coreutils-8.30-patches-01.tar.xz BLAKE2B SHA512 size ;-) ... [ ok ] 2020-05-12T02:56:29.8232786Z >>> Unpacking source... 2020-05-12T02:56:29.8253043Z >>> Unpacking coreutils-8.32.tar.xz to /tmp/gentoo/var/tmp/portage/sys-apps/coreutils-8.32-r1/work (Full log: https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/build/builds/1894/logs/40 ) I've tried re-running it 3 times (and there are 4 instances more running right now, just in case), and it never finishes unpacking. On amd64 it does go through, though. Saying it here in case you have any idea of what may be happening.
(In reply to Sammy Pfeiffer from comment #15) > This may be an issue of the CI platform... but weirdly on x86 the bootstrap > is stuck in stage 3 at this point: > > 2020-05-12T02:56:28.8791249Z >>> Emerging (2 of 2) > sys-apps/coreutils-8.32-r1::gentoo > 2020-05-12T02:56:29.2757868Z * coreutils-8.32.tar.xz BLAKE2B SHA512 size > ;-) ... [ ok ] > 2020-05-12T02:56:29.2758899Z * coreutils-8.30-patches-01.tar.xz BLAKE2B > SHA512 size ;-) ... [ ok ] > 2020-05-12T02:56:29.8232786Z >>> Unpacking source... > 2020-05-12T02:56:29.8253043Z >>> Unpacking coreutils-8.32.tar.xz to > /tmp/gentoo/var/tmp/portage/sys-apps/coreutils-8.32-r1/work > > (Full log: > https://dev.azure.com/12719821/e566c963-8f77-4f01-b7bc-ae2d91b1334f/_apis/ > build/builds/1894/logs/40 ) > > I've tried re-running it 3 times (and there are 4 instances more running > right now, just in case), and it never finishes unpacking. On amd64 it does > go through, though. > > Saying it here in case you have any idea of what may be happening. Sammy, please go ahead and open a separate bug.
