I assigned this to release for the meantime. Based on comments by jmbsvicetto in IRC it appears that this is likely a python issue in python-2.6.6-r2. The affected stages were already removed from the mirrors wrt bug 357835. jmbsvicetto, can you post the details of what you found with the stage generation? If it's the python ebuild/eclass for sure, please reassign this to them.
So it gets documented, here is what I found out about the python default interpreter and stage building: atlantis@atl4core /home/release $ ls -la buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python* lrwxrwxrwx 1 root root 14 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python -> python-wrapper lrwxrwxrwx 1 root root 9 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python2 -> python2.6 -rwxr-xr-x 1 root root 10080 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python2.6 lrwxrwxrwx 1 root root 9 Mar 8 08:10 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python3 -> python3.1 -rwxr-xr-x 1 root root 10080 Mar 8 08:10 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python3.1 -rwxr-xr-x 1 root root 217 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-config -rwxr-xr-x 1 root root 1200 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-config-2.6 -rwxr-xr-x 1 root root 1177 Mar 8 08:10 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-config-3.1 -rwxr-xr-x 1 root root 10080 Mar 8 09:07 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-wrapper atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/etc/env.d/python/config python2.6 atlantis@atl4core /home/release $ tail -f tmp/catalyst-auto.982/log/hardened_stage3.log >>> Installing (75 of 127) sys-apps/diffutils-2.8.7-r2 >>> Emerging (76 of 127) dev-libs/openssl-1.0.0d >>> Installing (76 of 127) dev-libs/openssl-1.0.0d >>> Emerging (77 of 127) app-misc/ca-certificates-20090709 >>> Installing (77 of 127) app-misc/ca-certificates-20090709 >>> Emerging (78 of 127) net-misc/curl-7.20.0-r2 >>> Installing (78 of 127) net-misc/curl-7.20.0-r2 >>> Emerging (79 of 127) dev-perl/Net-SSLeay-1.36 >>> Installing (79 of 127) dev-perl/Net-SSLeay-1.36 >>> Emerging (80 of 127) net-misc/iputils-20100418-r1 >>> Installing (80 of 127) net-misc/iputils-20100418-r1 >>> Emerging (81 of 127) dev-perl/IO-Socket-SSL-1.35 >>> Installing (81 of 127) dev-perl/IO-Socket-SSL-1.35 >>> Emerging (82 of 127) dev-perl/Net-SMTP-SSL-1.01 >>> Installing (82 of 127) dev-perl/Net-SMTP-SSL-1.01 >>> Emerging (83 of 127) dev-vcs/git-1.7.3.4-r1 >>> Installing (83 of 127) dev-vcs/git-1.7.3.4-r1 >>> Emerging (84 of 127) dev-lang/python-3.1.3-r1 ^C atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/etc/env.d/python/config python2.6 atlantis@atl4core /home/release $ tail -f -n1 tmp/catalyst-auto.982/log/hardened_stage3.log >>> Emerging (84 of 127) dev-lang/python-3.1.3-r1 >>> Installing (84 of 127) dev-lang/python-3.1.3-r1 >>> Emerging (85 of 127) dev-python/setuptools-0.6.14 ^C atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/etc/env.d/python/config python2.6 atlantis@atl4core /home/release $ tail -f -n1 tmp/catalyst-auto.982/log/hardened_stage3.log >>> Emerging (85 of 127) dev-python/setuptools-0.6.14 >>> Installing (85 of 127) dev-python/setuptools-0.6.14 >>> Emerging (86 of 127) dev-lang/python-2.6.6-r2 ^C atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/etc/env.d/python/config python2.6 atlantis@atl4core /home/release $ tail -f -n1 tmp/catalyst-auto.982/log/hardened_stage3.log >>> Emerging (86 of 127) dev-lang/python-2.6.6-r2 ^C atlantis@atl4core /home/release $ ls -la buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python* lrwxrwxrwx 1 root root 14 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python -> python-wrapper lrwxrwxrwx 1 root root 9 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python2 -> python2.6 -rwxr-xr-x 1 root root 10080 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python2.6 lrwxrwxrwx 1 root root 9 Mar 8 08:10 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python3 -> python3.1 -rwxr-xr-x 1 root root 10080 Mar 8 09:52 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python3.1 -rwxr-xr-x 1 root root 217 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-config -rwxr-xr-x 1 root root 1200 Mar 8 08:08 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-config-2.6 -rwxr-xr-x 1 root root 1177 Mar 8 09:52 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-config-3.1 -rwxr-xr-x 1 root root 10080 Mar 8 09:07 buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/usr/bin/python-wrapper atlantis@atl4core /home/release $ tail -f -n1 tmp/catalyst-auto.982/log/hardened_stage3.log >>> Emerging (86 of 127) dev-lang/python-2.6.6-r2 >>> Installing (86 of 127) dev-lang/python-2.6.6-r2 >>> Emerging (87 of 127) sys-libs/cracklib-2.8.16 ^C atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/hardened/stage3-amd64-hardened-20110308/etc/env.d/python/config python3.1 As can be seen above, it's python-2.6.6-r2 that switches the default to python3.1. At this time I suspect it might be caused by the ebuild using EROOT / EPREFIX while it's defined as EAPI-2. I'm currently running tests to see if this is true. I've tested chrooting to the stage3, switching python default interpreter back to python2.6 and emerged python-2.6.6-r2 again. That did not touch the default python version.
python-2.6.6-r2.ebuild contains the following code, which should work in EAPI="2": [[ -z "${EROOT}" ]] && EROOT="${ROOT%/}${EPREFIX}/"
I've completed a new build run where I changed the EAPI of python-2.6.6-r2 to 3 and the default python version is correctly set at python2.6: atlantis@atl4core /home/release $ tail tmp/catalyst-auto.10995/log/stage3.log Cleaning chroot: /root/*... Cleaning chroot: /usr/portage... Cleaning chroot: /etc/portage... --- Running action sequence: capture Creating stage tarball... --- Running action sequence: clear_autoresume Removing AutoResume Points: ... Emptying directory /home/release/buildroot/amd64-dev/tmp/default/.autoresume-stage3-amd64-20110308/ lockfile does not exist '/home/release/buildroot/amd64-dev/tmp/default/stage3-amd64-20110308/.catalyst_lock' lockfile does not exist '/home/release/buildroot/amd64-dev/tmp/default/stage3-amd64-20110308/.catalyst_lock' atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/default/stage3-amd64-20110308/etc/env.d/python/config python2.6 atlantis@atl4core /home/release $ cat buildroot/amd64-dev/tmp/default/stage3-amd64-20110308/var/db/pkg/dev-lang/python-2.6.6-r2/EAPI 3 atlantis@atl4core /home/release $ I'll try to configure catalyst to produce the build logs so I can test Arfrever's patch. In any case, I think we should "bump" the EAPI to version 3.
I forgot to mention it, but the following is the patch Arfrever suggested so we can try to understand better what's going on during the stage build --- python-2.6.6-r2.ebuild +++ python-2.6.6-r2.ebuild @@ -308,15 +308,30 @@ } eselect_python_update() { + set -x + + eselect python show + eselect python show --python2 + eselect python show --python3 + declare -p ROOT + declare -p EPREFIX + declare -p EROOT + [[ -z "${EROOT}" ]] && EROOT="${ROOT%/}${EPREFIX}/" if [[ -z "$(eselect python show)" || ! -f "${EROOT}usr/bin/$(eselect python show)" ]]; then + ls -l "${EROOT}usr/bin/$(eselect python show)" eselect python update + ls -l "${EROOT}usr/bin/$(eselect python show)" fi if [[ -z "$(eselect python show --python${PV%%.*})" || ! -f "${EROOT}usr/bin/$(eselect python show --python${PV%%.*})" ]]; then + ls -l "${EROOT}usr/bin/$(eselect python show --python${PV%%.*})" eselect python update --python${PV%%.*} + ls -l "${EROOT}usr/bin/$(eselect python show --python${PV%%.*})" fi + + set +x } pkg_postinst() {
Infra: This issue has continued into the stages generated on Mar. 17th. I only checked AMD64/x86 so far, but given the history, I think it's a safe bet that all were affected. Not to mention I don't have the time to download all the stages to check. So can we please take down all the stages generated around Mar. 17. Python / Release: This issue has persisted into the stages generated on Mar. 17. Has any progress been made to determine where the issue is? I don't really have the experience with Catalyst etc to dig into this myself or I would. Release/Infra: In the mean time, can we stop generating stages until this is resolved. I see no sense in pulling them down every week and reverting back to the Feb. ones until we know what's wrong and it is addressed. Thanks everyone.
Note, sparc doesn't seem to be affected. I'm going to guess the stages we had issues with in bug 357835 will be the problematic ones this time around as well.
Infra: As per your request, here is the exact list of stages that need to get nuked. stage3-amd64-hardened+nomultilib-20110317.tar.bz2 stage3-amd64-hardened-20110317.tar.bz2 stage3-amd64-20110317.tar.bz2 stage3-hppa1.1-20110318.tar.bz2 stage3-hppa2.0-20110318.tar.bz2 stage3-ppc-20110320.tar.bz2 stage3-ppc64-32ul-20110320.tar.bz2 stage3-ppc64-64ul-20110320.tar.bz2 stage3-i686-hardened-20110322.tar.bz2 stage3-i486-20110322.tar.bz2 stage3-i686-20110322.tar.bz2 Thanks.
(In reply to comment #7) > Infra: As per your request, here is the exact list of stages that need to get > nuked. Removed now.
Removing infra from this stale bug report. I strongly encourage some pre-validation before copying files to the staging location by the releng team to avoid this hassle in the future. (as applicable)
Fixed with the currently generated stages. Closing.
