Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 865835 - sys-devel/gcc: upgrade may break dev-lang/python:3.11 through app-crypt/libb2[openmp] preventing sys-apps/portage's env-update from updating libgomp.so.1 path in ld.so.cache
Summary: sys-devel/gcc: upgrade may break dev-lang/python:3.11 through app-crypt/libb2...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks:
 
Reported: 2022-08-19 22:54 UTC by Ionen Wolkens
Modified: 2023-09-25 02:07 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info and build.log.txt (emerge-info-build-log.txt.xz,226.72 KB, application/x-xz)
2022-08-20 00:34 UTC, Ionen Wolkens
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ionen Wolkens gentoo-dev 2022-08-19 22:54:14 UTC
Just filing for investigation, lacking full info/log (need to retry).

This should allow reproducing:
1. ensure portage/env-update is using python:3.11 (check python-exec.conf priorities)
2. app-crypt/libb2 built with USE=openmp using libgomp
3. emerge a version of gcc that would replace libb2's currently used libgomp with a different version, e.g.
/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1/libgomp.so.1 ->
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.0/libgomp.so.1
4. when gcc "finish" emerging, odds are things will break and leave with a /var/db/pkg/sys-devel/-MERGING-gcc*

Could be kind of bad given openmp is a profile default on libb2 and this isn't about users randomly adding -fopenmp.
Comment 1 Ionen Wolkens gentoo-dev 2022-08-19 23:44:40 UTC
Still no log due to some nonsense but:

 * Switching native-compiler to x86_64-pc-linux-gnu-12.2.0 ...
ERROR:root:code for hash blake2b was not found.
Traceback (most recent call last):
  File "/usr/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2b
ERROR:root:code for hash blake2s was not found.
Traceback (most recent call last):
  File "/usr/lib/python3.11/hashlib.py", line 307, in <module>
    globals()[__func_name] = __get_hash(__func_name)
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
    return __get_builtin_constructor(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2s
>>> Regenerating /etc/ld.so.cache...
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-08-19 23:54:01 UTC
A workaround (not going to be a full fix for e.g. 11.3.0 -> 12.1.0 though) will be using --with-gcc-major-version-only in GCC given we don't slot based on PV anymore.

Another rough thought to consider (not a serious suggestion, but I said it on IRC, so may as well put it here): the fragility of env-update wrt Python is a bit scary. We've had issues like this before with libffi (not just the preserve-libs bug, but issues in a new ROOT). Our custom ldconfig for musl is in shell. env-update in shell would help with issues like this.
Comment 3 Arfrever Frehtes Taifersar Arahesis 2022-08-20 00:31:29 UTC
Another untested idea:
In toolchain.eclass, at the beginning of function do_gcc_config():
  if [[ -z ${ROOT} ]] ; then
    local -x LD_LIBRARY_PATH="${LIBPATH}"
  fi


(Usage of shell script just makes this script fragile in case of problems with libraries used by grep, sed, awk etc.)
Comment 4 Ionen Wolkens gentoo-dev 2022-08-20 00:34:39 UTC
Created attachment 800205 [details]
emerge --info and build.log.txt

fwiw quick EXTRA_ECONF=--disable-bootstrap downgrade test log (gcc-12.2.0 selected -> gcc-12.1.1_p20220625), but it's not very interesting and just make it seem like it failed over nothing during merge

Tried twice from the same state, worked with libb2[-openmp], but with [openmp]:

# qfile -o /usr/lib/gcc/x86_64-pc-linux-gnu/*
/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.1
# ls -ld /var/db/pkg/sys-devel/*gcc*
drwxr-xr-x 2 root root  800 Aug 19 20:19 /var/db/pkg/sys-devel/-MERGING-gcc-12.1.1_p20220625
drwxr-xr-x 2 root root 4096 Apr 21 23:46 /var/db/pkg/sys-devel/gcc-11.3.0
drwxr-xr-x 2 root root 4096 Aug 19 18:06 /var/db/pkg/sys-devel/gcc-12.2.0
drwxr-xr-x 2 root root 4096 Jun  6 04:45 /var/db/pkg/sys-devel/gcc-config-2.5-r1

Albeit env-update worked after that. I've got mixed results while messing with this, but other tests weren't very controlled (e.g. don't always get the blake messages, or got messages and still properly merged -- initial failure was a long 12.1.1->12.2.0[pgo] which failed to save the log and cleaned the build dir, env-update was failing after until I did EPYTHON=python3.10)
Comment 5 Ionen Wolkens gentoo-dev 2022-08-21 20:15:17 UTC
(In reply to Ionen Wolkens from comment #4)
> Created attachment 800205 [details]
> emerge --info and build.log.txt
> 
> fwiw quick EXTRA_ECONF=--disable-bootstrap downgrade test log (gcc-12.2.0
> selected -> gcc-12.1.1_p20220625), but it's not very interesting and just
> make it seem like it failed over nothing during merge
> 
> Tried twice from the same state, worked with libb2[-openmp], but with
> [openmp]:
Actually, maybe I tested this wrong? I'm not sure if attempting to reproduce while downgrading is quite right. I tried again with libb2[-openmp] but it fails all the same.

In another case where I was upgrading I've got the env-update blake errors (right after it says the switching line), but not for this one -- so maybe it's something else for the downgrade.
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-09-29 21:45:13 UTC
[21:49:15]  <dwfreed> ionen: fwiw, 865835 generally wouldn't be an issue on major gcc upgrades, because they're still slotted by major version, so the old version wouldn't go away
[21:50:13]  <@ionen> yeah this became an issue when stopped doing :x.y.z for every versions I guess
[21:50:42]  <dwfreed> sam_'s suggestion to use --with-gcc-major-version-only would avoid the issue sufficiently
[21:51:08]  <dwfreed> (because minor upgrades would still install to the same path, thus ld.so.cache would still be valid)

I think really we should just bite the bullet and switch. I guess I can look at doing it before the next round of snapshots.

Two points of care:
1. Just do it for > 9 or whatever we changed the slotting scheme for;
2. Make sure gcc-config behaves correctly as a result.
Comment 7 Larry the Git Cow gentoo-dev 2022-10-06 12:55:48 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cac0fc8a9f9d5048410a01caeddfea835cee762a

commit cac0fc8a9f9d5048410a01caeddfea835cee762a
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-06 12:55:27 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-06 12:55:27 +0000

    sys-devel/gcc: add 12.2.1_p20221001
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc/Manifest                    |  1 +
 sys-devel/gcc/gcc-12.2.1_p20221001.ebuild | 52 +++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6cac63bb35eec7400b962ccf83064aed3ab47d88

commit 6cac63bb35eec7400b962ccf83064aed3ab47d88
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-06 12:55:27 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-06 12:55:27 +0000

    sys-devel/gcc: add 11.3.1_p20220930
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc/Manifest                    |  1 +
 sys-devel/gcc/gcc-11.3.1_p20220930.ebuild | 52 +++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1731af7b40321010d3f4baad02bdd50c8f340ec3

commit 1731af7b40321010d3f4baad02bdd50c8f340ec3
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-06 12:55:26 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-06 12:55:26 +0000

    sys-devel/gcc: add 10.4.1_p20220929
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc/Manifest                    |  1 +
 sys-devel/gcc/gcc-10.4.1_p20220929.ebuild | 58 +++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7b7a62d5e3f5e6e05c929f225ead22a2981c9f64

commit 7b7a62d5e3f5e6e05c929f225ead22a2981c9f64
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-06 12:55:26 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-06 12:55:26 +0000

    toolchain.eclass: use --with-major-version-only for newer install paths
    
    Flip over to major version install paths to reflect the
    change in slotting we made between 9 and 10. We only slot
    major versions now, so we should have our install paths
    reflect that for:
    1. principle of least surprise;
    2. we want LDPATH to remain valid between upgrades (bug #865835, bug #873505).
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>
    Closes: https://github.com/gentoo/gentoo/pull/27533
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/toolchain.eclass | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)
Comment 8 Larry the Git Cow gentoo-dev 2022-10-13 21:48:29 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ba2b124849499c618327021ca5ed81a4b142f9de

commit ba2b124849499c618327021ca5ed81a4b142f9de
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-13 21:46:30 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-13 21:48:15 +0000

    sys-devel/gcc: keyword 12.2.1_p20221008
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc/gcc-12.2.1_p20221008.ebuild | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=249c452eb7cf395249c8d23f2dfeb17838fec590

commit 249c452eb7cf395249c8d23f2dfeb17838fec590
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-13 21:46:11 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-13 21:48:14 +0000

    sys-devel/gcc: keyword 11.3.1_p20221007
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc/gcc-11.3.1_p20221007.ebuild | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b26d55e0f0b845ab0e6693abe58b1c517a1eacf8

commit b26d55e0f0b845ab0e6693abe58b1c517a1eacf8
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-10-13 21:45:50 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-10-13 21:48:14 +0000

    sys-devel/gcc: keyword 10.4.1_p20221006
    
    - Switches us over to the newer layout of using major version for
      install directory.
    
    - Includes patch for building Chromium
      (https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=2b2d97fc545635a0f6aa9c9ee3b017394bc494bf).
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-devel/gcc/gcc-10.4.1_p20221006.ebuild | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Comment 9 Larry the Git Cow gentoo-dev 2022-11-22 20:52:06 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3313fb4c7829d383aa6596092c41c377905e711d

commit 3313fb4c7829d383aa6596092c41c377905e711d
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-11-22 20:48:38 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-11-22 20:51:38 +0000

    cuda.eclass: adapt for newer GCC paths (major version only)
    
    Bug: https://bugs.gentoo.org/865835
    Closes: https://bugs.gentoo.org/880675
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/cuda.eclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=14ccbc70d9497a95ba3d093863af2ce2a6702f33

commit 14ccbc70d9497a95ba3d093863af2ce2a6702f33
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2022-11-22 20:48:08 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2022-11-22 20:51:37 +0000

    dev-util/nvidia-cuda-toolkit: allow new GCC paths for 10.x, 11.x
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/880675
    Signed-off-by: Sam James <sam@gentoo.org>

 ...da-toolkit-11.7.0-r2.ebuild => nvidia-cuda-toolkit-11.7.0-r3.ebuild} | 2 +-
 ...da-toolkit-11.8.0-r1.ebuild => nvidia-cuda-toolkit-11.8.0-r2.ebuild} | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Comment 10 Larry the Git Cow gentoo-dev 2023-09-25 02:07:07 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6302dd47e23ee32cfcb446355704ba8bc61b668e

commit 6302dd47e23ee32cfcb446355704ba8bc61b668e
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-09-25 00:05:20 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-09-25 02:06:51 +0000

    toolchain.eclass: fix should_we_gcc_config for major-version slotting
    
    Reported by the-horo on IRC.
    
    For example, with slot as major version:
    ```
    $ gcc-config -c x86_64-pc-linux-gnu
    x86_64-pc-linux-gnu-13
    $ gcc-config -S x86_64-pc-linux-gnu-13
    x86_64-pc-linux-gnu 13
    ```
    so we're indeed comparing 13 with 13.2 and hence we decide to run gcc-config
    unnecessarily because we think it's a major version change.
    
    Fix that by taking into account tc_use_major_version_only and comparing
    based on GCCMAJOR for that case.
    
    Bug: https://bugs.gentoo.org/865835
    Bug: https://bugs.gentoo.org/873505
    Reported-by: the-horo
    Closes: https://github.com/gentoo/gentoo/pull/33042
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/toolchain.eclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)