Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 838925 - sys-devel/binutils-2.38-r2: lto1: internal compiler error: original not compressed with zstd
Summary: sys-devel/binutils-2.38-r2: lto1: internal compiler error: original not compr...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: ARM64 Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on: 834720
Blocks:
  Show dependency tree
 
Reported: 2022-04-17 09:28 UTC by HougeLangley
Modified: 2023-08-03 20:30 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
full build log (Full_Build_Log.txt,251.40 KB, text/plain)
2022-04-17 09:29 UTC, HougeLangley
Details
emerge --info (emerge--info.txt,6.51 KB, text/plain)
2022-04-17 09:32 UTC, HougeLangley
Details
full build log (new) (Full_Build_Log.txt,251.89 KB, text/plain)
2022-04-26 08:05 UTC, HougeLangley
Details

Note You need to log in before you can comment on or make changes to this bug.
Description HougeLangley 2022-04-17 09:28:51 UTC
Update to sys-devel/binutils-2.38-r2 failed.

libtool: install: (cd /var/tmp/portage/sys-devel/binutils-2.38-r2/image/usr/lib64/binutils/aarch64-unknown-linux-gnu/2.38 && { ln -s -f libctf-nobfd-2.38.gentoo-sys-devel-binutils-st.so libctf-nobfd.so || { rm -f libctf-nobfd.so && ln -s libctf-nobfd-2.38.gentoo-sys-devel-binutils-st.so libctf-nobfd.so; }; })
libtool: install: /usr/lib/portage/python3.9/ebuild-helpers/xattr/install -c .libs/libctf-nobfd.lai /var/tmp/portage/sys-devel/binutils-2.38-r2/image/usr/lib64/binutils/aarch64-unknown-linux-gnu/2.38/libctf-nobfd.la
libtool: install: warning: remember to run `libtool --finish /usr/lib64/binutils/aarch64-unknown-linux-gnu/2.38'
make[3]: Leaving directory '/var/tmp/portage/sys-devel/binutils-2.38-r2/work/build/libctf'
make[2]: Leaving directory '/var/tmp/portage/sys-devel/binutils-2.38-r2/work/build/libctf'
lto1: internal compiler error: original not compressed with zstd
0xb0ce2f internal_error(char const*, ...)
        ???:0
0xc3dbfb lto_end_uncompression(lto_compression_stream*, lto_compression)
        ???:0
0xc3d273 lto_get_section_data(lto_file_decl_data*, lto_section_type, char const*, int, unsigned long*, bool)
        ???:0
0xca80fb cgraph_node::get_untransformed_body()
        ???:0
0xca6c1b cgraph_node::expand()
        ???:0
0x125789b symbol_table::compile()
        ???:0
0x124f2a3 lto_main()
        ???:0
Please submit a full bug report,

Reproducible: Always
Comment 1 HougeLangley 2022-04-17 09:29:57 UTC
Created attachment 771095 [details]
full build log
Comment 2 HougeLangley 2022-04-17 09:31:27 UTC
 ~ emerge -pqv '=sys-devel/binutils-2.38-r2::gentoo'
[ebuild     U ] sys-devel/binutils-2.38-r2 [2.38-r1] USE="gold nls plugins (-cet) (-default-gold) -doc -multitarget -pgo* -static-libs -test -vanilla"
Comment 3 HougeLangley 2022-04-17 09:32:18 UTC
Created attachment 771098 [details]
emerge --info
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-04-17 09:32:44 UTC
I'm guessing something like built some stuff on system w/ LTO but not USE=zstd on gcc, then enabled USE=zstd on GCC, now LTO builds fail for the stuff you built earlier?

What happens if you try building binutils and binutils-libs with LTO off (and USE=-pgo) as a one-off, then try again with it back on?
Comment 5 HougeLangley 2022-04-17 09:34:17 UTC
(In reply to Sam James from comment #4)
> I'm guessing something like built some stuff on system w/ LTO but not
> USE=zstd on gcc, then enabled USE=zstd on GCC, now LTO builds fail for the
> stuff you built earlier?
> 
> What happens if you try building binutils and binutils-libs with LTO off
> (and USE=-pgo) as a one-off, then try again with it back on?

OK, Thanks, I will try ;)
Comment 6 Andreas K. Hüttel archtester gentoo-dev 2022-04-25 21:57:16 UTC
(In reply to HougeLangley from comment #5)
> (In reply to Sam James from comment #4)
> > I'm guessing something like built some stuff on system w/ LTO but not
> > USE=zstd on gcc, then enabled USE=zstd on GCC, now LTO builds fail for the
> > stuff you built earlier?
> > 
> > What happens if you try building binutils and binutils-libs with LTO off
> > (and USE=-pgo) as a one-off, then try again with it back on?
> 
> OK, Thanks, I will try ;)

Any news here?
Comment 7 HougeLangley 2022-04-26 08:05:50 UTC
Created attachment 774600 [details]
full build log (new)
Comment 8 HougeLangley 2022-04-26 08:08:19 UTC
(In reply to Andreas K. Hüttel from comment #6)
> (In reply to HougeLangley from comment #5)
> > (In reply to Sam James from comment #4)
> > > I'm guessing something like built some stuff on system w/ LTO but not
> > > USE=zstd on gcc, then enabled USE=zstd on GCC, now LTO builds fail for the
> > > stuff you built earlier?
> > > 
> > > What happens if you try building binutils and binutils-libs with LTO off
> > > (and USE=-pgo) as a one-off, then try again with it back on?
> > 
> > OK, Thanks, I will try ;)
> 
> Any news here?

The problem still. I have been -avueDN @world. Widely USE=-lto -pgo, just enable USE=zstd.
Comment 9 eostre 2022-05-12 01:26:43 UTC
To add, experiencing this too with binutils-2.37_p1-r2 on amd64.
Have tried USE=-pgo, no change.
I could have sworn I built binutils-2.37 just fine a month ago, too; this is not a fresh install.
Comment 10 eostre 2022-05-13 01:13:48 UTC
Was able to reproduce in a fresh chroot.

Rebuilt entire default chroot just fine with default USE; went to pull in my custom make.conf, emerge -avuDN went fine including rebuilding binutils; emerge -av1'd binutils again, got error.
Something in my make.conf (presumably USE flags) causes the build failure, but only after the system has been recompiled.

Relevant USE is `pgo lto graphite jit lzo lzop zstd lzma lz4`. Currently recompiling whole system without `pgo lto graphite jit`, will update when finished.
Comment 11 Georgy Yakovlev archtester gentoo-dev 2022-05-15 06:58:29 UTC
also hit this on ppc64 on fresh ppc64le stage.
as expected it manifested after rebuilding gcc with zstd

trying to figure out what went wrong
Comment 12 eostre 2022-05-15 13:59:54 UTC
Recompiling without `pgo...jit` was a no-go; once the system has been touched by this bug it seems unable to compile a functioning binutils ever again. Started over in a fresh chroot.

Building the system with `pgo jit graphite` works and results in being able to build binutils however many times one wishes. Adding `lto` introduces the bug.

I'm going to give one more go with just `USE=lto` and report back. I feel I should buy my computer a cookie after forcing it to do nothing but build the GNU toolchain + python for the past 72 hours.
Comment 13 eostre 2022-05-15 16:54:01 UTC
USE=`lto` only works; the combination of `lto` and `pgo graphite jit` (and maybe zstd?) does not.
Calling it quits for now; searching out exactly what combinations of those flags triggers this bug would burn more CPU time than I am willing to.

Since a bugged system cannot fix itself, I am building an un-bugged binutils in chroot, quickpkg'ing it, and installing it in the host. Suggest similar amelioration for all such affected systems.
Comment 14 Georgy Yakovlev archtester gentoo-dev 2022-05-15 19:07:27 UTC
I left "emerge -e world" at night, still can't build binutils after full rebuild with zstd-enabled gcc. so probably not the deps are to blame and it needs more digging into gcc/binutils
Comment 15 Georgy Yakovlev archtester gentoo-dev 2022-05-15 20:15:57 UTC
found a way to recover from this:

install older binutils temprorary, I used sys-devel/binutils-2.36.1-r2

emerge -C newer binutils

build new binutils (will work fine)
rebuild binutils again for good measure

so as long as non-zstd-gcc built binutils of same version are present it's impossible to build new ones with zstd-gcc


so it seems binutils somehow tries to link against host binutils, not the just built ones.
Comment 16 Georgy Yakovlev archtester gentoo-dev 2022-05-15 20:17:56 UTC
^ to add to previous comment.

remove old binutils after succesfully building new ones first time.
Comment 17 Georgy Yakovlev archtester gentoo-dev 2022-05-15 21:03:01 UTC
exerpt from IRC conversation:


-Wl,/usr/lib64/binutils/powerpc64le-unknown-linux-gnu/2.37_p1 is added to the linker args

failure happens in src_install, not in src_compile

the reason seems to be different tooldir value passed between 2 phases:

> emake tooldir="${EPREFIX}${TOOLPATH}" all
> emake DESTDIR="${D}" tooldir="${EPREFIX}${LIBPATH}" install

seems to be an intended difference with unintended effect


different value of tooldir causes re-link to happen at src_install phase, because of change of paths.

so linker gets -Wl,/usr/lib64/binutils/powerpc64le-unknown-linux-gnu/2.37_p1 arg.
as long as this path is not empty and contains incompatible lto bytecode linking against it will fail.


I will dig deeper later, but feel free to pick up.
Comment 18 Andreas K. Hüttel archtester gentoo-dev 2022-06-11 19:05:50 UTC
> 
> different value of tooldir causes re-link to happen at src_install phase,
> because of change of paths.
> 
> so linker gets -Wl,/usr/lib64/binutils/powerpc64le-unknown-linux-gnu/2.37_p1
> arg.
> as long as this path is not empty and contains incompatible lto bytecode
> linking against it will fail.
> 

(mostly as note to self) This is the comment left by slyfox in the ebuild:

# Note [tooldir hack for ldscripts]
# ---------------------------------
# Build system does not allow ./configure to tweak every location
# we need for slotting binutils hence all the shuffling in
# src_install(). This note is about SCRIPTDIR define handling.
#
# SCRIPTDIR defines 'ldscripts/' directory location. SCRIPTDIR value
# is set at build-time in ld/Makefile.am as: 'scriptdir = $(tooldir)/lib'
# and hardcoded as -DSCRIPTDIR='"$(scriptdir)"' at compile time.
# Thus we can't just move files around after compilation finished.
#
# Our goal is the following:
# - at build-time set scriptdir to point to symlinked location:
#   ${TOOLPATH}: /usr/${CHOST} (or /usr/${CHOST}/${CTARGET} for cross-case)
# - at install-time set scriptdir to point to slotted location:
#   ${LIBPATH}: /usr/$(get_libdir)/binutils/${CTARGET}/${PV}
Comment 19 Andreas K. Hüttel archtester gentoo-dev 2022-06-11 22:57:16 UTC
Problem introduced by 

commit b023986de7d466d361798bae98f45f8ba7a42e8a (HEAD)
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: Sat Apr 7 11:40:14 2018 +0100
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: Sat Apr 7 11:42:41 2018 +0100

    sys-devel/binutils: fix ldscrips ${CTARGET} search path, bug #147155
    
    This change fixes long-standing search path issue in Gentoo's binutils:
    
    Before the change search path was the following:
        /usr/${CTARGET}/lib/ldscripts
    Note: it points to $SYSROOT, not to native cross-tools.
    
    After the change search path is the following:
        /usr/${CHOST}/${CTARGET}/lib/ldscripts
    
    Added two notes to the ebuild on how things are supposed to work:
    - Note [slotting support]
    - Note [tooldir hack for ldscripts]
    
    Applied change to 2.30-r1 and live ebuilds.
    
    Reported-by: Heiko Rosemann
    Closes: https://bugs.gentoo.org/147155
    Package-Manager: Portage-2.3.28, Repoman-2.3.9
Comment 20 Larry the Git Cow gentoo-dev 2022-06-12 19:46:44 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8ea456d072e46530ea53f04fe8935693fab59093

commit 8ea456d072e46530ea53f04fe8935693fab59093
Author:     Andreas K. Hüttel <dilfridge@gentoo.org>
AuthorDate: 2022-06-12 19:43:40 +0000
Commit:     Andreas K. Hüttel <dilfridge@gentoo.org>
CommitDate: 2022-06-12 19:46:30 +0000

    sys-devel/binutils: Prevent relinking on install, try 1
    
    The tooldir hack introduced to solve bug 147155 makes libtool
    relink libraries during the installation phase. This fails in
    enough cases to be an annoyance. Use the hack only for cross
    installations.
    
    *Untested*, needs some careful observation before regaining keywords.
    
    Bug: https://bugs.gentoo.org/147155
    Bug: https://bugs.gentoo.org/834720
    Bug: https://bugs.gentoo.org/838925
    Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>

 sys-devel/binutils/binutils-2.37_p1-r3.ebuild | 476 +++++++++++++++++++++++++
 sys-devel/binutils/binutils-2.38-r3.ebuild    | 484 ++++++++++++++++++++++++++
 sys-devel/binutils/binutils-9999.ebuild       |  21 +-
 3 files changed, 978 insertions(+), 3 deletions(-)
Comment 21 Andreas K. Hüttel archtester gentoo-dev 2022-07-10 08:26:51 UTC
(In reply to Larry the Git Cow from comment #20)
> The bug has been referenced in the following commit(s):
> 
> https://gitweb.gentoo.org/repo/gentoo.git/commit/
> ?id=8ea456d072e46530ea53f04fe8935693fab59093
> 
> commit 8ea456d072e46530ea53f04fe8935693fab59093
> Author:     Andreas K. Hüttel <dilfridge@gentoo.org>
> AuthorDate: 2022-06-12 19:43:40 +0000
> Commit:     Andreas K. Hüttel <dilfridge@gentoo.org>
> CommitDate: 2022-06-12 19:46:30 +0000
> 
>     sys-devel/binutils: Prevent relinking on install, try 1
>     
>     The tooldir hack introduced to solve bug 147155 makes libtool
>     relink libraries during the installation phase. This fails in
>     enough cases to be an annoyance. Use the hack only for cross
>     installations.
>     
>     *Untested*, needs some careful observation before regaining keywords.
>     
>     Bug: https://bugs.gentoo.org/147155
>     Bug: https://bugs.gentoo.org/834720
>     Bug: https://bugs.gentoo.org/838925
>     Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>
> 
>  sys-devel/binutils/binutils-2.37_p1-r3.ebuild | 476
> +++++++++++++++++++++++++
>  sys-devel/binutils/binutils-2.38-r3.ebuild    | 484
> ++++++++++++++++++++++++++
>  sys-devel/binutils/binutils-9999.ebuild       |  21 +-
>  3 files changed, 978 insertions(+), 3 deletions(-)

I reverted this because it didnt help in bug 834720 (which only affects 2.37 though).
The binaries were relinked anyway, which also actually makes sense.
Just the order of paths makes no sense.
Comment 22 Andreas K. Hüttel archtester gentoo-dev 2022-07-23 13:02:48 UTC
OK so I'm not yet able to reproduce this. Tried to build both binutils 2.37 and 2.38 with gcc 11 and 12 with and without USE=zstd.

Please, if you hit this bug, try to add a complete set of information:

* which gcc's are installed, with which useflags (in particular, lto pgo zstd ?), and which gcc is enabled?

* which binutils are installed, with which useflags, and which binutils is enabled?

* what are your CFLAGS?

* what rebuild precisely fails? (targeted version, used binutils and gcc)
Comment 23 Andreas K. Hüttel archtester gentoo-dev 2022-07-25 23:07:18 UTC
(In reply to Andreas K. Hüttel from comment #22)
> OK so I'm not yet able to reproduce this. Tried to build both binutils 2.37
> and 2.38 with gcc 11 and 12 with and without USE=zstd.
> 
> Please, if you hit this bug, try to add a complete set of information:
> 
> * which gcc's are installed, with which useflags (in particular, lto pgo
> zstd ?), and which gcc is enabled?
> 
> * which binutils are installed, with which useflags, and which binutils is
> enabled?
> 
> * what are your CFLAGS?
> 
> * what rebuild precisely fails? (targeted version, used binutils and gcc)

I can reproduce the bug now, but only with 2.37; 2.38 always builds fine.
No idea what changed.

Let's not have this block stabilization of 2.38 anymore.