Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 672042 - sys-devel/gcc-8.2.0-r4 fail to emerge with gcov error collect-ld ld: Invalid argument
Summary: sys-devel/gcc-8.2.0-r4 fail to emerge with gcov error collect-ld ld: Invalid ...
Status: RESOLVED CANTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL: https://bugs.launchpad.net/ubuntu/+so...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-11-27 06:12 UTC by Sammy Pfeiffer
Modified: 2018-11-28 19:46 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
gcc-build-logs.tar.bz2 (gcc-build-logs.tar.bz2,222.54 KB, application/x-bzip)
2018-11-27 06:12 UTC, Sammy Pfeiffer
Details
emerge info command (gcc_4.2.0-r4_emerge_info.txt,5.02 KB, text/plain)
2018-11-27 06:12 UTC, Sammy Pfeiffer
Details
emerge pqv command (gcc_4.2.0-r4_emerge -pqv.txt,16.68 KB, text/plain)
2018-11-27 06:13 UTC, Sammy Pfeiffer
Details
Last 500 lines of the build log (last500lines_sys-devel_gcc-8.2.0-r4_temp_build.log,905.43 KB, text/plain)
2018-11-27 06:13 UTC, Sammy Pfeiffer
Details
dmesg command just after error (dmesg.txt,69.16 KB, text/plain)
2018-11-28 02:43 UTC, Sammy Pfeiffer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sammy Pfeiffer 2018-11-27 06:12:17 UTC
Created attachment 556434 [details]
gcc-build-logs.tar.bz2

Hello, I'm bootstrapping Gentoo Prefix and on Stage 3 it fails due to gcc 8.2.0-r4 not compiling.

The error:

/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc/collect-ld: 106: exec: /tmp/gentoo/usr/x86_64-pc-linux-gnu/bin/ld: Invalid argument
collect2: error: ld returned 2 exit status
Makefile:2935: recipe for target 'gcov' failed
make[3]: *** [gcov] Error 1


I generated a Docker image with the system state that you can use to inspect the state and try stuff (note: this is Gentoo Prefix bootstrapped in /tmp/gentoo, so you have everything gentoo-related under /tmp/gentoo, the Docker is a Ubuntu 16.04):

  docker pull awesomebytes/gentoo_prefix_latest_image
  docker run -it awesomebytes/gentoo_prefix_latest_image /bin/bash


Attaching gcc-build-logs.tar.bz2
Comment 1 Sammy Pfeiffer 2018-11-27 06:12:45 UTC
Created attachment 556436 [details]
emerge info command
Comment 2 Sammy Pfeiffer 2018-11-27 06:13:06 UTC
Created attachment 556438 [details]
emerge pqv command
Comment 3 Sammy Pfeiffer 2018-11-27 06:13:29 UTC
Created attachment 556440 [details]
Last 500 lines of the build log
Comment 4 Sammy Pfeiffer 2018-11-27 10:58:57 UTC
I investigated a bit more. If you run that docker image...

And you modify /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc/collect-ld to echo the command that is being executed (at the end of collect-ld):

  *)
    echo "Executing:"
    echo $original ${1+"$@"}
    exec $original ${1+"$@"}
    ;;
esac

And you run the same command (I'm not sure if should be from the prev-gcc folder or the gcc folder, but they have the same result):

cd /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-gcc

/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc/xg++ -B/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc/ -B/tmp/gentoo/usr/x86_64-pc-linux-gnu/bin/ -nostdinc++ -B/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -B/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/libsupc++/.libs  -isystem /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu  -isystem /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/include  -isystem /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/gcc-8.2.0/libstdc++-v3/libsupc++ -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/libsupc++/.libs -no-pie -fno-PIE    -m64 -O2 -pipe -O2 -pipe -gtoggle -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc  gcov.o \
> hash-table.o ggc-none.o libcommon.a ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a  -o gcov

You get the actual command executed:

/tmp/gentoo/usr/x86_64-pc-linux-gnu/bin/ld --eh-frame-hdr -m elf_x86_64 -dynamic-linker /tmp/gentoo/lib64/ld-linux-x86-64.so.2 -o gcov /tmp/gentoo/usr/lib/../lib64/crt1.o /tmp/gentoo/usr/lib/../lib64/crti.o /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc/crtbegin.o -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/libsupc++/.libs -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc -L/tmp/gentoo/usr/x86_64-pc-linux-gnu/bin -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -L/tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/prev-x86_64-pc-linux-gnu/libstdc++-v3/libsupc++/.libs -L/tmp/gentoo/lib/../lib64 -L/tmp/gentoo/usr/lib/../lib64 -L/tmp/gentoo/lib -L/tmp/gentoo/usr/lib gcov.o hash-table.o ggc-none.o libcommon.a ../libcpp/libcpp.a ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a -Bstatic -lstdc++ -Bdynamic -lm -lgcc -lgcc_eh -lc -lgcc -lgcc_eh /tmp/gentoo/var/tmp/portage/sys-devel/gcc-8.2.0-r4/work/build/./prev-gcc/crtend.o /tmp/gentoo/usr/lib/../lib64/crtn.o

Which works...

I actually ran again the bootstrapping of Gentoo Prefix in that same image (just doing ./bootstrap-prefix.sh again) and gcc-8.2.0-r4 emerged correctly.


Hmmm I have no idea what is going on. In summary if I just try to emerge again gcc-8.2.0-r4 after the failed emerge it emerges correctly the second time. But the first time is consistently failing.
Comment 5 Sammy Pfeiffer 2018-11-27 13:20:16 UTC
I ran the bootstrap of Gentoo Prefix on a 16 core machine with 12GB RAM with -j8 this time (the previous runs were run on a 2 core machine 7GB RAM with -j2) and... this error didn't happen.
Comment 6 Sergei Trofimovich gentoo-dev 2018-11-27 20:27:06 UTC
Did you have a dmesg output by chance? I wonder if it was an OOM.
Comment 7 Sammy Pfeiffer 2018-11-27 23:01:14 UTC
I don't have a dmesg output unfortunately. But I'll send the job again adding a dmesg command.
Comment 8 Sammy Pfeiffer 2018-11-28 02:43:37 UTC
Created attachment 556550 [details]
dmesg command just after error
Comment 9 Sammy Pfeiffer 2018-11-28 02:44:20 UTC
The dmesg output shows on OOM error. The only meaningful error is:

2018-11-28T00:47:42.2545234Z [ 6043.603811] Invalid argument reading file caps for /tmp/gentoo/usr/x86_64-pc-linux-gnu/bin/ld

Which doesn't help much.
Comment 10 Sergei Trofimovich gentoo-dev 2018-11-28 08:12:18 UTC
(In reply to Sammy Pfeiffer from comment #9)
> The dmesg output shows on OOM error. The only meaningful error is:
> 
> 2018-11-28T00:47:42.2545234Z [ 6043.603811] Invalid argument reading file
> caps for /tmp/gentoo/usr/x86_64-pc-linux-gnu/bin/ld
> 
> Which doesn't help much.

The error comes from kernel:
    https://github.com/torvalds/linux/blob/638820d8da8ededd6dc609beaef02d5396599c03/security/commoncap.c#L679

	rc = get_vfs_caps_from_disk(bprm->file->f_path.dentry, &vcaps);
	if (rc < 0) {
		if (rc == -EINVAL)
			printk(KERN_NOTICE "Invalid argument reading file caps for %s\n",
bprm->filename);

https://github.com/torvalds/linux/blob/638820d8da8ededd6dc609beaef02d5396599c03/security/commoncap.c#L578

    int get_vfs_caps_from_disk(const struct dentry *dentry, struct cpu_vfs_cap_data *cpu_caps)
    {
        ....

Kernel blocks ld execution because it failed to read ld's caps from extended attributes (-EINVAL status).

Here we have a bunch of possible places that return -EINVAL. If the failure is indeed memory-related it should be ENOMEM.

As you use host debian kernel I suggest reporting the error to Debian. It will likely not solve your problem but should yield clearer error message about memory exhaustion (or something else).
Comment 11 Sammy Pfeiffer 2018-11-28 08:36:40 UTC
(In reply to Sergei Trofimovich from comment #10)
> As you use host debian kernel I suggest reporting the error to Debian. It
> will likely not solve your problem but should yield clearer error message
> about memory exhaustion (or something else).


The kernel of the machine where this is running is (first dmesg line):

Linux version 4.15.0-1023-azure (buildd@lcy01-amd64-025) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)) #24~16.04.1-Ubuntu SMP Wed Aug 29 12:54:36 UTC 2018 (Ubuntu 4.15.0-1023.24~16.04.1-azure 4.15.18)


I found this bug report from Ubuntu that "I think" reports a similar error: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1736808

Which they seem to report that it's fixed in a posterior kernel version. I lack the skills to follow this error deeper.

I guess there is nothing we can do, it's probably an OOM error but reported in a weird way. I'll give a go on running a job with -j1 to see if that would help.
Comment 12 Sammy Pfeiffer 2018-11-28 13:29:12 UTC
(In reply to Sammy Pfeiffer from comment #11)
> I guess there is nothing we can do, it's probably an OOM error but reported
> in a weird way. I'll give a go on running a job with -j1 to see if that
> would help.

Doing -j1 yielded the same results unfortunately.
Comment 13 Sergei Trofimovich gentoo-dev 2018-11-28 19:46:29 UTC
The bug you linked is not about OOM condition specifically but about invalid entries after directory move.

My guess is that when you don't have a lot of RAM kernel evicts already cached ld inode from RAM and has to reload it. But due to a host kernel bug reload from disk fails with -EINVAL.

Looking at the ubuntu bug you linked it's not yet backported anywhere and is planned only for next ubuntu release (if I read status correctly).

I'm afraid Gentoo can't do much about it. You might want to ask ubuntu devs to reconsider backporting a fix to stable kernels.