While bootstrapping prefix, rebuilding of binutils fails due to redefinition of iovec. This problem has been mentioned earlier in bug 574794 , but since its a separate problem, I am filing it separately. To reproduce: * Remove nls from bootstrap-prefix.sh script to avoid hitting bug 574794 * Remove emerge --depclean in the end of stage3 in the bootstrap-prefix.sh script ( otherwise binutils are unmerged ). Compilation fais in emerge -e system phase. System at which it was tested: current Gentoo with sys-devel/binutils-2.25.1-r1. Same problem was on CentOS release 6.5 (Final) Error message: x86_64-pc-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold -I/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold -I/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/../include -I/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/../elfcpp -DLOCALEDIR="\"/shared/envs/sysbio/gentoo-1602/usr/share/binutils-data/x86_64-pc-linux-gnu/2.24/locale\"" -DBINDIR="\"/shared/envs/sysbio/gentoo-1602/usr/x86_64-pc-linux-gnu/binutils-bin/2.24\"" -DTOOLBINDIR="\"/shared/envs/sysbio/gentoo-1602/usr/x86_64-pc-linux-gnu/bin\"" -DTOOLLIBDIR="\"/shared/envs/sysbio/gentoo-1602/usr/x86_64-pc-linux-gnu/lib\"" -W -Wall -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=fileread.o -O2 -pipe -O2 -pipe -MT fileread.o -MD -MP -MF .deps/fileread.Tpo -c -o fileread.o /shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc In file included from /shared/envs/sysbio/gentoo-1602/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.5/include/g++-v4/ext/hash_map:60:0, from /shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/system.h:92, from /shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/gold.h:35, from /shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc:24: /shared/envs/sysbio/gentoo-1602/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.5/include/g++-v4/backward/backward_warning.h:32:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp] #warning \ ^ /shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc:88:8: error: redefinition of 'struct iovec' struct iovec { void* iov_base; size_t iov_len; }; ^ In file included from /usr/include/bits/fcntl-linux.h:38:0, from /usr/include/bits/fcntl.h:61, from /usr/include/fcntl.h:35, from /shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc:29: /usr/include/bits/uio.h:43:8: error: previous definition of 'struct iovec' struct iovec ^ Makefile:836: recipe for target 'fileread.o' failed make[4]: *** [fileread.o] Error 1 make[4]: Leaving directory '/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/build/gold' Makefile:859: recipe for target 'all-recursive' failed make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory '/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/build/gold' Makefile:613: recipe for target 'all' failed make[2]: *** [all] Error 2 make[2]: Leaving directory '/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/build/gold' Makefile:6013: recipe for target 'all-gold' failed make[1]: *** [all-gold] Error 2 make[1]: Leaving directory '/shared/envs/sysbio/gentoo-1602/var/tmp/portage/sys-devel/binutils-2.24-r2/work/build' Makefile:830: recipe for target 'all' failed make: *** [all] Error 2
I see this on CentOS 5.4
confirming on 6.5
On these hosts it seems that the initial installation of binutils-2.24 succeeds, because of USE=-cxx which disables gold (c++ code). After that, c++ compilation fails during linking with missing symbols for _Unwind_Resume_or_Rethrow.
Problem appears to be that our gcc doesn't think -lgcc_s is needed (which holds this symbol). When adding, it links fine.
I pushed a workaround in the bootstrap-prefix.sh script. It continued sofar, but needs to be seen if it is enough.
workaround in bootstrap-prefix.sh seems to work (for me on RHEL 6.5)
Unfortunately this fix did not work for me on RHES 6.2. Everything did work when I moved [[ ${CHOST} != *-darwin* ]] && export CXX="${CHOST}-g++ -lgcc_s" in front of emerge_pkgs --nodeps "${pkgs[@]}" || return 1 This was last week and the bottstrap-prefix.sh seems to have changed a bit...
Reopening this bug, as bootstraping still doesn't work on CentOS 7.2 either. I get the following error: /micfs/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc:88:8: error: redefinition of 'struct iovec' In file included from /usr/include/bits/fcntl-linux.h:38:0, from /usr/include/bits/fcntl.h:61, from /usr/include/fcntl.h:35, from /micfs/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc:29: /usr/include/bits/uio.h:43:8: error: previous definition of 'struct iovec'
I confirm the bug on CentOS 6.7 with error: In file included from /usr/include/bits/fcntl.h:28:0, from /usr/include/fcntl.h:34, from /home/alberto/gentoo/tmp/var/tmp/portage/sys-devel/binutils-2.24-r2/work/binutils-2.24/gold/fileread.cc:29: /usr/include/bits/uio.h:44:8: error: previous definition of 'struct iovec'
Could you please test if RAP works on the affected platforms? https://wiki.gentoo.org/wiki/Prefix/libc
Created attachment 436634 [details] Prefix RAP failing on stage3: stage3 log Host: Gentoo AMD64
Created attachment 436636 [details] Prefix RAP failing on stage3: perl build log Host: Gentoo AMD64
prefix-rap bootstrap failed on stage3 while compiling perl (see below). Host: Gentoo amd64. Prefix environment is a way users can customize their setup in the system. That's why we use Prefix on top of regular Gentoo.
(In reply to Marko Vendelin from comment #13) > prefix-rap bootstrap failed on stage3 while compiling perl (see below). > Host: Gentoo amd64. Prefix environment is a way users can customize their > setup in the system. That's why we use Prefix on top of regular Gentoo. Interesting. Is your EPREFIX/bin/sh pointing to EPREFIX/tmp/bin/sh ? Please try $ export EPREFIX=/shared/envs/sysbio/gentoo-1606-rap $ ldd ${EPREFIX}/bin/sh #paste the output $ readelf -ld ${EPREFIX}/bin/sh #paste the output $ rm $EPREFIX/bin/sh $ ln -s bash $EPREFIX/bin/sh $ ./bootstrap-rap.sh # continue from the last failure point.
Here we go. From ls bin/ you could see that $EPREFIX/bin/bash is already linked to sh. So, trick by making a new symbolic link didn't help. Here is the output of the commands (note that since starting from command #3 there was no difference introduced into the system => bootstrap failed on continuation immediately due to missing ncurses lib). ############# ls -l $EPREFIX/bin/ ... -rwxr-xr-x 1 sysbio-admin sysbio 709488 Jun 6 16:36 bash ... lrwxrwxrwx 1 sysbio-admin sysbio 4 Jun 6 16:36 sh -> bash ############# ldd ${EPREFIX}/bin/sh linux-vdso.so.1 (0x00007ffee15d2000) libreadline.so.6 => /lib64/libreadline.so.6 (0x00007f443924e000) libhistory.so.6 => /lib64/libhistory.so.6 (0x00007f4439045000) libncurses.so.6 => not found libc.so.6 => /lib64/libc.so.6 (0x00007f4438ca3000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00007f4438a50000) /lib64/ld-linux-x86-64.so.2 (0x00007f4439497000) ############# readelf -ld ${EPREFIX}/bin/sh Elf file type is EXEC (Executable file) Entry point 0x409180 There are 10 program headers, starting at offset 64 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040 0x0000000000000230 0x0000000000000230 R E 8 INTERP 0x0000000000000270 0x0000000000400270 0x0000000000400270 0x000000000000001c 0x000000000000001c R 1 [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x00000000000a86e4 0x00000000000a86e4 R E 200000 LOAD 0x00000000000a8de0 0x00000000006a8de0 0x00000000006a8de0 0x0000000000003e30 0x000000000000db10 RW 200000 DYNAMIC 0x00000000000a8df8 0x00000000006a8df8 0x00000000006a8df8 0x0000000000000200 0x0000000000000200 RW 8 NOTE 0x000000000000028c 0x000000000040028c 0x000000000040028c 0x0000000000000020 0x0000000000000020 R 4 GNU_EH_FRAME 0x00000000000949e0 0x00000000004949e0 0x00000000004949e0 0x0000000000002e74 0x0000000000002e74 R 4 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 10 GNU_RELRO 0x00000000000a8de0 0x00000000006a8de0 0x00000000006a8de0 0x0000000000000220 0x0000000000000220 R 1 PAX_FLAGS 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 8 Section to Segment mapping: Segment Sections... 00 01 .interp 02 .interp .note.ABI-tag .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 03 .init_array .fini_array .jcr .dynamic .got .got.plt .data .bss 04 .dynamic 05 .note.ABI-tag 06 .eh_frame_hdr 07 08 .init_array .fini_array .jcr .dynamic .got 09 Dynamic section at offset 0xa8df8 contains 27 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libreadline.so.6] 0x0000000000000001 (NEEDED) Shared library: [libhistory.so.6] 0x0000000000000001 (NEEDED) Shared library: [libncurses.so.6] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000000c (INIT) 0x405fb0 0x000000000000000d (FINI) 0x4799a4 0x0000000000000019 (INIT_ARRAY) 0x6a8de0 0x000000000000001b (INIT_ARRAYSZ) 8 (bytes) 0x000000000000001a (FINI_ARRAY) 0x6a8de8 0x000000000000001c (FINI_ARRAYSZ) 8 (bytes) 0x000000006ffffef5 (GNU_HASH) 0x4002b0 0x0000000000000005 (STRTAB) 0x402890 0x0000000000000006 (SYMTAB) 0x400658 0x000000000000000a (STRSZ) 4977 (bytes) 0x000000000000000b (SYMENT) 24 (bytes) 0x0000000000000015 (DEBUG) 0x0 0x0000000000000003 (PLTGOT) 0x6a9000 0x0000000000000002 (PLTRELSZ) 6552 (bytes) 0x0000000000000014 (PLTREL) RELA 0x0000000000000017 (JMPREL) 0x404618 0x0000000000000007 (RELA) 0x403f70 0x0000000000000008 (RELASZ) 1704 (bytes) 0x0000000000000009 (RELAENT) 24 (bytes) 0x000000006ffffffe (VERNEED) 0x403ee0 0x000000006fffffff (VERNEEDNUM) 1 0x000000006ffffff0 (VERSYM) 0x403c02 0x0000000000000000 (NULL) 0x0
(In reply to Marko Vendelin from comment #15) > Here we go. From ls bin/ you could see that $EPREFIX/bin/bash is already > linked to sh. So, trick by making a new symbolic link didn't help. Here is > the output of the commands (note that since starting from command #3 there > was no difference introduced into the system => bootstrap failed on > continuation immediately due to missing ncurses lib). Thanks Marko, I understand the root cause now. emerge failed to pick up newly available tools in stage3. I have updated the bootstrap script, and verified an old Gentoo stable 13.0 worked.[1] As I have also updated the stage repository, please test again by starting from scratch. Sorry guys. I have been abusing this sys-devel/binutils bug report. Hopefully with Prefix glibc we can circumvent this headache. 1. https://wiki.gentoo.org/wiki/Prefix/libc#Tested_Distributions
(In reply to Benda Xu from comment #16) > Thanks Marko, I understand the root cause now. emerge failed to pick up > newly available tools in stage3. I have updated the bootstrap script, and > verified an old Gentoo stable 13.0 worked.[1] As I have also updated the > stage repository, please test again by starting from scratch. Brenda, now it went through. The catch with RAP seems to be related to the use of LDAP (or any other non-standard authentication) in the host. We use LDAP and to connect to the main server we have to use some binding passwords that users don't know. As a result, if I understand correctly, I cannot configure nss_ldap in prefix, since the binding passwords are unknown. The workaround seems to be addition of prefix user(s) into prefix $EPREFIX/etc/{passwd,group} During the bootstrap, the process stopped in stage 3 due to the lack of user in prefix environment (bootstrap was performed by a user specified in LDAP). After adding that user in $EPREFIX/etc/{passwd,group}, I was able to finish the bootstrap.
(In reply to Marko Vendelin from comment #17) > Benda, now it went through. > > The catch with RAP seems to be related to the use of LDAP (or any other > non-standard authentication) in the host. We use LDAP and to connect to the > main server we have to use some binding passwords that users don't know. As > a result, if I understand correctly, I cannot configure nss_ldap in prefix, > since the binding passwords are unknown. The workaround seems to be addition > of prefix user(s) into prefix > > $EPREFIX/etc/{passwd,group} > > During the bootstrap, the process stopped in stage 3 due to the lack of user > in prefix environment (bootstrap was performed by a user specified in LDAP). > After adding that user in $EPREFIX/etc/{passwd,group}, I was able to finish > the bootstrap. Thanks Marko! Glad it worked. The uid/gid mapping issue is long standing. We don't have an ultimate answer. I have summarized what we know at, https://wiki.gentoo.org/wiki/Prefix/libc#Username_becomes_invalid_inside_RAP Extensions welcome if a better solution is developed.
(In reply to Benda Xu from comment #18) > Thanks Marko! Glad it worked. The uid/gid mapping issue is long standing. > We don't have an ultimate answer. I have summarized what we know at, > > > https://wiki.gentoo.org/wiki/Prefix/libc#Username_becomes_invalid_inside_RAP > > Extensions welcome if a better solution is developed. In our system, getent does not return passwd part that comes from LDAP. Maybe, just a simple script that runs in host environment through /home/users and populates RAP passwd/group would do.
(In reply to Marko Vendelin from comment #19) > In our system, getent does not return passwd part that comes from LDAP. > Maybe, just a simple script that runs in host environment through > /home/users and populates RAP passwd/group would do. Great idea! I have updated, https://wiki.gentoo.org/wiki/Prefix/libc#Username_becomes_invalid_inside_RAP to document how to enumerate users on /home/*
(In reply to Benda Xu from comment #10) > Could you please test if RAP works on the affected platforms? > > https://wiki.gentoo.org/wiki/Prefix/libc I could finish building successfully on RHEL 6.2. Unfortunately, prefix bash seems to source /etc/profile instead of ${EPREFIX}/etc/profile, which renders it unusable. I managed to get it working somewhat by adding --noprofile to startprefix and sourcing ${EPREFIX}/etc/profile manually. Any ideas on how to fix this?
Hi nanikata, (In reply to nanikata from comment #21) > I could finish building successfully on RHEL 6.2. > Unfortunately, prefix bash seems to source /etc/profile instead of > ${EPREFIX}/etc/profile, which renders it unusable. > I managed to get it working somewhat by adding --noprofile to startprefix > and sourcing ${EPREFIX}/etc/profile manually. > Any ideas on how to fix this? Sorry it is a bug of bash. Please sync the rap repo, https://wiki.gentoo.org/wiki/Prefix/libc#Synchronize_with_the_RAP_repository Then install the newest bash with 'emerge -1 bash'.
(In reply to Benda Xu from comment #22) > Sorry it is a bug of bash. Please sync the rap repo, > > > https://wiki.gentoo.org/wiki/Prefix/libc#Synchronize_with_the_RAP_repository > > Then install the newest bash with 'emerge -1 bash'. Thank you, that fixed it and everything seems to be working properly.
in short, this is known as RAP, which is working successfully for this
(In reply to Fabian Groffen from comment #24) > in short, this is known as RAP, which is working successfully for this Yay!
