This is a very annoying bug - even today, about two months after I encountered it, I feel the anger... The situation ------------- Imagine this: you are cross-compiling (say, from 32-bit to 64-bit, as I was at the time), you have barely built (recompiled) a base system - then you realize you must compile binutils (sys-devel/binutils). Old or new version, is irrelevant. What IS relevant though, is that you DON'T have your usual 3300 packages, all in perfect form around - you are on a MINIMAL system, not on STEROIDS! And then comes this binutils package that wants AT ALL COSTS to build its docs - with a special tool, that DOES NOT COMPILE on your minimal system for some miniscule, irrelevant, irreparable, unknown, fine reason. Fine binutils NEEDS to build its fine 'chew' tool - to spit (after chewing, that is) its fine documentation at you - that you DON'T NEED AT ALL at that point! 'chew', in turn, coughs at something that is missing (I don't remember what exactly, didn't take notes at the time and I really, really don't care) - so you cannot build binutils, therefore you cannot build anything, you can stop here... What I tried ------------ I tried to unset the 'doc' USE flag (it was unset anyway...). No chance. binutils insists on building its docs - no 'chew', no docs - no docs, no binutils - no binutils, no system. Very, very annoying... Who I cursed ------------ You can imagine... Anyway, it's good that I am writing this two months later - I wouldn't be so...moderate otherwise! :devil: Solution -------- The true solution would be for the toolchain-binutils.eclass to check the 'doc' USE flag and allow building the docs through 'chew' ONLY in the case 'doc' is set. I resorted in putting the following code just before the end of the tc-binutils_unpack() function: # Prevent 'make info' if doc is unset. if ! use doc ; then # DELETE the 'doc' subdirectories! # This is radical, but is the only way to avoid things like # the creation of the 'chew' program with the purpose of # 'chewing' documenbtation (before spitting it, that is...). # The problem with programs created during compile time, # like chew (in the bfd dir) is: THEY WILL *NOT* RUN! # I REPEAT: THEY WILL NOT RUN - if we are cross-compiling... for subdir in $(find ${WORKDIR}/ -type d -name 'doc'); do rm -rf $subdir done # The above is NOT enough, though! # We must eradicate any mention of 'doc/Makefile' # in all configure* scripts! That includes configure.ac # AND configure scripts - take no prisoners! find ${WORKDIR}/ -name 'configure.ac' | xargs sed -i.bak -e '/AC_CONFIG_FILES/s/doc\/Makefile//' find ${WORKDIR}/ -name 'configure' | xargs sed -i.bak -e '/ac_config_files/s/doc\/Makefile//' find ${WORKDIR}/ -name 'Makefile*' | xargs sed -i.bak -e 's/all *diststuff: *info/diststuff: info/' find ${WORKDIR}/ -name 'Makefile*' | xargs sed -i.bak -e 's/SUBDIRS *= *doc *po/SUBDIRS = po/' fi Moral ----- Try to see things from a 'cage' (like cross-compiling) and not from a palace, when it comes to binutils. Keep it not just 'simple and stupid' - but basic too, as basic as you can, with minimal requirements.
It is very hard to follow what you have written. Please do: - summarize the issue in 1-2 sentences. - attach 'build.log' and 'emerge --info' - attach a patch in a patch form Thank you!
O.K. it seems I did take some notes after all - so here are some more details (I just copy/paste from those notes): ####################### NOTES: START ####################### During compilation in bfd/doc, binutils breaks with "exec format error" - of course, because it builds its own (64-bit) tool (chew) to spit its docs inside a 32-bit system... :roll: ##################################################################################### /bin/sh /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/../../move-if-change \ chw$$ chew; \ touch chew.stamp ./chew -f /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/doc.str < /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/../aoutx.h >aoutx.tmp ./chew -f /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/doc.str < /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/../archive.c >archive.tmp /bin/sh: ./chew: cannot execute binary file: Exec format error Makefile:848: recipe for target 'aoutx.stamp' failed make[3]: *** [aoutx.stamp] Error 126 ##################################################################################### Passing EXTRA_ECONF=--without-docs on the command line: CBUILD=i686-pc-linux-gnu EXTRA_ECONF=--without-docs x86_64-pc-linux-gnu-emerge -uva --keep-going binutils does NOT work. I had to change the eclass /usr/portage/eclass/toolchain-binutils.eclass [...here are some changes to see, but only part of the final ones, which I posted above...] (NOTE: After two days of trying, there are many more changes than the above!) After going past the 'make info in doc' step that was causing trouble with 'chew', compilation stopped at another point: ##################################################################################### make[3]: Leaving directory '/usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/build/binutils' gcc -c -I. -I/usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/binutils -W -Wall -Wstrict-prototy pes -Wmissing-prototypes -Wshadow /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/binutils/s yslex_wrap.c gcc -o sysinfo sysinfo.o syslex_wrap.o ./sysinfo -d </usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/binutils/sysroff.info >sysroff.h /bin/sh: ./sysinfo: cannot execute binary file: Exec format error Makefile:1367: recipe for target 'sysroff.h' failed make[2]: *** [sysroff.h] Error 126 ##################################################################################### Again, gcc (the 64-bit cross-compiler!) is used to create a program, sysinfo, that is used in turn to produce a (needed?) sysroff.h header file. It is clear that this programming practice will NOT work during cross-compilation! The newly created sysinfo executable is 64-bit and CANNOT run on a 32-bit build host! :roll: ####################### NOTES: END ####################### NOTE: At the time I was writing this, I was still on the 32-bit system trying to cross-compile the base 64-bit system. Due to this unfortunate binutils error, I had to abandon at that point and continue on the 64-bit machine - prematurely, as it turned out (something that cost me extra blood, sweat and tears). Conclusion ---------- One should need no 'chew' and no 'sysinfo' for such a basic tool like binutils! At least (and this is th epoint of this report) the eclass should do all at its power to avoid building 'chew' if 'doc' is not set. This would at least eliminate the first problem. I should probably open a new bug report (upstrem?) for the second - but I don't have the nerves...
Sergei, you are right in asking for a patch - there are a few more changes to the code... The patch I will attach is against /usr/portage/eclass/toolchain-binutils.eclass of sys-apps/portage-2.3.51-r1. The toolchain-binutils.eclass with my changes comes from a slightly older version, though (I just realized that). Nevertheless, I tried to create a patch that will patch /usr/portage/eclass/toolchain-binutils.eclass with my changes only (avoiding changes due to the different versions). Anyway, you will see what changes are required by just looking at the patch, I am sure.
Created attachment 564160 [details, diff] Patch for toolchain-binutils.eclass
toolchain-binutils.eclass already has quite a few hacks to evade 'makeinfo' and friends. Release tarballs contain built docs. To justify the need of change we need to understand what fails for you and why it fails. I can't reproduce your failure locally. binutils supports cross-compilation just fine and should not try to run foreign binaries. Here is my ARCH=86 chroot session (it has no x86_64 interpreter): # crossdev -t x86_64-pc-linux-gnu # x86_64-pc-linux-gnu-emerge -v1 binutils:2.27 # x86_64-pc-linux-gnu-emerge -v1 binutils Both installed fine. Please provide at least failed 'build.log', 'emerge --info' and 'x86_64-pc-linux-gnu-emerge --info'. Maybe we'll find the culprit.
The problem with 'build.log', 'emerge --info' etc. now is that I have already decommissioned the 32-bit machine where all this happened. I might try to boot it again and search - but this will take some time. Besides, it is not certain I still have *that* PORTAGE_TMPDIR available, where extraction and compilation took place... I know what you mean when you say that binutils-2.27 installs fine with crossdev - I was there too. I mean, that's how one starts cross-compiling, according to the crossdev docs. That was not the problem. As I am writing this from memory, the problem *seems* to be that, at some point, I must have started a 'emerge -1auv @system' - without excluding binutils... ...right, I found it in my notes, I did: x86_64-pc-linux-gnu-emerge -uva --keep-going @system So the question to you is: what happens when you do that? I tell you what will happen - here are my notes again (to save you the trouble of getting through the obstacles, supposing you do this on a 32-bit system, trying to cross-compile for x86_64): ##################### NOTES: START ##################### This, as it will have to run inside the chroot environment, will not manage to compile all 110 packages of the @system set. The problem is mainly that the configure script(s) will fail at the very first text, namely compiling and running conftest.c. Here, conftest.c is the bare minimum of a C program: ##################################################################################### /* confdefs.h */ #define PACKAGE_NAME "GNU coreutils" #define PACKAGE_TARNAME "coreutils" #define PACKAGE_VERSION "8.25" #define PACKAGE_STRING "GNU coreutils 8.25" #define PACKAGE_BUGREPORT "bug-coreutils@gnu.org" #define PACKAGE_URL "http://www.gnu.org/software/coreutils/" #define PACKAGE "coreutils" #define VERSION "8.25" /* end confdefs.h. */ int main () { ; return 0; } ##################################################################################### The configure scripts of the ebuilds compile it with x86_64-pc-linux-gnu-gcc -O2 -march=native -pipe -O2 -march=native -pipe -Wl,-O1 -Wl,--as-needed conftest.c where march and the other options are set in the make.conf file that drives the compilation, in our case: /usr/x86_64-pc-linux-gnu/etc/portage/make.conf The compilation creates an a.out executable, which cannot be run, because it is 64-bit and we are inside a 32-bit system: ##################################################################################### configure:4504: checking whether we are cross compiling configure:4512: x86_64-pc-linux-gnu-gcc -o conftest -O2 -pipe -fomit-frame-pointer -Wl,-O1 -Wl,--as-needed conftest.c >&5 configure:4516: $? = 0 configure:4523: ./conftest /usr/x86_64-pc-linux-gnu/tmp/portage/dev-libs/libpipeline-1.4.1/work/libpipeline-1.4.1/configure: line 4525: ./conftest: cannot execute binary file: Exec format error configure:4527: $? = 126 configure:4534: error: in `/usr/x86_64-pc-linux-gnu/tmp/portage/dev-libs/libpipeline-1.4.1/work/libpipeline-1.4.1_build': configure:4536: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. See `config.log' for more details ##################################################################################### To avoid the nasty 'error: cannot run C compiled programs' from the configure scripts when cross-compiling, set CBUILD=i686-pc-linux-gnu before the invocation: CBUILD=i686-pc-linux-gnu x86_64-pc-linux-gnu-emerge -uva --keep-going binutils For some reason namely, crossdev overwrites CBUILD from its make.conf file and uses CBUILD=x86_64-pc-linux-gnu (you can see that in the output of the binutils configure script). Setting CBUILD to a value different from CHOST will motivate the configure script to go past that silly "check whether C compiler works" test while cross-compiling ##################### NOTES: END ##################### At this point is where binutils is recompiled and the error(s) happen as described above, in my previous post. That is, my notes continue with the notes I posted in my previous post. I should probably have excluded binutils from the 'emerge ... @system' command...but perhaps all this helps you find the culprit. Anyway, if you look at the errors in my previous post, you will see that they happen for binutils-2.27 - that is my emerge command above CBUILD=i686-pc-linux-gnu x86_64-pc-linux-gnu-emerge -uva --keep-going binutils caused binutils-2.27 to be re-built, which led to those 'chew' errors...? Maybe the CBUILD/CHOST values I used played a role? To reproduce it, you will have to run this on a 32-bit machine, after the steps you already did... ...or after the steps *I* did - I almost forgot: if you want to see the exact steps of how I started this, get a cup of coffee and go read https://bugs.gentoo.org/show_bug.cgi?id=620422 It was a long journey, but I am writing this on the cross-compiled 64-bit system - works like a charm (after two months of struggle)!
(In reply to segmentation fault from comment #6) > The problem with 'build.log', 'emerge --info' etc. now is that I have > already decommissioned the 32-bit machine where all this happened. I might > try to boot it again and search - but this will take some time. Besides, it > is not certain I still have *that* PORTAGE_TMPDIR available, where > extraction and compilation took place... If you don't use this setup anymore then no need to do an extra effort to extract details. We can debug the problem as in happens in a new bug if you encounter it again. Having exact details is crucial here. Closing as NEEDINFO. > As I am writing > this from memory, the problem *seems* to be that, at some point, I must have > started a 'emerge -1auv @system' - without excluding binutils... Installing binutils into target should not cause problems. > So the question to you is: what happens when you do that? I get 90 of 105 packages compiled fine. Even after them reemerging binutils and binutils:2.27 goes fine. > #define PACKAGE_STRING "GNU coreutils 8.25" This is not a binutils package. > /usr/x86_64-pc-linux-gnu/tmp/portage/dev-libs/libpipeline-1.4.1/work/ > libpipeline-1.4.1/configure: line 4525: ./conftest: cannot execute binary This is not a binutils package either. > https://bugs.gentoo.org/show_bug.cgi?id=620422 That bug is about single bit flip in bad memory? Sounds irrelevant to toolchain-binutils.eclass or cross-compilation.
Just to be clear here: This bug we are talking about here happened to me on a 32-bit system during cross-compilation of binutils-2.27 for x86_64. How it came to happen is documented above - I quote: During compilation in bfd/doc, binutils breaks with "exec format error" - of course, because it builds its own (64-bit) tool (chew) to spit its docs inside a 32-bit system... :roll: ##################################################################################### /bin/sh /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/../../move-if-change \ chw$$ chew; \ touch chew.stamp ./chew -f /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/doc.str < /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/../aoutx.h >aoutx.tmp ./chew -f /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/doc.str < /usr/x86_64-pc-linux-gnu/tmp/portage/sys-devel/binutils-2.27/work/binutils-2.27/bfd/doc/../archive.c >archive.tmp /bin/sh: ./chew: cannot execute binary file: Exec format error Makefile:848: recipe for target 'aoutx.stamp' failed make[3]: *** [aoutx.stamp] Error 126 ##################################################################################### --- end-of-quote To avoid this, I had to make the changes shown in the patch I posted. The changes implement the following logic: if the 'doc' USE flag is not set, then do not build the docs. This is because the docs in binutils require an extra program ('chew', you can see it in the part of build.log in the quote above, which I had copied in my notes) and this program must be compiled first. When you cross-compile on 32-bit for 64-bit, the compiled 'chew' will be 64-bit and it will not run on your 32-bit machine to produce the docs - cross-compiling binutils will fail. The conftest.c example is from coreutils, but I had copied it in my notes as an *example*. A conftest like this is (it seems) at the start of every configure script - and I wanted to avoid the error it caused, namely 'error: cannot run C compiled programs', so I wrote in my notes: --- start-of-quote To avoid the nasty 'error: cannot run C compiled programs' from the configure scripts when cross-compiling, set CBUILD=i686-pc-linux-gnu before the invocation: CBUILD=i686-pc-linux-gnu x86_64-pc-linux-gnu-emerge -uva --keep-going binutils --- end-of-quote Why it worked with you and not with me, I don't know. I can only say: lucky you! But with the above excerpt from build.log (I only have the two parts of build.log I saved at that time in my notes and posted here) and the patch, together with my explanation, you can follow the logic, you can find the place where this happens ("Makefile:848: recipe for target 'aoutx.stamp' failed"), you can see (in the code) where and when 'chew' is needed - and can think about scenarios where chew is compiled as 64-bit on a 32-bit system, even though 'doc' is not set. What you ultimately decide to do out of this information (which to me looks more than concrete), is up to you. Thank you for looking into this.
