Gentoo profile is 17.1 I was used to set abi_x86_32 USE flag to compile boehm-gc, zlib, wine-staging. It has worked pretty well until now. After upgrading to gcc 8.3.0 the .a libraries are no more compatible w/ lto1 7.1 bytecode and need to be recompiled. The nightmare begins. Portage sets automatically CBUILD to i386-pc-linux-gnu-8.3.0 that I don't have installed on my system. gcc-config search for it and fails as well for the multilib ebuilds that complains it could not create a valid ELF image (CONFIG_IA32_EMULATION is enabled in the kernel). To check CONFIG_IA32_EMULATION presence, I suggest you to zgrep /proc/config.gz before /usr/src/linux/.config. If the second doesn't exist, emerge could lead to a false positive answer. I'm not trying to cross-compile for a cross-compiler but instead to use amd64 ability to emulate i386 with gcc -m32 to compile a native build. To do so, and as a workaround, I manually unset CBUILD in src_compile() multilib ebuilds that need be compiled with the abi_x86_32 flag activated for the amd64 platform. Setting CBUILD to x86_64-pc-linux-gnu-8.3.0 works as well but needs to be changed at every gcc release and that's a pain in the neck. It's probably not portable and I need your advise for a better solution. CBUILD can only be set with an indirection. For instance, CBUILD_x86 is set to x86_64-pc-linux-gnu-8.3.0 in package.env via a wrapper. cat /etc/portage/env/abi_x86_32 USE="${USE} abi_x86_32" CBUILD_x86=${CHOST} For zlib it gives something like that, @@ -40,6 +40,12 @@ echoit() { echo "$@"; "$@"; } multilib_src_configure() { + set -x + case "$(uname -m) ${CHOST}" in + #('x86_64 i686-'*) CBUILD=${CBUILD_x86} ;; + ('x86_64 i686-'*) CBUILD= ;; + esac + set +x case ${CHOST} in *-mingw*|mingw*) ;; A example of what I'm talking about is to run glibc ebuild. Its configuration is: ABI: x86 CBUILD: x86_64-pc-linux-gnu CHOST: x86_64-pc-linux-gnu CTARGET: x86_64-pc-linux-gnu CBUILD_OPT: i686-pc-linux-gnu CTARGET_OPT: i686-pc-linux-gnu CC: gcc -m32 So should it be for multilib ebuilds. Reproducible: Always Steps to Reproduce: 1.echo USE="$USE abi_x86_32" > /etc/portage/env.d/abi_x86_32 2.echo 'sys-libs/zlib abi_x86_32' >> /etc/portage/package.env 3.ebuild sys-libs/zlib/zlib-1.2.11-r2.ebuild configure Actual Results: checking build system type... i686-pc-linux-gnu checking host system type... i686-pc-linux-gnu checking how to print strings... printf checking whether make supports the include directive... yes (GNU style) checking for i686-pc-linux-gnu-gcc... gcc -m32 checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... configure: error: in `/var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11-abi_x86_32.x86/contrib/minizip': configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. Expected Results: a working ebuild This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by minizip configure 1.2.11, which was generated by GNU Autoconf 2.69. Invocation command line was $ /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/configure --prefix=/usr --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --disable-dependency-tracking --disable-silent-rules --libdir=/usr/lib --enable-static ## --------- ## ## Platform. ## ## --------- ## hostname = seal uname -m = x86_64 uname -r = 4.20.8-radeon uname -s = Linux uname -v = #3 SMP PREEMPT Fri Feb 15 20:28:29 CET 2019 /usr/bin/uname -p = AMD A10-5800K APU with Radeon(tm) HD Graphics /bin/uname -X = unknown /bin/arch = x86_64 /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown /usr/bin/hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown PATH: /usr/lib64/ccache/bin PATH: /usr/lib/portage/python3.7/ebuild-helpers/xattr PATH: /usr/lib/portage/python3.7/ebuild-helpers PATH: /usr/lib/llvm/7/bin PATH: /usr/local/sbin PATH: /usr/local/bin PATH: /usr/sbin PATH: /usr/bin PATH: /sbin PATH: /bin PATH: /opt/bin ## ----------- ## ## Core tests. ## ## ----------- ## configure:2262: checking for a BSD-compatible install configure:2330: result: /usr/bin/install -c configure:2341: checking whether build environment is sane configure:2396: result: yes configure:2545: checking for a thread-safe mkdir -p configure:2584: result: /bin/mkdir -p configure:2591: checking for gawk configure:2607: found /usr/bin/gawk configure:2618: result: gawk configure:2629: checking whether make sets $(MAKE) configure:2651: result: yes configure:2680: checking whether make supports nested variables configure:2697: result: yes configure:2852: checking build system type configure:2866: result: i686-pc-linux-gnu configure:2886: checking host system type configure:2899: result: i686-pc-linux-gnu configure:2940: checking how to print strings configure:2967: result: printf configure:2992: checking whether make supports the include directive configure:3007: make -f confmf.GNU && cat confinc.out this is the am__doit target configure:3010: $? = 0 configure:3029: result: yes (GNU style) configure:3059: checking for i686-pc-linux-gnu-gcc configure:3086: result: gcc -m32 configure:3355: checking for C compiler version configure:3364: gcc -m32 --version >&5 gcc (Gentoo 8.3.0 p1.0) 8.3.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. configure:3375: $? = 0 configure:3364: gcc -m32 -v >&5 Using built-in specs. COLLECT_GCC=/usr/bin/gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/8.3.0/lto-wrapper Target: x86_64-pc-linux-gnu Configured with: /var/cache/portage/tmp/portage/sys-devel/gcc-8.3.0/work/gcc-8.3.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/8.3.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/8.3.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/8.3.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/8.3.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include/g++-v8 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/8.3.0/python --enable-languages=c,c++,go,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 8.3.0 p1.0' --disable-esp --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-altivec --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libmudflap --disable-libssp --disable-libmpx --disable-systemtap --enable-vtable-verify --enable-lto --with-isl --disable-isl-version-check --enable-default-pie --enable-default-ssp Thread model: posix gcc version 8.3.0 (Gentoo 8.3.0 p1.0) configure:3375: $? = 0 configure:3364: gcc -m32 -V >&5 gcc: error: unrecognized command line option '-V' gcc: fatal error: no input files compilation terminated. configure:3375: $? = 1 configure:3364: gcc -m32 -qversion >&5 gcc: error: unrecognized command line option '-qversion'; did you mean '--version'? gcc: fatal error: no input files compilation terminated. configure:3375: $? = 1 configure:3395: checking whether the C compiler works configure:3417: gcc -m32 -march=bdver2 -pipe -ffast-math -fuse-linker-plugin -frename-registers -fomit-frame-pointer -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-nest-optimize -fgraphite-identity -ftree-loop-distribute-patterns -ftree-loop-ivcanon -fvariable-expansion-in-unroller -Wl,--sort-common -Wl,--hash-style=gnu,--enable-new-dtags -Wl,-z,relro -flto -Ofast -freport-bug -Winline -Wformat -Werror=format-security -w -fno-working-directory -ffast-math conftest.c >&5 configure:3421: $? = 0 configure:3469: result: yes configure:3472: checking for C compiler default output file name configure:3474: result: a.out configure:3480: checking for suffix of executables configure:3487: gcc -m32 -o conftest -march=bdver2 -pipe -ffast-math -fuse-linker-plugin -frename-registers -fomit-frame-pointer -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-nest-optimize -fgraphite-identity -ftree-loop-distribute-patterns -ftree-loop-ivcanon -fvariable-expansion-in-unroller -Wl,--sort-common -Wl,--hash-style=gnu,--enable-new-dtags -Wl,-z,relro -flto -Ofast -freport-bug -Winline -Wformat -Werror=format-security -w -fno-working-directory -ffast-math conftest.c >&5 configure:3491: $? = 0 configure:3513: result: configure:3535: checking whether we are cross compiling configure:3543: gcc -m32 -o conftest -march=bdver2 -pipe -ffast-math -fuse-linker-plugin -frename-registers -fomit-frame-pointer -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-nest-optimize -fgraphite-identity -ftree-loop-distribute-patterns -ftree-loop-ivcanon -fvariable-expansion-in-unroller -Wl,--sort-common -Wl,--hash-style=gnu,--enable-new-dtags -Wl,-z,relro -flto -Ofast -freport-bug -Winline -Wformat -Werror=format-security -w -fno-working-directory -ffast-math conftest.c >&5 configure:3547: $? = 0 configure:3554: ./conftest ./conftest: Invalid ELF image for this architecture configure:3558: $? = 255 configure:3565: error: in `/var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11-abi_x86_32.x86/contrib/minizip': configure:3567: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. See `config.log' for more details ## ---------------- ## ## Cache variables. ## ## ---------------- ## ac_cv_build=i686-pc-linux-gnu ac_cv_env_CC_set=set ac_cv_env_CC_value='gcc -m32' ac_cv_env_CFLAGS_set=set ac_cv_env_CFLAGS_value=' -march=bdver2 -pipe -ffast-math -fuse-linker-plugin -frename-registers -fomit-frame-pointer -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-nest-optimize -fgraphite-identity -ftree-loop-distribute-patterns -ftree-loop-ivcanon -fvariable-expansion-in-unroller -Wl,--sort-common -Wl,--hash-style=gnu,--enable-new-dtags -Wl,-z,relro -flto -Ofast -freport-bug -Winline -Wformat -Werror=format-security ' ac_cv_env_CPPFLAGS_set=set ac_cv_env_CPPFLAGS_value='-w -fno-working-directory -ffast-math' ac_cv_env_CPP_set=set ac_cv_env_CPP_value=cpp ac_cv_env_LDFLAGS_set=set ac_cv_env_LDFLAGS_value=' ' ac_cv_env_LIBS_set= ac_cv_env_LIBS_value= ac_cv_env_LT_SYS_LIBRARY_PATH_set= ac_cv_env_LT_SYS_LIBRARY_PATH_value= ac_cv_env_build_alias_set=set ac_cv_env_build_alias_value=i686-pc-linux-gnu ac_cv_env_host_alias_set=set ac_cv_env_host_alias_value=i686-pc-linux-gnu ac_cv_env_target_alias_set= ac_cv_env_target_alias_value= ac_cv_host=i686-pc-linux-gnu ac_cv_path_install='/usr/bin/install -c' ac_cv_path_mkdir=/bin/mkdir ac_cv_prog_AWK=gawk ac_cv_prog_CC='gcc -m32' ac_cv_prog_make_make_set=yes am_cv_make_support_nested_variables=yes ## ----------------- ## ## Output variables. ## ## ----------------- ## ACLOCAL='${SHELL} /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/missing aclocal-1.16' AMDEPBACKSLASH='' AMDEP_FALSE='' AMDEP_TRUE='#' AMTAR='$${TAR-tar}' AM_BACKSLASH='\' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' AM_DEFAULT_VERBOSITY='1' AM_V='$(V)' AR='gcc-ar' AUTOCONF='${SHELL} /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/missing autoconf' AUTOHEADER='${SHELL} /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/missing autoheader' AUTOMAKE='${SHELL} /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/missing automake-1.16' AWK='gawk' CC='gcc -m32' CCDEPMODE='' CFLAGS=' -march=bdver2 -pipe -ffast-math -fuse-linker-plugin -frename-registers -fomit-frame-pointer -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-nest-optimize -fgraphite-identity -ftree-loop-distribute-patterns -ftree-loop-ivcanon -fvariable-expansion-in-unroller -Wl,--sort-common -Wl,--hash-style=gnu,--enable-new-dtags -Wl,-z,relro -flto -Ofast -freport-bug -Winline -Wformat -Werror=format-security ' COND_DEMOS_FALSE='' COND_DEMOS_TRUE='' CPP='cpp' CPPFLAGS='-w -fno-working-directory -ffast-math' CYGPATH_W='echo' DEFS='' DEPDIR='.deps' DLLTOOL='' DSYMUTIL='' DUMPBIN='' ECHO_C='' ECHO_N='-n' ECHO_T='' EGREP='' EXEEXT='' FGREP='' GREP='' HAVE_UNISTD_H='' INSTALL_DATA='${INSTALL} -m 644' INSTALL_PROGRAM='${INSTALL}' INSTALL_SCRIPT='${INSTALL}' INSTALL_STRIP_PROGRAM='$(install_sh) -c -s' LD='/etc/portage/bin/gcc-ld.sh -m elf_i386' LDFLAGS=' ' LIBOBJS='' LIBS='' LIBTOOL='' LIPO='' LN_S='' LTLIBOBJS='' LT_SYS_LIBRARY_PATH='' MAKEINFO='${SHELL} /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/missing makeinfo' MANIFEST_TOOL='' MKDIR_P='/bin/mkdir -p' NM='gcc-nm' NMEDIT='' OBJDUMP='' OBJEXT='' OTOOL64='' OTOOL='' PACKAGE='minizip' PACKAGE_BUGREPORT='bugzilla.redhat.com' PACKAGE_NAME='minizip' PACKAGE_STRING='minizip 1.2.11' PACKAGE_TARNAME='minizip' PACKAGE_URL='' PACKAGE_VERSION='1.2.11' PATH_SEPARATOR=':' RANLIB='gcc-ranlib' SED='' SET_MAKE='' SHELL='/bin/bash' STRIP='' VERSION='1.2.11' WIN32_FALSE='' WIN32_TRUE='' ac_ct_AR='' ac_ct_CC='' ac_ct_DUMPBIN='' am__EXEEXT_FALSE='' am__EXEEXT_TRUE='' am__fastdepCC_FALSE='' am__fastdepCC_TRUE='' am__include='include' am__isrc=' -I$(srcdir)' am__leading_dot='.' am__nodep='' am__quote='' am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -' bindir='${exec_prefix}/bin' build='i686-pc-linux-gnu' build_alias='i686-pc-linux-gnu' build_cpu='i686' build_os='linux-gnu' build_vendor='pc' datadir='/usr/share' datarootdir='${prefix}/share' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' dvidir='${docdir}' exec_prefix='NONE' host='i686-pc-linux-gnu' host_alias='i686-pc-linux-gnu' host_cpu='i686' host_os='linux-gnu' host_vendor='pc' htmldir='${docdir}' includedir='${prefix}/include' infodir='/usr/share/info' install_sh='${SHELL} /var/cache/portage/tmp/portage/sys-libs/zlib-1.2.11-r2/work/zlib-1.2.11/contrib/minizip/install-sh' libdir='/usr/lib' libexecdir='${exec_prefix}/libexec' localedir='${datarootdir}/locale' localstatedir='/var/lib' mandir='/usr/share/man' mkdir_p='$(MKDIR_P)' oldincludedir='/usr/include' pdfdir='${docdir}' prefix='/usr' program_transform_name='s,x,x,' psdir='${docdir}' sbindir='${exec_prefix}/sbin' sharedstatedir='${prefix}/com' sysconfdir='/etc' target_alias='' ## ----------- ## ## confdefs.h. ## ## ----------- ## /* confdefs.h */ #define PACKAGE_NAME "minizip" #define PACKAGE_TARNAME "minizip" #define PACKAGE_VERSION "1.2.11" #define PACKAGE_STRING "minizip 1.2.11" #define PACKAGE_BUGREPORT "bugzilla.redhat.com" #define PACKAGE_URL "" #define PACKAGE "minizip" #define VERSION "1.2.11" configure: exit 1
Created attachment 567266 [details] emerge --info sys-libs/zlib
> Portage sets automatically CBUILD to i386-pc-linux-gnu-8.3.0 that I don't have installed on my system. I don't see any evidence of portage doing this. How did you come to this conclusion? I am able to do a multilib build of sys-libs/zlib without any problems. I assume you have misconfigured your system is some way.
(In reply to Mike Gilbert from comment #2) > > Portage sets automatically CBUILD to i386-pc-linux-gnu-8.3.0 that I don't have installed on my system. > > I don't see any evidence of portage doing this. How did you come to this > conclusion? Because I didn't setup CBUILD myself. Anyway not with i386-pc-linux- prefix. Moreover, I'm able to compile it manually (without portage). > I am able to do a multilib build of sys-libs/zlib without any problems. I > assume you have misconfigured your system is some way. Please... I wouldn't open a report in that case. I have double checked. Have you the same configuration ? profile 17.1 same cflags as mine no i386 gcc compiled on the system ld.gold lto gcc 8.3.0 portage release distcc ccache /etc/env.d/gcc/ doesn't contain neither config-i386-pc-* nor i386-pc-* files - Did you compared the config.log with yours ? Have you found any differences ? - Have you set CBUILD or unset CBUILD to whatever valid value and see if it affects portage compilation ? Does CBUILD stay the same ? - what does gcc-config return ? Also, it affects my sandboxed environment as well. That lower the possibility of a misconfiguration.
(In reply to Jimmy.Jazz from comment #3) The string "i386-pc-linux-gnu" does not appear in any of the log output you have provided. Where are you seeing that?
(In reply to Mike Gilbert from comment #4) > (In reply to Jimmy.Jazz from comment #3) > > The string "i386-pc-linux-gnu" does not appear in any of the log output you > have provided. Where are you seeing that? Please ignore the typo. Read i686-pc-linux-gnu
I seriously doubt your build failure has anything do do with CBUILD. It is set to i686-pc-linux-gnu intentionally for a 32-bit build. Please prune your CFLAGS/CXXFLAGS to something simple (like "-O2") and see if it works. If so, I will re-assign this bug to the zlib maintainer.
(In reply to Mike Gilbert from comment #6) > I seriously doubt your build failure has anything do do with CBUILD. It is > set to i686-pc-linux-gnu intentionally for a 32-bit build. Actually it was the trigger > Please prune your CFLAGS/CXXFLAGS to something simple (like "-O2") and see > if it works. If so, I will re-assign this bug to the zlib maintainer. or better to the kernel maintainer. Finally I find out the cause of the problem. It is linked to the binfmt_misc kernel module probably since 4.20.8 The problem appears after I followed gentoo howto: https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Compiling_with_qemu_user_chroot The issue is not related to the compiler flags except when it comes to qemu compilation. Over -O2 'qemu' compilation stays in an infinite loop if you try to build more targets as the x86 family. When I register a handler for all the different machines qemu emulates, like for instance: echo ':sh4:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2a\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfb\xff\xff\xff:/usr/bin/qemu-sh4:' >/proc/sys/fs/binfmt_misc/register the kernel becomes unable to distinguish anything else but sh4 binary. /proc/sys/fs/binfmt_misc/arm or /proc/sys/fs/binfmt_misc/sh4 have the same magic and mask for instance. My system was in the paste able to compile glibc. So I tried it again. Emerge begins actually to complain that CONFIG_IA32_EMULATION=y was not set. That is wrong. Portage test is a little a.out program. I traced it. It calls /usr/bin/qemu-sh4 and /usr/bin/gnemul/qemu-sh4 as you can see below ! That's wrong too. execve("./a.out", ["./a.out"], 0x7ffd609e55c0 /* 45 vars */) = 0 brk(NULL) = 0x63132000 brk(0x63133280) = 0x63133280 arch_prctl(ARCH_SET_FS, 0x63132940) = 0 uname({sysname="Linux", nodename="seal", ...}) = 0 set_tid_address(0x63132c10) = 28282 set_robust_list(0x63132c20, 24) = 0 rt_sigaction(SIGRTMIN, {sa_handler=0x6015a590, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x60162e90}, NULL, 8) = 0 rt_sigaction(SIGRT_1, {sa_handler=0x6015a640, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x60162e90}, NULL, 8) = 0 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=32768*1024}) = 0 readlink("/proc/self/exe", "/usr/bin/qemu-sh4", 4096) = 17 brk(0x63154280) = 0x63154280 brk(0x63155000) = 0x63155000 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 mmap(NULL, 8392704, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f647de85000 mprotect(0x7f647de86000, 8388608, PROT_READ|PROT_WRITE) = 0 clone(child_stack=0x7f647e684db0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f647e6859d0, tls=0x7f647e685700, child_tidptr=0x7f647e6859d0) = 28283 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 clock_gettime(CLOCK_MONOTONIC, {tv_sec=3023, tv_nsec=199114812}) = 0 gettimeofday({tv_sec=1551640813, tv_usec=504510}, NULL) = 0 prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=32768*1024}) = 0 time(NULL) = 1551640813 (2019-03-03T20:20:13+0100) openat(AT_FDCWD, "/usr/gnemul/qemu-sh4", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 ENOENT (Aucun fichier ou dossier de ce type) uname({sysname="Linux", nodename="seal", ...}) = 0 openat(AT_FDCWD, "./a.out", O_RDONLY) = 3 lseek(3, 0, SEEK_SET) = 0 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\3\0\0004\0\0\0"..., 52) = 52 lseek(3, 0, SEEK_SET) = 0 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f647de04000 mprotect(0x60349000, 33550336, PROT_READ|PROT_WRITE|PROT_EXEC) = 0 madvise(0x60349000, 33550336, MADV_HUGEPAGE) = 0 mmap(NULL, 2147483648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7f63f8000000 openat(AT_FDCWD, "/proc/sys/vm/mmap_min_addr", O_RDONLY) = 4 fstat(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 read(4, "65535\n", 1024) = 6 close(4) = 0 gettid() = 28282 fstat(3, {st_mode=S_IFREG|0755, st_size=7100, ...}) = 0 geteuid() = 0 getegid() = 0 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\3\0\0004\0\0\0"..., 1024) = 1024 write(2, "./a.out: Invalid ELF image for t"..., 49./a.out: Invalid ELF image for this architecture ) = 49 exit_group(-1) = ? +++ exited with 255 +++ Moreover, ldd /bin/bash for instance returns at it first line /lib/ld-linux.so.2 Invalid ELF image Disabling /proc/sys/fs/binfmt_misc/sh4 didn't work. The kernel has eyes for sh4 magic only. Also, I condition out the lines that fed /proc/sys/fs/binfmt_misc/register and retarted the computer. zlib and glibc was able to be compiled in multilib mode again as well as 'ldd /bin/bash' doesn't call '/lib/ld-linux.so.2' anymore. I don't have any idea how to correctly register an handler that won't conflict with an i686 or arm binary. The documentation doesn't provide any other method. It is necessary when you call qemu in user mode. - Please could you try to register qemu targets with kernel 4.20.13 and compile zlib ? - Does it compile well ? I have emerged 'qemu' as follow, [ebuild R ~] app-emulation/qemu [3.1.0-r2::gentoo] USE="aio alsa bzip2 caps curl fdt filecaps gtk jpeg ncurses nls opengl pin-upstream-blobs png seccomp ssh static-user tci usb vde vhost-net virtfs vnc xattr -accessibility -capstone -debug -glusterfs -gnutls -infiniband -iscsi -lzo -nfs -numa -pulseaudio -python -rbd -sasl -sdl (-selinux) -smartcard -snappy -spice (-static) -systemtap -test -usbredir -virgl -vte -xen -xfs" PYTHON_TARGETS="python2_7 python3_7 -python3_4 -python3_5 -python3_6" QEMU_SOFTMMU_TARGETS="aarch64 alpha arm cris hppa i386 lm32 m68k microblaze microblazeel mips mips64 mips64el mipsel moxie nios2 or1k ppc ppc64 riscv32 riscv64 s390x sh4 sh4eb sparc sparc64 tricore unicore32 x86_64 xtensa xtensaeb" QEMU_USER_TARGETS="aarch64 aarch64_be alpha arm armeb cris hppa i386 m68k microblaze microblazeel mips mips64 mips64el mipsel mipsn32 mipsn32el nios2 or1k ppc ppc64 ppc64abi32 ppc64le riscv32 riscv64 s390x sh4 sh4eb sparc sparc32plus sparc64 tilegx x86_64 xtensa xtensaeb" 0 KiB
(In reply to Jimmy.Jazz from comment #7) > When I register a handler for all the different machines qemu emulates, like > for instance: > > echo > ':sh4:M:: > \x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x2a\x00: > \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfb\xff\xff\ > xff:/usr/bin/qemu-sh4:' >/proc/sys/fs/binfmt_misc/register > > the kernel becomes unable to distinguish anything else but sh4 binary. > > /proc/sys/fs/binfmt_misc/arm or /proc/sys/fs/binfmt_misc/sh4 have the same > magic and mask for instance. Please post the output of $ cat /proc/sys/fs/binfmt_misc/sh4 > Portage test is a little a.out program. Please attach the binary that was compiled to double-check the elf header. Your shell points to: sh mksh 56c Is it POSIX-compatible? I wonder if unusual shell is causing binfmt loader to register broken entries.
bash/mksh certainly don't behave the same WRT '\xNN' handling: sf / # bash -c "echo '\xFF'" | hexdump -C 00000000 5c 78 46 46 0a |\xFF.| 00000005 sf / # mksh -c "echo '\xFF'" | hexdump -C 00000000 ff 0a |..| 00000002
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html says handling of backslashed characters is implementation-defined: "A string to be written to standard output. If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined." The wiki page needs more portable way to pass "\xFF" to kernel as a string.
/etc/init.d/qemu-binfmt will need a fix on qemu side. It is generated here: https://gitweb.gentoo.org/repo/gentoo.git/tree/app-emulation/qemu/qemu-3.1.0-r2.ebuild#n652
Created attachment 567736 [details] glibc ebuild generated a.out to test CONFIG_IA32_EMULATION Its a normal elf file. What are your /proc/sys/fs/binfmt_misc/* output ? In my case the magic and mask respectively for sh4 and arm are the same and seem to be truncated. The mask was all of 0xf if I remember well. I can renew the test if you like as soon as the bunch of compilations I have to finish are over. # file a.out a.out: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 3.2.0, not stripped # ldd a.out linux-gate.so.1 (0xf7f00000) libc.so.6 => /lib/libc.so.6 (0xf7c90000) /lib/ld-linux.so.2 (0xf7ed0000) # objdump -t a.out a.out: format de fichier elf32-i386 For /bin/bash it was exactly the following output at that time, # ldd /bin/bash /lib/ld-linux.so.2: Invalid ELF image for this architecture linux-vdso.so.1 (0x00007fff55beb000) libreadline.so.8 => /lib64/libreadline.so.8 (0x00007f4746750000) libhistory.so.8 => /lib64/libhistory.so.8 (0x00007f4746740000) libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007f4746700000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f47466f8000) libc.so.6 => /lib64/libc.so.6 (0x00007f4746528000) libncurses.so.6 => /lib64/libncurses.so.6 (0x00007f47464f8000) /lib64/ld-linux-x86-64.so.2 (0x00007f4746910000)
(In reply to Sergei Trofimovich from comment #8) > Is it POSIX-compatible? I wonder if unusual shell is causing binfmt loader > to register broken entries. I thought it could be binfmt module related. So I installed kernel 5.0.0. The issue still remains. I use now /bin/echo binary instead of shell built-in. There is some improvement, magic and mask are no more truncated. Still 'ldd' searches for qemu binary. /lib/ld-linux.so.2: Invalid ELF image for this architecture [..] After unmounting binfmt_misc mount point it is not possible to remove its module via rmmod It becomes impossible to compile for instance zlib in ab_x86_32 mode. qemu binding is still active. If the /proc/sys/fs/binfmt_misc/XXX file exists you cannot modify or remove it and /bin/echo will complains invariably. # cat binfmt_misc.output /bin/echo: erreur d'écriture: Le fichier existe 000000 3a 61 61 72 63 68 36 34 3a 4d 3a 3a 7f 45 4c 46 >:aarch64:M::.ELF< 000010 02 01 01 00 00 00 00 00 00 00 00 00 02 00 b7 3a >...............:< 000020 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >................< 000030 fe ff ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d >...:/usr/bin/qem< 000040 75 2d 61 61 72 63 68 36 34 3a 0a >u-aarch64:.< 00004b /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-aarch64 flags: offset 0 magic 7f454c460201010000000000000000000200b7 mask fffffffffffffffffffffffffffffffffeffff 000000 3a 61 72 6d 3a 4d 3a 3a 7f 45 4c 46 01 01 01 00 >:arm:M::.ELF....< 000010 00 00 00 00 00 00 00 00 02 00 28 00 3a ff ff ff >..........(.:...< 000020 ff ff ff ff 00 ff ff ff ff ff ff ff ff fe ff ff >................< 000030 ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d 75 2d >.:/usr/bin/qemu-< 000040 61 72 6d 3a 0a >arm:.< 000045 /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-arm flags: offset 0 magic 7f454c46010101 mask ffffffffffffff 000000 3a 6d 69 70 73 3a 4d 3a 3a 7f 45 4c 46 01 02 01 >:mips:M::.ELF...< 000010 00 00 00 00 00 00 00 00 00 00 02 00 08 3a ff ff >.............:..< 000020 ff ff ff ff ff 00 ff ff ff ff ff ff ff ff ff fe >................< 000030 ff ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d 75 >..:/usr/bin/qemu< 000040 2d 6d 69 70 73 3a 0a >-mips:.< 000047 /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-mips flags: offset 0 magic 7f454c46010201 mask ffffffffffffff 000000 3a 6d 69 70 73 65 6c 3a 4d 3a 3a 7f 45 4c 46 01 >:mipsel:M::.ELF.< 000010 01 01 00 00 00 00 00 00 00 00 00 02 00 08 00 3a >...............:< 000020 ff ff ff ff ff ff ff 00 ff ff ff ff ff ff ff ff >................< 000030 fe ff ff ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 >....:/usr/bin/qe< 000040 6d 75 2d 6d 69 70 73 65 6c 3a 0a >mu-mipsel:.< 00004b /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-mipsel flags: offset 0 magic 7f454c46010101 mask ffffffffffffff 000000 3a 70 70 63 3a 4d 3a 3a 7f 45 4c 46 01 02 01 00 >:ppc:M::.ELF....< 000010 00 00 00 00 00 00 00 00 00 02 00 14 3a ff ff ff >............:...< 000020 ff ff ff ff ff ff ff ff ff ff ff ff ff ff fe ff >................< 000030 ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d 75 2d >.:/usr/bin/qemu-< 000040 70 70 63 3a 0a >ppc:.< 000045 /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-ppc flags: offset 0 magic 7f454c4601020100000000000000000000020014 mask fffffffffffffffffffffffffffffffffffeffff 000000 3a 73 68 34 3a 4d 3a 3a 7f 45 4c 46 01 01 01 00 >:sh4:M::.ELF....< 000010 00 00 00 00 00 00 00 00 02 00 2a 00 3a ff ff ff >..........*.:...< 000020 ff ff ff ff 00 ff ff ff ff ff ff ff ff fb ff ff >................< 000030 ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d 75 2d >.:/usr/bin/qemu-< 000040 73 68 34 3a 0a >sh4:.< 000045 /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-sh4 flags: offset 0 magic 7f454c46010101 mask ffffffffffffff 000000 3a 73 68 34 65 62 3a 4d 3a 3a 7f 45 4c 46 01 02 >:sh4eb:M::.ELF..< 000010 01 00 00 00 00 00 00 00 00 00 00 02 00 2a 3a ff >.............*:.< 000020 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >................< 000030 fe ff ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d >...:/usr/bin/qem< 000040 75 2d 73 68 34 65 62 3a 0a >u-sh4eb:.< 000049 /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-sh4eb flags: offset 0 magic 7f454c460102010000000000000000000002002a mask fffffffffffffffffffffffffffffffffffeffff 000000 3a 73 70 61 72 63 3a 4d 3a 3a 7f 45 4c 46 01 02 >:sparc:M::.ELF..< 000010 01 00 00 00 00 00 00 00 00 00 00 02 00 02 3a ff >..............:.< 000020 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >................< 000030 fe ff ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d >...:/usr/bin/qem< 000040 75 2d 73 70 61 72 63 3a 0a >u-sparc:.< 000049 /bin/echo: erreur d'écriture: Le fichier existe enabled interpreter /usr/bin/qemu-sparc flags: offset 0 magic 7f454c4601020100000000000000000000020002 mask fffffffffffffffffffffffffffffffffffeffff
(In reply to Jimmy.Jazz from comment #13) > # cat binfmt_misc.output > /bin/echo: erreur d'écriture: Le fichier existe > 000000 3a 61 61 72 63 68 36 34 3a 4d 3a 3a 7f 45 4c 46 >:aarch64:M::.ELF< > 000010 02 01 01 00 00 00 00 00 00 00 00 00 02 00 b7 3a >...............:< > 000020 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff >................< > 000030 fe ff ff 3a 2f 75 73 72 2f 62 69 6e 2f 71 65 6d >...:/usr/bin/qem< > 000040 75 2d 61 61 72 63 68 36 34 3a 0a >u-aarch64:.< > 00004b I did not quite understand how you got binfmt_misc.output but it does not look correct. There should be no non-ascii data present. Output on bash machine (I have slightly different absolute path): $ cat /proc/sys/fs/binfmt_misc/aarch64 enabled interpreter /qemu-aarch64 flags: offset 0 magic 7f454c460201010000000000000000000200b700 mask ffffffffffffff00fffffffffffffffffeffffff
Can you test this patch? It changes "echo '\xFF...'" to 'printf "%s\n" "\xFF..."'. --- a/app-emulation/qemu/qemu-3.1.0-r2.ebuild +++ b/app-emulation/qemu/qemu-3.1.0-r2.ebuild @@ -651,5 +651,5 @@ generate_initd() { cat <<EOF >>"${out}" if [ "\${cpu}" != "${qcpu}" -a -x "${interpreter}" ] ; then - echo ':${package}:M::${magic}:${mask}:${interpreter}:'"\${QEMU_BINFMT_FLAGS}" >/proc/sys/fs/binfmt_misc/register + printf '%s\n' ':${package}:M::${magic}:${mask}:${interpreter}:'"\${QEMU_BINFMT_FLAGS}" >/proc/sys/fs/binfmt_misc/register fi EOF
Created attachment 568352 [details, diff] 0001-app-emulation-qemu-fix-init.d-qemu-binfmt-for-mksh-b.patch
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cb16168c4ea6976b638148758a18992eafdfd235 commit cb16168c4ea6976b638148758a18992eafdfd235 Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2019-03-09 19:56:20 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2019-03-10 00:19:16 +0000 app-emulation/qemu: fix init.d/qemu-binfmt for mksh, bug #679168 Handling of escaped by echo is an implementation-defined behaviour according to POSIX: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html "A string to be written to standard output. If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined." Practically this makes $ echo '\xFF' behave differently on bash/dash and mksh: bash outputs '\xFF' string of 4 bytes while mksh outputs single 0xFF into stdout. The change is to use "printf '%s\n' '\xFF'" to get consistent behaviour. Reported-by: Jimmy.Jazz@gmx.net Reviewed-by: Matthias Maier Closes: https://bugs.gentoo.org/679168 Package-Manager: Portage-2.3.62, Repoman-2.3.12 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> app-emulation/qemu/qemu-3.1.0-r3.ebuild | 815 ++++++++++++++++++++++++++++++++ 1 file changed, 815 insertions(+)
