xen-tools fails after it can't find gnu/stubs-32.h when being built in a amd64 only profile. (In my case: hardened/linux/amd64/no-multilib/selinux) Not sure why the environment is being set with target arch as x86_32. File is normally provided by glibc. XEN_TARGET_ARCH=x86_32 make -f blowfish.mk all make[5]: Entering directory '/var/tmp/portage/app-emulation/xen-tools-4.4.0-r7/work/xen-4.4.0/tools/tests/x86_emulator' x86_64-pc-linux-gnu-gcc -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable -Wno-unused-local-typedefs -D__XEN_TOOLS__ -MMD -MF .blowfish.bin.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -mno-tls-direct-seg-refs -nopie -fno-stack-protector -fno-exceptions -fno-builtin -msoft-float -c blowfish.c In file included from /usr/include/features.h:403:0, from /usr/include/stdint.h:25, from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/include/stdint.h:9, from blowfish.c:19: /usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory # include <gnu/stubs-32.h> Reproducible: Always
we have following logic in the ebuild file: pkg_setup() { if [[ -z ${XEN_TARGET_ARCH} ]] ; then if use x86 && use amd64; then die "Confusion! Both x86 and amd64 are set in your use flags!" elif use x86; then export XEN_TARGET_ARCH="x86_32" elif use amd64 ; then export XEN_TARGET_ARCH="x86_64" elif use arm; then export XEN_TARGET_ARCH="arm32" else die "Unsupported architecture!" fi fi } src_prepare() { if ! use hvm; then sed -e '/SUBDIRS-$(CONFIG_X86) += firmware/d' -i tools/Makefile || die # Bug 351648 elif ! use x86 && ! has x86 $(get_all_abis); then mkdir -p "${WORKDIR}"/extra-headers/gnu || die touch "${WORKDIR}"/extra-headers/gnu/stubs-32.h || die export CPATH="${WORKDIR}"/extra-headers fi } could you paste your "emerge --info app-emulation/xen-tools", and attach the file of environment & build.log, you can find them under /var/tmp/portage/app-emulation/xen-tools-4.4.0-r7/temp
Portage 2.2.10 (hardened/linux/amd64/no-multilib/selinux, gcc-4.8.3, glibc-2.19-r1, 3.15.6-hardened x86_64) ================================================================= System Settings ================================================================= System uname: Linux-3.15.6-hardened-x86_64-AMD_Opteron_23xx_-Gen_3_Class_Opteron-with-gentoo-2.2 KiB Mem: 2051464 total, 1843304 free KiB Swap: 0 total, 0 free Timestamp of tree: Fri, 25 Jul 2014 21:30:01 +0000 ld GNU ld (Gentoo 2.24 p1.4) 2.24 app-shells/bash: 4.2_p47 dev-lang/python: 2.7.7, 3.3.5, 3.4.1 dev-util/cmake: 2.8.12.2-r1 dev-util/pkgconfig: 0.28-r1 sys-apps/baselayout: 2.2 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.69 sys-devel/automake: 1.14.1 sys-devel/binutils: 2.24-r3 sys-devel/gcc: 4.8.3 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2-r1 sys-devel/make: 4.0-r1 sys-kernel/linux-headers: 3.15 (virtual/os-headers) sys-libs/glibc: 2.19-r1 Repositories: gentoo ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -mtune=generic -march=x86-64 -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -mtune=generic -march=x86-64 -mfpmath=sse -ftree-vectorize -fomit-frame-pointer -pipe -fvisibility-inlines-hidden" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news nodoc parallel-fetch preserve-libs protect-owned sandbox selinux sesandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LDFLAGS="-Wl,-O1 -Wl,--sort-common -Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" USE="acl acpi amd64 audit berkdb bzip2 caps cgroup cli cracklib crypt cxx dbus device-mapper dri gdbm hardened iconv introspection ipv6 justify kvm lzma lzo mmap mmx modules ncurses networkmanager nls nptl open_perms openmp pam pax_kernel pcre policykit posix readline selinux session sse sse2 ssl systemd tcpd threads udev unconfined unicode urandom xattr xml xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="xen efi-64 emu" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset empty_gif geo gzip limit_conn limit_req lua map memcached proxy referer rewrite split_clients ssi upstream_ip_hash userid flv geoip gzip_static headers_more metrics mp4 push_stream spdy upload_progress stub_status xslt" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4" RUBY_TARGETS="ruby19 ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="geoip" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON
Created attachment 381684 [details] Build log
Created attachment 381686 [details] Environment
After a little investigation, I think xen-tools won't support no-multilib profile. xen-tools hardcoded to build x86_32 image even with x86_64 machine and using "x86_64-pc-linux-gnu-gcc -m32 -march=i686 " will doom to fail in no-multilib profile, fail to link to final executable image. == config/x86_64.mk == CONFIG_X86 := y == tools/tests/Makefile == SUBDIRS-$(CONFIG_X86) += x86_emulator == tools/tests/x86_emulator/Makefile == blowfish.h: rm -f blowfish.bin XEN_TARGET_ARCH=x86_32 $(MAKE) -f blowfish.mk all (echo "static unsigned int blowfish32_code[] = {"; \ od -v -t x blowfish.bin | sed 's/^[0-9]* /0x/' | sed 's/ /, 0x/g' | sed 's/$$/,/';\ echo "};") >$@ rm -f blowfish.bin ~ # cat hello.c #include <stdio.h> int main(int argc, char *argv[]) { printf("hello, no-multilib\n"); return 0; } ~ # gcc -o hello hello.c ~ # file hello hello: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, not stripped ~ # ./hello hello, no-multilib ~ # mkdir -p gnu/ && touch gnu/stubs.h ~ # gcc -I. -m32 -march=i686 -o hello hello.c /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libgcc.a when searching for -lgcc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lgcc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libgcc_s.so when searching for -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../lib64/libc.so when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../lib64/libc.a when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/../lib64/libc.so when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/../lib64/libc.a when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../libc.so when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../libc.a when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/libc.a when searching for -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libgcc.a when searching for -lgcc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lgcc /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/libgcc_s.so when searching for -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lgcc_s collect2: error: ld returned 1 exit status
I'm curious to know if this is simply a matter of a hardwired build system, or if there are limitations in the code preventing a purely 64bit build from being functional. I'm somewhat surprised that this problem hasn't been encountered earlier.
It's a build issue now, and can't tell if it actually a code problem as I haven't dug that far. I'd suggest writing to xen-devel@lists.xen.org for clarification.
+ 29 Jul 2014; Yixun Lan <dlan@gentoo.org> -xen-tools-4.4.0-r7.ebuild, + +xen-tools-4.4.0-r8.ebuild: + bug 518136, disable build x86_emulator under nomultilib profile Please give version 4.4.0-r8 a try, also it won't built with USE="hvm" http://www.gossamer-threads.com/lists/xen/devel/341562
(In reply to Reuben Martin from comment #6) > I'm curious to know if this is simply a matter of a hardwired build system, > or if there are limitations in the code preventing a purely 64bit build from > being functional. > > I'm somewhat surprised that this problem hasn't been encountered earlier. What Yixun didn't pass on was the 10 minute lecture I gave him on how I attempted to get this fixed around 2 years ago. Somewhere in bugzilla there is a now closed bug that also centres around the absent gnu/stubs-32.h. I thought it was still open but seems it's not. I put this to the apt. teams at the time and they basically never rose to the the challenge, leaving xen unviable in a non multilib profile. Someone started a re-write of some key .c files in how they access the .h files but it was never completed.
(In reply to Ian Delaney from comment #9) > What Yixun didn't pass on was the 10 minute lecture I gave him on how I > attempted to get this fixed around 2 years ago. heh. That's kinda funny, in a rather depressing sort of way. For my purposes, this is simply a dependency for grub:2[xen] in an image that is purely a paravirt guest. So the 16bit BIOS voodoo from hvm will not be missed. Thanks to both of you for looking into this.
This bug is applicable to previous versions, such as the current stable 4.3.2-r5 ebuild: make[5]: Entering directory `/var/tmp/portage/app-emulation/xen-tools-4.3.2-r5/work/xen-4.3.2/tools/tests/x86_emulator' x86_64-pc-linux-gnu-gcc -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing -std=gnu99 -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable -D__XEN_TOOLS__ -MMD -MF .blowfish.bin.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -mno-tls-direct-seg-refs -nopie -fno-stack-protector -fno-exceptions -fno-builtin -msoft-float -c blowfish.c In file included from /usr/include/features.h:400:0, from /usr/include/stdint.h:25, from /usr/lib/gcc/x86_64-pc-linux-gnu/4.6.3/include/stdint.h:3, from blowfish.c:19: /usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory compilation terminated.
This bug is specific to the non multilib profile, ascertained at least 2 years ago. /usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory is what happens. To my knowledge upstream don't account for non multilib profiles and the fix requires a reqrite of the build.
(In reply to Ian Delaney from comment #9) > Somewhere in bugzilla there > is a now closed bug that also centres around the absent gnu/stubs-32.h. Yes, we are talking about Bug 351648 . > I > thought it was still open but seems it's not. Yes, because it's fixed. > I put this to the apt. teams > at the time and they basically never rose to the the challenge, leaving xen > unviable in a non multilib profile. I could compile xen-tools without problems on my box (hardened/linux/amd64/no-multilib) with the following use flags: hvm pam qemu screen python2_7, rest are off. hvmloader builds, which is the only thing one need to run a hvm domain. I tried with 4.4.0-r9 now successfully. > Someone started a re-write of some key > .c files in how they access the .h files but it was never completed. No, just add an extra empty file to the build system, and add the extra dummy skeleton path to the build system. The fix for the ebuild was these lines: if ! use hvm; then sed -e '/SUBDIRS-$(CONFIG_X86) += firmware/d' -i tools/Makefile || die # Bug 351648 elif ! use x86 && ! has x86 $(get_all_abis); then mkdir -p "${WORKDIR}"/extra-headers/gnu || die touch "${WORKDIR}"/extra-headers/gnu/stubs-32.h || die export CPATH="${WORKDIR}"/extra-headers fi Maybe at the current case we just need to adjust the conditionals for the build system to work. The problem roots in the source files, which includes stdint.h, which defines standard integer sizes, but the installed header files refer to the glibc capabilities, which it doesn't need at all, because the compiled 32bit files will *NOT* use any libc at all, because they are paravirtual kernel parts, not local system executables. So what's in stubs-32.h is totally indifferent. @Yikus Lan: Also, there is a misconception about no-multilib gcc cannot create 32bit objects. It *CAN*. It just cannot *LINK* to local 32bit system parts, as they don't exist, but *IT DO NOT NEED TO* because it effectively compiles a kernel, not a local executable. For example, gcc -m32 -c -o hello.o hello.c *WILL* compile on a 64bit, no-multilib system. For example, the build system does this: x86_64-pc-linux-gnu-gcc -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable -Wno-unused-local-typedefs -D__XEN_TOOLS__ -MMD -MF .hvmloader.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -mno-tls-direct-seg-refs -Werror -nopie -fno-stack-protector -fno-exceptions -fno-builtin -msoft-float -I/var/tmp/portage/app-emulation/xen-tools-4.4.0-r9/work/xen-4.4.0/tools/firmware/hvmloader/../../../tools/include -DENABLE_ROMBIOS -DENABLE_SEABIOS -c -o hvmloader.o hvmloader.c [...] Later it will do this: x86_64-pc-linux-gnu-ld -melf_i386 -N -Ttext 0x100000 -o hvmloader.tmp hvmloader.o mp_tables.o util.o smbios.o smp.o cacheattr.o xenbus.o e820.o pci.o pir.o ctype.o tests.o optionroms.o 32bitbios_support.o rombios.o seabios.o acpi/acpi.a objcopy hvmloader.tmp hvmloader Nothing prevents in the no-multilib system to do these, except the missing stubs-32.h which the build system actually doesn't need.
(In reply to mike@marineau.org from comment #11) > This bug is applicable to previous versions, such as the current stable > 4.3.2-r5 ebuild: > this should already fixed in bug 351648, please attach the build.log
No, just add an extra empty file to the build system, and add the extra dummy skeleton path to the build system. This is more a work around than an adequate fix, which is why I rejected it outright back then, however whatever works. It still appears a corner case but one which would be nice to have rectified. Th dev who started making a re-write but never completed it knew of this option and in similar style refused to endorse it. Not sure.