Hit a sandbox failure which appeared to kill glibc-2..9_p20081201-r2. Note that this was building -march=opteron on an Intel Xeon 55xx (Nehalem system). I haven't tested this extensively and it could be due to bad asm instructions, as this is the first rebuild of glibc on the Nehalem system. However, the sandbox failure didn't seem to mention anything about an invalid instruction, so for now I'll pass it along so people can take a look at it. I will continue testing with other non-Opteron builds that the Xeon should definitely handle and see if this error pops up again. Reproducible: Didn't try Actual Results: SE:_do_ptrace ptrace(PTRACE_PEEKDATA, ..., 0x0000000000000b10, 0x0000000000000000): Input/output error/usr/lib/libsandbox.so[0x2b356995f7b2] /usr/lib/libsandbox.so[0x2b356995f843] /usr/lib/libsandbox.so[0x2b3569961495] /usr/lib/libsandbox.so[0x2b3569961676] /usr/lib/libsandbox.so[0x2b356996209e] /usr/lib/libsandbox.so[0x2b3569964c9d]/usr/lib/libsandbox.so(execve+0x56)[0x2b3569964fa6] /bin/sh(shell_execve+0x33)[0x42ef63] /bin/sh[0x434338] /bin/sh(execute_command_internal+0xb43)[0x430ec3]/proc/2827/cmdline: /var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/elf/ld-linux-x86-64.so.2 --library-path /var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gn u-nptl:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/math:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/elf:/var/tmp/portage/sys-libs/glibc-2.9_p 20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/dlfcn:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/nss:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/nis:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/rt:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/resolv:/var/tmp/portage/sys-libs/gl ibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/crypt:/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/nptl /var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_ 64-pc-linux-gnu-nptl/sunrpc/rpcgen -Y ../scripts -h rpcsvc/spray.x -o /var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl/sunrpc/rpcsvc/spray.T
Created attachment 192250 [details] excerpt of build log
emerge --info please.
I'm getting various types of build failures on the sunrpc code, on various subarches - this is likely a parallel build failure as I am running MAKEOPTS="-j12" on the new system.
I have not seen the libsandbox errors on other arches, though. Just failures.
I am going to disable sandbox in the metro stage2 builds for the time being and see if this problem goes away. If it does, this should be a pretty good indication that the issue is being caused by sandbox. If not, that at least rules out sandbox causing some of the build failures.
drobbins: can you try hard to rule out hardware issues? I've got a brand new i7 Xeon W5580 box here (Supermicro X8DTU) w/ 24GiB of RAM, and I cannot reproduce it at all. Your emerge --info too if you do manage to reproduce it.
This is easy to reproduce and does appear to be directly related to sandbox. Here's what I did: I disabled sandbox in my stage2 Metro builds - and the problem goes away. But today, I had a build failure in one of my Metro builds -- because glibc was getting rebuilt during stage3 for some reason. So it triggered the issue, and I got some libsandbox debug output. I will be attaching the truncated build log and emerge --info...
Created attachment 193013 [details] this is a stage3 build failure when glibc was getting rebuilt with sandbox on You can see libsandbox error output in the log.
Created attachment 193014 [details] this is the build script that was used...
Created attachment 193016 [details] emerge.info run from the chroot.... This means that some of the data here may not reflect portage env vars set by the build script. So the best way to look at this data is probably to use it as a base, and then refer to run.metro and take into account the env vars set in that script.
This isn't perfect, I think the working dir is wrong, but should be close enough for you to try and track down a single command and set of inputs to reproduce on your system: From the first one: # G="/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-amd64-x86_64-pc-linux-gnu-nptl" # cd ${G}/sunrpc/ # maybe wrong? # ${G}/elf/ld-linux-x86-64.so.2 --library-path ${G}:${G}/math:${G}/elf:${G}/dlfcn:${G}/nss:${G}/nis:${G}/rt:${G}/resolv:${G}/crypt:${G}/nptl ${G}/sunrpc/rpcgen -Y ../scripts -h rpcsvc/spray.x -o ${G}/sunrpc/rpcsvc/spray.T From the second one: # G="/var/tmp/portage/sys-libs/glibc-2.9_p20081201-r2/work/build-x86-x86_64-pc-linux-gnu-nptl" # cd ${G}/sunrpc/ # maybe wrong? # ${G}/elf/ld-linux.so.2 --library-path ${G}:${G}/math:${G}/elf:${G}/dlfcn:${G}/nss:${G}/nis:${G}/rt:${G}/resolv:${G}/crypt:${G}/nptl ${G}/sunrpc/rpcgen -Y ../scripts -h rpcsvc/sm_inter.x -o ${G}/sunrpc/rpcsvc/sm_inter.T I'm wondering about ptrace kernel interaction issue. Is the second output on the same i7 machine? It shows a i686 kernel, while before you were building a 64-bit glibc, so either you're using linux32 and ran emerge --info inside that, or it's a different system.
Same system. Out of habit, I probably typed "linux32 chroot" by mistake...
Here's the emerge --info from my system where glibc compiles fine with sandbox-1.9. On the chance that it's kernel related, can you maybe try a different kernel (without the VPS patchset, not sure if you are in a VPS at the moment or outside it). Portage 2.2_rc33 (!/etc/managed-portage/hosts/lucene0/make.profile, gcc-4.3.3, glibc-2.9_p20081201-r0, 2.6.30-rc5-i7 x86_64) ================================================================= System uname: Linux-2.6.30-rc5-i7-x86_64-Intel-R-_Xeon-R-_CPU_W5580_@_3.20GHz-with-gentoo-2.0.0 Timestamp of tree: Mon, 04 May 2009 04:45:01 +0000 app-shells/bash: 4.0_p17-r1 dev-java/java-config: 1.3.7-r1, 2.1.7 dev-lang/python: 2.5.4-r2, 2.6.2 sys-apps/baselayout: 2.0.0 sys-apps/openrc: 0.4.3-r2 sys-apps/sandbox: 1.9 sys-devel/autoconf: 2.63-r1 sys-devel/automake: 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.19.1-r1 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.28-r1 ACCEPT_KEYWORDS="amd64 ~amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=core2" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -march=core2" DISTDIR="/home/gentoo/distfiles" FEATURES="buildpkg collision-protect distlocks fixpackages parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://216.165.129.134/gentoo http://gentoo.mirrors.tds.net/gentoo http://gentoo.osuosl.org/" LDFLAGS="-Wl,-O1" MAKEOPTS="-j5" PKGDIR="/home/gentoo/packages/lucene-nehalem" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/dev/shm" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://beta01/local-portage" USE="amd64 async authdaemond authlib bash-completion berkdb bzip2 bzlib cap caps cdb chroot ck-server cli cracklib crypt curlwrappers dba dbm dbx dio dri erandom exif extensions f2c fam fastcgi flatfile gd-external gdbm gnutls hpn iconv imap innodb ipv6 isdnlog jpeg lids lm_sensors logrotate memlimit midi mmx mpi mudflap multilib ncurses nethack nfs nls no-htdocs nptl nptlonly offensive openmp pam pcre perl png posix pppd python qmail readline reflection reiserfs romio rrdtool session sftp sftplogging sharedext snmp soap sockets spl sqlite sse sse2 ssl symlink sysfs sysvipc tcpd threads truetype unicode urandom vim-syntax vpopmail xattr xml xmlrpc xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
This build is happening in a VPS. Here's a question for you -- is sandbox 1.9 using ptrace? If it is, then I could see it possibly being a kernel issue. If sandbox is using library wrappers, shouldn't it be independent of the kernel and thus (under most cases) isolated from any potential kernel issues? Of course, it is very possible it is due to the kernel, I'd just like to understand why. Also, I was able to test this build with sydbox 1.0 in place of sandbox and the the compile completed successfully inside the VPS -- sydbox uses ptrace exclusively. If we can get some that confirms that it is likely a kernel bug, I can bring it up with the OpenVZ development team. Is there any test I can do to narrow it down a bit more? Regards, Daniel
Yes, sandbox-1.9 does use ptrace. It was introduced in sandbox-1.7. I had suggested trying a different kernel because I know that there have been fixes to the ptrace support in the kernel since the 2.6.27 that you are running, and I've suspected VPS of some ptrace breakage in the past as well. I don't know the internals of sydbox, vapier might be able to compare in usage and see if there's anything that would trigger only one side and not the other.
i dont know anything about sydbox sandbox uses ptrace on static binaries only, and tries to detect the mode in use so that it works on multilib systems dynamically (32bit/64bit)
sydbox uses ptrace for everything, so it's currently quite slow on kernels less than 2.6.29. On 2.6.29+, it is supposed to be about as fast as sandbox. http://github.com/alip/sydbox/tree/master
Ok, the base ${G}/elf/ld-linux.so.2 itself is statically linked anyway, so we would be using ptrace on it in sandbox. I'm still suspecting kernel given the lack of repeatability on my very similar box. > Is there any test I can do to narrow it down a bit more? drobbins: I gave some commands below that invoke the newly built rpcgen with the newly built ld-linux loader, and hopefully with those you can reproduce it and trim from there.
you should give sandbox-2.0 a try, not that i'm terribly confident it fixes things. but glibc does build fine for me on multiple systems on 1.9+. it's generally one of the packages i test before doing a sb release ...
