Portage 2.3.89 (python 3.6.10-final-0, default/linux/arm64/17.0, gcc-9.2.0, glibc-2.30-r8, 4.9.0-4-arm64 aarch64) ================================================================= System uname: Linux-4.9.0-4-arm64-aarch64-with-gentoo-2.6 KiB Mem: 131544964 total, 71042340 free KiB Swap: 3321056 total, 2644464 free Timestamp of repository gentoo: Sun, 26 Apr 2020 00:30:01 +0000 Head commit of repository gentoo: 8b5014db8b6b79a2831236bdc1b564247473e50a sh bash 4.4_p23-r1 ld GNU ld (Gentoo 2.33.1 p2) 2.33.1 app-shells/bash: 4.4_p23-r1::gentoo dev-lang/perl: 5.30.1::gentoo dev-lang/python: 2.7.17-r2::gentoo, 3.6.10-r1::gentoo, 3.7.7-r1::gentoo, 3.8.2-r1::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.69-r4::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.33.1-r1::gentoo sys-devel/gcc: 9.2.0-r2::gentoo, 9.3.0::gentoo sys-devel/gcc-config: 2.2.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.4::gentoo (virtual/os-headers) sys-libs/glibc: 2.30-r8::gentoo Repositories: gentoo location: /usr/gentoo sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-metamanifest: yes sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 ACCEPT_KEYWORDS="arm64" ACCEPT_LICENSE="@FREE" CBUILD="aarch64-unknown-linux-gnu" CFLAGS="-O2 -pipe -march=native" CHOST="aarch64-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=native" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -march=native" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news nodoc noinfo noman parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=native" GENTOO_MIRRORS="http://distfiles.gentoo.org" INSTALL_MASK="/usr/share/doc/*/*.pdf /usr/share/man/*/*" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j40" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl arm64 berkdb bzip2 cli crypt dri fortran gdbm iconv ipv6 libtirpc ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl tcpd unicode xattr zlib" ADA_TARGET="gnat_2018" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp thumb vfp vfpv3 vfpv4 vfp-d32 aes sha1 sha2 crc32 v4 v5 v6 v7 v8 thumb2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Build log is 60MB. Will upload somewhere shortly... ---- XPASS: elf/tst-latepthread UNSUPPORTED: elf/tst-ldconfig-bad-aux-cache UNSUPPORTED: elf/tst-pldd XPASS: elf/tst-protected1a XPASS: elf/tst-protected1b UNSUPPORTED: iconv/tst-gconv-init-failure UNSUPPORTED: io/tst-getcwd-abspath FAIL: io/tst-open-tmpfile XPASS: locale/tst-locale-locpath UNSUPPORTED: math/test-fesetexcept-traps UNSUPPORTED: math/test-fexcept-traps UNSUPPORTED: math/test-nearbyint-except-2 FAIL: misc/tst-pkey UNSUPPORTED: nptl/test-cond-printers UNSUPPORTED: nptl/test-condattr-printers UNSUPPORTED: nptl/test-mutex-printers UNSUPPORTED: nptl/test-mutexattr-printers UNSUPPORTED: nptl/test-rwlock-printers UNSUPPORTED: nptl/test-rwlockattr-printers FAIL: nptl/tst-rwlock18 FAIL: nptl/tst-rwlock9 UNSUPPORTED: nptl/tst-sem16 UNSUPPORTED: nss/tst-nss-db-endgrent UNSUPPORTED: nss/tst-nss-db-endpwent UNSUPPORTED: nss/tst-nss-files-alias-leak UNSUPPORTED: nss/tst-nss-files-alias-truncated UNSUPPORTED: nss/tst-nss-files-hosts-erange UNSUPPORTED: nss/tst-nss-files-hosts-getent UNSUPPORTED: nss/tst-nss-files-hosts-long UNSUPPORTED: nss/tst-nss-test3 UNSUPPORTED: posix/tst-spawn4-compat UNSUPPORTED: posix/tst-sysconf-empty-chroot UNSUPPORTED: resolv/tst-resolv-threads FAIL: rt/tst-shm UNSUPPORTED: rt/tst-shm-cancel UNSUPPORTED: sunrpc/tst-svc_register Summary of test results: 5 FAIL 5945 PASS 27 UNSUPPORTED 18 XFAIL 4 XPASS make[1]: *** [Makefile:413: tests] Error 1 make[1]: Leaving directory '/var/tmp/portage/sys-libs/glibc-2.30-r8/work/glibc-2.30' make: *** [Makefile:9: check] Error 2 ^[[31;01m*^[[0m ERROR: sys-libs/glibc-2.30-r8::gentoo failed (test phase): ^[[31;01m*^[[0m emake failed
Failures from 2.29-r8: UNSUPPORTED: elf/tst-pldd XPASS: elf/tst-protected1a XPASS: elf/tst-protected1b UNSUPPORTED: iconv/tst-gconv-init-failure UNSUPPORTED: io/tst-getcwd-abspath FAIL: io/tst-open-tmpfile UNSUPPORTED: math/test-fesetexcept-traps UNSUPPORTED: math/test-fexcept-traps UNSUPPORTED: math/test-nearbyint-except-2 FAIL: misc/tst-pkey UNSUPPORTED: nptl/test-cond-printers UNSUPPORTED: nptl/test-condattr-printers UNSUPPORTED: nptl/test-mutex-printers UNSUPPORTED: nptl/test-mutexattr-printers UNSUPPORTED: nptl/test-rwlock-printers UNSUPPORTED: nptl/test-rwlockattr-printers UNSUPPORTED: nptl/tst-sem16 FAIL: nptl/tst-stack4 UNSUPPORTED: nss/tst-nss-db-endgrent UNSUPPORTED: nss/tst-nss-db-endpwent UNSUPPORTED: nss/tst-nss-files-alias-leak UNSUPPORTED: nss/tst-nss-files-hosts-erange UNSUPPORTED: nss/tst-nss-files-hosts-getent UNSUPPORTED: nss/tst-nss-test3 UNSUPPORTED: posix/tst-spawn4-compat UNSUPPORTED: posix/tst-sysconf-empty-chroot UNSUPPORTED: resolv/tst-resolv-threads FAIL: rt/tst-shm UNSUPPORTED: rt/tst-shm-cancel UNSUPPORTED: sunrpc/tst-svc_register Summary of test results: 4 FAIL 5809 PASS 24 UNSUPPORTED 17 XFAIL 2 XPASS ---- So, new failures?: FAIL: nptl/tst-rwlock18 FAIL: nptl/tst-rwlock9
Log: https://cmpct.info/~sam/gentoo/glibc-2.30-r8-build.log
Can you attach *.out files? They usually contain actual tests' stderr that explais the failure.
Okay, here's an updated one: FAIL: misc/tst-pkey UNSUPPORTED: nptl/test-cond-printers UNSUPPORTED: nptl/test-condattr-printers UNSUPPORTED: nptl/test-mutex-printers UNSUPPORTED: nptl/test-mutexattr-printers UNSUPPORTED: nptl/test-rwlock-printers UNSUPPORTED: nptl/test-rwlockattr-printers FAIL: nptl/tst-rwlock18 FAIL: nptl/tst-rwlock9 FAIL: nptl/tst-stack4 UNSUPPORTED: nss/tst-nss-db-endgrent UNSUPPORTED: nss/tst-nss-db-endpwent UNSUPPORTED: nss/tst-nss-files-alias-leak UNSUPPORTED: nss/tst-nss-files-alias-truncated UNSUPPORTED: nss/tst-nss-files-hosts-erange UNSUPPORTED: nss/tst-nss-files-hosts-getent UNSUPPORTED: nss/tst-nss-files-hosts-long UNSUPPORTED: nss/tst-nss-test3 UNSUPPORTED: posix/tst-spawn4-compat UNSUPPORTED: posix/tst-sysconf-empty-chroot UNSUPPORTED: resolv/tst-resolv-threads UNSUPPORTED: sunrpc/tst-svc_register Summary of test results: 4 FAIL 5948 PASS 25 UNSUPPORTED 18 XFAIL 4 XPASS ---- Note that only the nptl ones matter here. The rest (inc. tst-pkey) are known to fail by upstream. Attaching each of the 3 logs now.
(In reply to Sam James (sec padawan) from comment #5) > Attaching each of the 3 logs now. https://cmpct.info/~sam/gentoo/tst-rwlock9.out https://cmpct.info/~sam/gentoo/tst-rwlock18.out https://cmpct.info/~sam/gentoo/tst-stack4.out
(In reply to Sam James (sec padawan) from comment #6) > (In reply to Sam James (sec padawan) from comment #5) > > Attaching each of the 3 logs now. > > https://cmpct.info/~sam/gentoo/tst-rwlock9.out > https://cmpct.info/~sam/gentoo/tst-rwlock18.out > https://cmpct.info/~sam/gentoo/tst-stack4.out These don't seem to provide much more detail. You can pisk a command from build.log that ran the test and explore why each test fails.
Looking at tst-rwlock9 test now: https://sourceware.org/git/?p=glibc.git;a=blob;f=nptl/tst-rwlock9.c;h=408bbcdd5d47f93337c87cdb0be6e3cb20115efb;hb=HEAD It's not a new test. tst-rwlock9 is a simple timed rwlock test: - it spawns 15 writer threads, 15 readers threads - against a single rwlock with priority for writes - each write thread acquires the lock , waits for 1 ms and releases it. repeats 10 times - each read thread acquires the lock , waits for 1 ms and releases it. repeats 15 times - each test is ran for 3 clock types My understanding is that: - best case is when all writers sequentially acquire the lock (15 * 10 * 1 ms = 150ms), then all readers in parallel acquire the lock (15 * 10 * 1 ms = 150ms). All in all 300 ms. 3 x 300ms = 900ms. - worst case: readers and writers are interspersed and effectively serialize one another. Still not too bad: 15 * 10 * 1ms writers + 15 * 15 * 1 ms = 375 ms. 3 x 375ms = 1.125s. This does not account for actual locking overhead, which might be very significant. My math is off somewhere as on amd64 tests finish in 537ms for me: $ time { build-x86-x86_64-pc-linux-gnu-nptl/nptl/tst-rwlock9 > /dev/null; } real 0m0,537s user 0m0,049s sys 0m0,099s But that at least matches order of magnitude for best case. On 96-core arm64 tests timeout in 30 seconds for me. That looks wrong. rwlock should be somewhat fair for mostly sleeping threads. We have 25 contending cores though and they all have the same timedlock timeout.
It's also probably not an effect of contention across cores: $ time { nptl/tst-rwlock9 >o; } real 0m30.121s user 6m53.276s sys 0m5.408s $ time { taskset -c 0 nptl/tst-rwlock9 >o; } real 0m30.108s user 0m20.936s sys 0m8.932s That's quite a lot of system time. As if nanosleep was implemented as busy non-preemptible loop.
I also created up-to-date chroot with gcc-9.3.0 and now my tests pass in 0.6 seconds as expected: """ # time { env GCONV_PATH=/var/tmp/portage/sy... real 0m0.633s user 0m0.100s sys 0m0.404s """ Can you retest against latest stable gcc?
(In reply to Sergei Trofimovich from comment #10) > Can you retest against latest stable gcc? Specificall since gcc-9.2.0 we had a few potential fixes: https://gcc.gnu.org/PR92692 / https://sourceware.org/PR24924 which mentions the same tests.
With gcc-9.3.0 I'm down to 1 test failure: """ FAIL: nptl/tst-stack4 Summary of test results: 1 FAIL 5949 PASS 25 UNSUPPORTED 21 XFAIL 3 XPASS """ Looking at it now: cat $ nptl/tst-stack4.out Didn't expect signal from child: got `Aborted' Test on it's own takes 14 seconds: real 0m14.315s user 0m7.176s sys 0m9.136s Yet another threaded test: https://sourceware.org/git/?p=glibc.git;a=blob;f=nptl/tst-stack4.c;h=3030f75027f837c92f4c833a146d4148119a898f;hb=HEAD - it has 100s timeout (ok, 10s is a lot) - test is flaky: If it run in in the loop it passes most of time time, but occasionally fails with various crashes related to seeming memory corruption # while :; do date; env GCONV_PATH=/var/tmp/portage/sys...; done Sun 03 May 2020 10:42:08 AM UTC Sun 03 May 2020 10:42:21 AM UTC Sun 03 May 2020 10:42:34 AM UTC Didn't expect signal from child: got `Segmentation fault' Sun 03 May 2020 10:42:43 AM UTC Sun 03 May 2020 10:42:56 AM UTC malloc(): invalid size (unsorted) Didn't expect signal from child: got `Aborted' Sun 03 May 2020 10:42:57 AM UTC Sun 03 May 2020 10:43:12 AM UTC Sun 03 May 2020 10:43:28 AM UTC Sun 03 May 2020 10:43:46 AM UTC Sun 03 May 2020 10:44:01 AM UTC malloc(): invalid size (unsorted) Didn't expect signal from child: got `Aborted' Sun 03 May 2020 10:44:03 AM UTC Sun 03 May 2020 10:44:18 AM UTC
(In reply to Sergei Trofimovich from comment #12) > FAIL: nptl/tst-stack4 https://sourceware.org/PR19329 looks relevant.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f55f2be1bd6791465453c7b0dfc60bf4c3c75613 commit f55f2be1bd6791465453c7b0dfc60bf4c3c75613 Author: Sergei Trofimovich <slyfox@gentoo.org> AuthorDate: 2020-05-03 11:01:18 +0000 Commit: Sergei Trofimovich <slyfox@gentoo.org> CommitDate: 2020-05-03 11:01:35 +0000 sys-libs/glibc: disable flaky tst-stack4, bug #719674 tst-stack4 exposes known race condition in glibc (https://sourceware.org/PR19329). Let's disable this test until it's fixed upstream. Reported-by: Sam James Bug: https://bugs.gentoo.org/719674 Bug: https://sourceware.org/PR19329 Package-Manager: Portage-2.3.99, Repoman-2.3.22 Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org> sys-libs/glibc/glibc-2.30-r8.ebuild | 5 +++++ sys-libs/glibc/glibc-2.31-r2.ebuild | 5 +++++ sys-libs/glibc/glibc-9999.ebuild | 5 +++++ 3 files changed, 15 insertions(+)
I assume tests don't fail anymore.
