https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/ Issue: dev-libs/rocksdb-8.10.0 fails tests. Discovered on: amd64 (internal ref: ci) Info about the issue: https://wiki.gentoo.org/wiki/Project:Tinderbox/Common_Issues_Helper#CF0015
Created attachment 883546 [details] build.log.xz build log and emerge --info (compressed because it exceeds attachment limit, use 'xzless' to read it)
I have no idea on why there isn't the usual emerge log at the end of each failure, however the emerge process returned an exit code of 1
dmesg also reports: [Mon Jan 29 11:46:58 2024] db_tailing_iter[16872]: segfault at 5e45226dbc56 ip 000071e8207fe516 sp 00007ffeba6d2570 error 4 in librocksdb.so.8.10.0[71e820648000+755000] likely on CPU 0 (core 0, socket 0) [Mon Jan 29 11:46:58 2024] db_tailing_iter[16209]: segfault at 5b478a230081 ip 000072b6b66fc516 sp 00007ffc8f117d70 error 4 in librocksdb.so.8.10.0[72b6b6546000+755000] likely on CPU 40 (core 8, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[39967]: segfault at 5d4ebc88e ip 0000745738214516 sp 00007ffd590e38d0 error 4 in librocksdb.so.8.10.0[74573805e000+755000] likely on CPU 16 (core 16, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[40030]: segfault at 5a2616a00 ip 00007434a5cc3516 sp 00007ffd9b59d200 error 4 in librocksdb.so.8.10.0[7434a5b0d000+755000] likely on CPU 44 (core 12, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[41361]: segfault at 5fb14e660f6c ip 00007cc9cde8c516 sp 00007fff46b08020 error 4 in librocksdb.so.8.10.0[7cc9cdcd6000+755000] likely on CPU 60 (core 28, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[41392]: segfault at 5db47fc7a92c ip 000078521c48f516 sp 00007ffec965bab0 error 4 in librocksdb.so.8.10.0[78521c2d9000+755000] likely on CPU 42 (core 10, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[40411]: segfault at 5af598eba ip 00007063501e7516 sp 00007ffd2526e4d0 error 4 in librocksdb.so.8.10.0[706350031000+755000] likely on CPU 11 (core 11, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[40365]: segfault at 56ae06f0f690 ip 00007eac5f20c516 sp 00007ffeb2041250 error 4 in librocksdb.so.8.10.0[7eac5f056000+755000] likely on CPU 53 (core 21, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[41048]: segfault at 650a03d83457 ip 00007c1386bf4516 sp 00007fff9210d970 error 4 in librocksdb.so.8.10.0[7c1386a3e000+755000] likely on CPU 41 (core 9, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[41214]: segfault at 5d5ad0c3b180 ip 000076766c04b516 sp 00007ffc042dddf0 error 4 in librocksdb.so.8.10.0[76766be95000+755000] likely on CPU 14 (core 14, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[41083]: segfault at 597e53f8ecf7 ip 00007bad6fe2c516 sp 00007fff01c64e50 error 4 in librocksdb.so.8.10.0[7bad6fc76000+755000] likely on CPU 59 (core 27, socket 0) [Mon Jan 29 11:47:44 2024] prefetch_test[42284]: segfault at 58c8c16cba0a ip 00007a0372c04516 sp 00007ffcb3e16410 error 4 in librocksdb.so.8.10.0[7a0372a4e000+755000] likely on CPU 7 (core 7, socket 0)
Hi, I think the package is fine. I ran all the tests locally and they are quite demanding. I saw a single test failure out of ~17k tests: ... 17040/17040 Test #16898: RangeLockingTest.BasicRangeLocking ............................................................................................................................................................................................................. Passed 3.10 sec 99% tests passed, 1 tests failed out of 16978 Total Test time (real) = 864.82 sec ... The following tests FAILED: 7799 - ArenaTest.UnmappedAllocation (Failed) Could it be a local problem in the tinderbox?
(In reply to Guilherme Amadio from comment #4) > Could it be a local problem in the tinderbox? I gave it another try. I will post the logs
Created attachment 883547 [details] build.log.xz build.log.xz
Created attachment 883548 [details] LastTest.log.xz LastTest.log.xz
I tried with the same CXXFLAGS/LDFLAGS as you, but could not reproduce the crashes reported here. Can you try with empty CXXFLAGS and LDFLAGS? That is, try # env CFLAGS= CXXFLAGS= LDFLAGS= FEATURES=test USE='static-libs test' emerge --buildpkgonly rocksdb I remember with previous versions that some optimization flags could cause problems. Also, try to disable ccache if you can. FYI, this is what I have for emerge --info: gentoo ~ $ emerge --info Portage 3.0.61 (python 3.11.7-final-0, default/linux/amd64/17.1/systemd/merged-usr, gcc-12, glibc-2.38-r9, 6.7.2-gentoo x86_64) ================================================================= System uname: Linux-6.7.2-gentoo-x86_64-AMD_Ryzen_9_3950X_16-Core_Processor-with-glibc2.38 KiB Mem: 32748948 total, 4990996 free KiB Swap: 0 total, 0 free Head commit of repository amadio: 471ca43229429d5379d7a0d1b4b50328ebebff53 Head commit of repository gentoo: b71344e061f6d6edc8396e8b5acfcbf1acad25ef sh bash 5.2_p26 ld GNU ld (Gentoo 2.41 p5) 2.41.0 ccache version 4.9 [disabled] app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.2_p26::gentoo dev-build/autoconf: 2.72-r1::gentoo dev-build/automake: 1.16.5-r1::gentoo dev-build/cmake: 3.28.1-r1::gentoo dev-build/libtool: 2.4.7-r2::gentoo dev-build/make: 4.4.1-r1::gentoo dev-build/meson: 1.3.1-r1::gentoo dev-java/java-config: 2.3.3-r1::gentoo dev-lang/perl: 5.38.2-r1::gentoo dev-lang/python: 2.7.18_p16-r1::gentoo, 3.10.13::gentoo, 3.11.7::gentoo, 3.12.1_p1::gentoo dev-lang/rust: 1.74.1::gentoo dev-util/ccache: 4.9::gentoo sys-apps/baselayout: 2.14-r1::gentoo sys-apps/sandbox: 2.38::gentoo sys-apps/systemd: 255.3::gentoo sys-devel/binutils: 2.41-r5::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/clang: 15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.6::gentoo sys-devel/gcc: 11.4.1_p20240111::gentoo, 12.3.1_p20240112::gentoo, 13.2.1_p20240113-r1::gentoo, 14.0.1.9999::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/llvm: 15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.6::gentoo sys-kernel/linux-headers: 6.7::gentoo (virtual/os-headers) sys-libs/glibc: 2.38-r9::gentoo Repositories: amadio location: /var/db/repos/amadio sync-type: git sync-uri: git+ssh@github.com/amadio/gentoo-overlay.git masters: gentoo priority: 0 volatile: True sync-git-pull-extra-opts: --quiet gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: git+ssh@git.gentoo.org/repo/gentoo.git priority: 1 volatile: True sync-git-pull-extra-opts: --quiet ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -Wl,--build-id -pipe -w" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -Wl,--build-id -pipe -w" DISTDIR="/srv/portage/distfiles" EMERGE_DEFAULT_OPTS="--jobs=4 --keep-going" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-march=native -O2 -Wl,--build-id -pipe -w" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync warn-on-large-env xattr" FFLAGS="-march=native -O2 -Wl,--build-id -pipe -w" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="C.UTF-8" LDFLAGS="-Wl,-O1" LEX="flex" LINGUAS="en ja pt pt_BR" MAKEOPTS="-j12 -l12" PKGDIR="/srv/portage/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/tmp" SHELL="/bin/bash" USE="X acl alsa amd64 bzip2 cairo cjk cli corefonts crypt curl dbus djvu dri egl fontconfig fortran gdbm gif gpm gssapi gzip iconv icu infinality ipv6 jpeg jpeg2k kerberos lcms libglvnd libtirpc lz4 lzma lzo mng multilib ncurses nls opengl openmp pam pango pcre pdf png postscript readline seccomp sqlite ssl svg systemd test-rust threads tiff truetype udev unicode vulkan webp xattr xcb xft xkb xml xpm zip zlib zstd" ABI_X86="64 32" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="evdev libinput" KERNEL="linux" L10N="en ja pt pt-BR" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1 lua5-3 lua5-4 luajit" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11 python3_10 python3_12" RUBY_TARGETS="ruby31 ruby32 ruby33" VIDEO_CARDS="nouveau nvidia virgl" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
/var/tmp/portage/dev-libs/rocksdb-8.10.0/work/rocksdb-8.10.0/db/db_tailing_iter_test.cc:365: Failure iter->status() Not implemented: ReadAsync <end of output> Test time = 0.14 sec /var/tmp/portage/dev-libs/rocksdb-8.10.0/work/rocksdb-8.10.0/db/db_tailing_iter_test.cc:572: Failure Expected equality of these values: 0 immutable_seeks Which is: 1 Command: "/var/tmp/portage/dev-libs/rocksdb-8.10.0/work/rocksdb-8.10.0_build/env_test" "--gtest_filter=EnvPosixTest.LoadRocksDBLibrary" "--gtest_also_run_disabled_tests" Directory: /var/tmp/portage/dev-libs/rocksdb-8.10.0/work/rocksdb-8.10.0_build "EnvPosixTest.LoadRocksDBLibrary" start time: Jan 29 12:25 CET Output: ---------------------------------------------------------- Note: Google Test filter = EnvPosixTest.LoadRocksDBLibrary [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from EnvPosixTest [ RUN ] EnvPosixTest.LoadRocksDBLibrary ERROR: something wrong with flag 'block_cache_trace_path' in file '/var/tmp/portage/dev-libs/rocksdb-8.10.0/work/rocksdb-8.10.0/tools/block_cache_analyzer/block_cache_trace_analyzer.cc'. One possibility: file '/var/tmp/portage/dev-libs/rocksdb-8.10.0/work/rocksdb-8.10.0/tools/block_cache_analyzer/block_cache_trace_analyzer.cc' is being linked both statically and dynamically into this executable. free(): double free detected in tcache 2 <end of output> Test time = 0.06 sec ... if I had to guess, I'd check the fs PORTAGE_TMPDIR is on first.
Yes, XRootD needs a filesystem with support for attributes to run tests, and rocksdb likely does as well. I think you could enable support in your kernel to avoid problems like this in the future. I have gentoo ~ # zgrep -i config_tmpfs /proc/config.gz CONFIG_TMPFS=y CONFIG_TMPFS_POSIX_ACL=y CONFIG_TMPFS_XATTR=y CONFIG_TMPFS_INODE64=y # CONFIG_TMPFS_QUOTA is not set so maybe that's why it works for me and not in tinderbox.
