Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 880249 - dev-lang/rust: stage0 rustc segfaults on -system-bootstrap build on phenom
Summary: dev-lang/rust: stage0 rustc segfaults on -system-bootstrap build on phenom
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Gentoo Rust Project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-11-07 19:37 UTC by JohnLM
Modified: 2024-02-14 10:26 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
output of a failed emerge (emerge.log,30.17 KB, text/x-log)
2022-11-07 19:55 UTC, JohnLM
Details
emerge failure, glibc w/o clone3 (emerge_run1.log,53.60 KB, text/plain)
2022-11-10 18:55 UTC, JohnLM
Details

Note You need to log in before you can comment on or make changes to this bug.
Description JohnLM 2022-11-07 19:37:07 UTC
Binary Rust compiler used as stage0 in building Rust segfaults on emerge. (rustup installed stable Rust compiler also segfaults in similar manner compiling seemingly anything.)

System runs AMD Phenom 9950, which -- I hope -- shouldn't be a factor. (That's no AVX instructions).

At the same time +system-bootstrap build finishes without a problem compiles itself (bootstrap) and other Rust-dependent packages fine.

And rustup-installed compiler seems to work when running another distro (Linux Mint) on the same machine.

Reproducible: Always

Steps to Reproduce:
1. USE=-system-bootstrap emerge rust

"My machine" factor applies.
Actual Results:  
emerge fails! rustc fails with a segfault, syslog/journal logs the segfault producing a core dump. emerge either exits or hangs on "warning: build failed, waiting for other jobs to finish..." (requiring ctrl+c to stop).

Expected Results:  
Rust compiles, no segfaults occur.

$ emerge --info
Portage 3.0.38.1 (python 3.10.8-final-0, default/linux/amd64/17.1/systemd, gcc-11.3.0, glibc-2.35-r8, 5.15.75-gentoo x86_64)
=================================================================
System uname: Linux-5.15.75-gentoo-x86_64-AMD_Phenom-tm-_9950_Quad-Core_Processor-with-glibc2.35
KiB Mem:     8151524 total,    475132 free
KiB Swap:    4194300 total,   4185596 free
Timestamp of repository gentoo: Sat, 05 Nov 2022 17:15:01 +0000
Head commit of repository gentoo: f1edb9f1f4cf108f4c683e366c7272aaddce68f5
sh bash 5.1_p16-r2
ld GNU ld (Gentoo 2.38 p4) 2.38
distcc 3.4 x86_64-pc-linux-gnu [disabled]
app-misc/pax-utils:        1.3.5::gentoo
app-shells/bash:           5.1_p16-r2::gentoo
dev-java/java-config:      2.3.1::gentoo
dev-lang/perl:             5.34.1-r4::gentoo
dev-lang/python:           2.7.18_p16::gentoo, 3.10.8_p2::gentoo
dev-lang/rust:             1.64.0-r1::gentoo
dev-util/cmake:            3.24.2::gentoo
dev-util/meson:            0.63.2-r1::gentoo
sys-apps/baselayout:       2.8::gentoo
sys-apps/sandbox:          2.29::gentoo
sys-apps/systemd:          251.4::gentoo
sys-devel/autoconf:        2.71-r1::gentoo
sys-devel/automake:        1.16.5::gentoo
sys-devel/binutils:        2.38-r2::gentoo
sys-devel/binutils-config: 5.4.1::gentoo
sys-devel/clang:           14.0.6-r1::gentoo, 15.0.3::gentoo
sys-devel/gcc:             11.3.0::gentoo
sys-devel/gcc-config:      2.8::gentoo
sys-devel/libtool:         2.4.7::gentoo
sys-devel/llvm:            14.0.6-r2::gentoo, 15.0.3::gentoo
sys-devel/make:            4.3::gentoo
sys-kernel/linux-headers:  5.15-r3::gentoo (virtual/os-headers)
sys-libs/glibc:            2.35-r8::gentoo
Repositories:

cross-portage
    location: /usr/local/portage-crossdev
    masters: gentoo
    priority: 10

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.europe.gentoo.org/gentoo-portage
    priority: 25
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: yes

esteid
    location: /var/lib/layman/esteid
    sync-type: laymansync
    sync-uri: https://github.com/open-eid/gentoo.git
    masters: gentoo
    priority: 50

java
    location: /var/lib/layman/java
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/java.git
    masters: gentoo
    priority: 50

mv
    location: /var/lib/layman/mv
    sync-type: laymansync
    sync-uri: https://anongit.gentoo.org/git/user/mv.git
    masters: gentoo
    priority: 50

johnlm-overlay
    location: /var/lib/layman/johnlm-overlay
    sync-type: laymansync
    sync-uri: file:///home/johnlm/src/johnlm-overlay.git
    masters: gentoo
    priority: 60

local-overlay
    location: /usr/local/portage
    masters: gentoo
    priority: 95

Installed sets: @archivers, @major-bdeps
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=barcelona -mcx16 -fomit-frame-pointer -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=barcelona -mcx16 -fomit-frame-pointer -O2 -pipe"
DISTDIR="/mnt/megadisk/system/tmp/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=n --verbose-conflicts"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news noinfo nostrip parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en en_GB lv"
MAKEOPTS="-j4"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="3dnow 3dnowext X a52 acl acpi amd64 bash-completion branding bzip2 cairo cli crypt css dbus dri dts evdev exif ffmpeg flac fontconfig fortran g3dvl gallium gdbm gmp gnome gnome-keyring gstreamer gtk gtk3 gtkstyle gzip iconv jpeg kpathsea libglvnd libnotify libtirpc mad matroska mmx mmxext mp3 multilib ncurses nls nptl ogg openexr opengl openmp opus oss oss4 pam pcre png policykit qt3support readline seccomp smp sndfile split-usr sse sse2 sse3 sse4a ssl svg systemd test-rust threads tiff truetype udev unicode vdpau vorbis x264 xa xattr xcb xcomposite xinerama xml xv xvid xvmc zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="ptp2 kodak_ez200" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en en-GB lv" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Comment 1 JohnLM 2022-11-07 19:40:35 UTC
$ emerge --info glibc
[...]
sys-libs/glibc-2.35-r8::gentoo was built with the following:
USE="clone3 multiarch (multilib) ssp stack-realign (static-libs) systemd -audit -caps (-cet) -compile-locales (-crypt) (-custom-cflags) -doc (-experimental-loong) -gd -headers-only -multilib-bootstrap -nscd -profile (-selinux) -suid -systemtap -test (-vanilla)" ABI_X86="(64)"
CFLAGS="-march=barcelona -pipe -fno-builtin-strlen -O2"
CXXFLAGS="-march=barcelona -pipe -fno-builtin-strlen -O2"
FEATURES="binpkg-docompress network-sandbox userfetch binpkg-logs ipc-sandbox pid-sandbox usersync merge-sync config-protect-if-modified distlocks protect-owned assume-digests userpriv unknown-features-warn parallel-fetch noinfo qa-unresolved-soname-deps fixlafiles multilib-strict preserve-libs nostrip sfperms binpkg-dostrip news sandbox xattr usersandbox buildpkg-live unmerge-logs unmerge-orphans strict ebuild-locks"
Comment 2 JohnLM 2022-11-07 19:55:58 UTC
Created attachment 828519 [details]
output of a failed emerge

The exact place of error varies, but it's not completely random. I've noticed some pattern in the address offset. On last build the syslog got:

Nov 07 21:29:29 isg005 kernel: opt build_scrip[263489]: segfault at 18 ip 00007f6c4c8a5a85 sp 00007f6c3d1fb318 error 4 in libc.so.6[7f6c4c82b000+16f000]
Nov 07 21:29:29 isg005 kernel: Code: 83 c8 01 c3 90 ff ca 7e dc 0f b7 0e 0f b7 07 0f c9 0f c8 d1 e9 d1 e8 8a 0c 16 0f b6 3c 17 09 f8 29 c8 c3 66 90 b9 ff ff 00 00 <0f> 10 06 0f 10 0f 66 0f 74 c8 66 0f d7 c1 29 c8 75 39 48 83 ea 20
Nov 07 21:29:29 isg005 systemd[1]: Started Process Core Dump (PID 263491/UID 0).
Nov 07 21:29:29 isg005 kernel: opt build_scrip[263510]: segfault at 10 ip 00007fcb05eda53c sp 00007fcaf83fc8a0 error 4 in libLLVM-14-rust-1.63.0-stable.so[7fcb0536f000+29f1000]
Nov 07 21:29:29 isg005 kernel: Code: 48 89 84 24 18 01 00 00 31 db 4c 89 74 24 30 4c 89 7c 24 28 66 2e 0f 1f 84 00 00 00 00 00 49 8b 42 30 4c 8b 2c c8 49 8b 45 00 <48> 8b 40 10 48 3b 05 41 41 2b 02 89 9c 24 0c 01 00 00 48 89 8c 24
Nov 07 21:29:29 isg005 systemd[1]: Started Process Core Dump (PID 263512/UID 0).
Nov 07 21:29:30 isg005 systemd-coredump[263492]: elfutils disabled, parsing ELF objects not supported
Nov 07 21:29:30 isg005 systemd-coredump[263492]: [🡕] Process 263459 (rustc) of user 250 dumped core.
Nov 07 21:29:30 isg005 systemd[1]: systemd-coredump@19-263491-0.service: Deactivated successfully.
Nov 07 21:29:30 isg005 systemd-coredump[263513]: elfutils disabled, parsing ELF objects not supported
Nov 07 21:29:30 isg005 systemd-coredump[263513]: [🡕] Process 263457 (rustc) of user 250 dumped core.
Nov 07 21:29:30 isg005 systemd[1]: systemd-coredump@20-263512-0.service: Deactivated successfully.

Emerge output including traceback in the attachement. (Copied from terminal, the portage didn't leave the tmpdir around).
Comment 3 JohnLM 2022-11-07 20:03:59 UTC
Oh! I should probably tell that this started versions 1.53.0 and 1.58.0 as dev-lang/rust-1.58.0 was first one to fail to build. Somehow I ended up trying system-bootstrapping, and I bootstrapped from 1.53.0 (I had installed at the time) through 1.58.0 (latest then) to the 1.64.0-r1 (latest now).
Comment 4 JohnLM 2022-11-07 20:04:54 UTC
* _between_ versions 1.53.0 and 1.58.0
Comment 5 Georgy Yakovlev archtester gentoo-dev 2022-11-07 22:56:45 UTC
if you re-emerge systemd with USE=elfutils, you'd get more info in logs.

I highly recommend enabling it for time being. very useful.

also you probably should be able to get a stacktrace with 

    coredumpctl info rustc


ideally try those steps to reproduce and provide more info

1. install gdb, if not installed yet
2. re-install systemd with USE=elfutils
3. reproduce the crash, do not clean up /var/tmp/portage/... yet
4. run: coredumpctl info rustc (provide output)
5. run: coredumpctl debug rustc
6. in the gdb prompt, which opens as result of step 5, type: bt (provide output)
7. provide output of 'qlop -vmu'


qlop output will help me see what else of significance has been updated in time period around rust-1.53 and later.

you may need to rebuild glibc with debugging information (https://wiki.gentoo.org/wiki/Debugging), -ggdb in CFLAGS and nostrip or splitdebug in FEATURES should be enough.

but since you already have fno-builtin-strlen for it I guess you already have debuginfo for it as well.

basically I'd really like to see gdb backtrace or at least coredumpctl backtrace.
Comment 6 Georgy Yakovlev archtester gentoo-dev 2022-11-09 19:32:45 UTC
Log in New API Help About
Rust failure gdb backtrace
6.8 KB of Plain text
Created 3 minutes ago — expires in 7 days
Viewed 2 times
COPY TO CLIPBOARD SOFT WRAP RAW TEXT DUPLICATE DIFF

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

	

(gdb) bt full
#0  arena_get_from_edata (edata=0x0) at include/jemalloc/internal/arena_inlines_b.h:16
No locals.
#1  _rjem_je_large_dalloc (tsdn=0x7f5cb99febf8, edata=0x0) at src/large.c:271
        arena = <optimized out>
#2  0x000055c51b8dc48c in arena_dalloc_large (ptr=<optimized out>, tcache=<optimized out>, szind=<optimized out>, slow_path=false, tsdn=<optimized out>) at include/jemalloc/internal/arena_inlines_b.h:297
        edata = 0x0
        edata = <optimized out>
#3  arena_dalloc (ptr=<optimized out>, tcache=<optimized out>, slow_path=false, tsdn=<optimized out>, caller_alloc_ctx=<optimized out>) at include/jemalloc/internal/arena_inlines_b.h:334
        alloc_ctx = <optimized out>
#4  idalloctm (ptr=<optimized out>, tcache=<optimized out>, is_internal=false, slow_path=false, tsdn=<optimized out>, alloc_ctx=<optimized out>) at include/jemalloc/internal/jemalloc_internal_inlines_c.h:120
No locals.
#5  ifree (ptr=<optimized out>, tcache=<optimized out>, slow_path=false, tsd=<optimized out>) at src/jemalloc.c:2887
        alloc_ctx = <optimized out>
        usize = 289078108240281600
#6  _rjem_je_free_default (ptr=<optimized out>) at src/jemalloc.c:3014
        tcache = <optimized out>
        tsd = <optimized out>
#7  0x00007f5cd30cf097 in llvm::BranchProbabilityInfo::calculate(llvm::Function const&, llvm::LoopInfo const&, llvm::TargetLibraryInfo const*, llvm::DominatorTree*, llvm::PostDominatorTree*) ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#8  0x00007f5cd39c2f94 in llvm::detail::AnalysisPassModel<llvm::Function, llvm::BranchProbabilityAnalysis, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Function>::Invalidator>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#9  0x00007f5cd269c6e7 in llvm::AnalysisManager<llvm::Function>::getResultImpl(llvm::AnalysisKey*, llvm::Function&) ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#10 0x00007f5cd29f3cb6 in llvm::detail::AnalysisPassModel<llvm::Function, llvm::BlockFrequencyAnalysis, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Function>::Invalidator>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&) () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#11 0x00007f5cd269c6e7 in llvm::AnalysisManager<llvm::Function>::getResultImpl(llvm::AnalysisKey*, llvm::Function&) ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#12 0x00007f5cd2a1cdaf in llvm::AlwaysInlinerPass::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#13 0x00007f5cd2a1caad in llvm::detail::PassModel<llvm::Module, llvm::AlwaysInlinerPass, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&)
    () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#14 0x00007f5cd35060e6 in llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.65.0-stable.so
No symbol table info available.
#15 0x00007f5cd7d8b341 in LLVMRustOptimizeWithNewPassManager () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so
No symbol table info available.
#16 0x00007f5cd7d877ca in rustc_codegen_llvm::back::write::optimize_with_new_llvm_pass_manager () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so
No symbol table info available.
#17 0x00007f5cd7d86815 in rustc_codegen_llvm::back::write::optimize () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so
No symbol table info available.
#18 0x00007f5cd7bb172b in rustc_codegen_ssa::back::write::execute_work_item::<rustc_codegen_llvm::LlvmCodegenBackend> ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so
No symbol table info available.
#19 0x00007f5cd7bafc38 in std::sys_common::backtrace::__rust_begin_short_backtrace::<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()> ()
   from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so
No symbol table info available.
--Type <RET> for more, q to quit, c to continue without paging--c
#20 0x00007f5cd7b8a738 in <<std::thread::Builder>::spawn_unchecked_<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} () from /home/johnlm/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so
No symbol table info available.
#21 0x00007f5cd58cd003 in alloc::boxed::{impl#44}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1940
No locals.
#22 alloc::boxed::{impl#44}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:1940
No locals.
#23 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
No locals.
#24 0x00007f5cd56246da in start_thread (arg=<optimized out>) at pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140036239554608, 5389326673050754667, 140036227987008, 0, 140036693705728, 0, -5443779712600351125, -5443982464774676885}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#25 0x00007f5cd56a9bdc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
No locals.

Share:
Comment 7 Georgy Yakovlev archtester gentoo-dev 2022-11-09 19:33:29 UTC
looks like clone3() is responsible.

disabling that flag on glibc might be a workaround, however I can't answer why it's happening.
Comment 8 Andreas K. Hüttel archtester gentoo-dev 2022-11-09 19:52:05 UTC
> 
> $ emerge --info
> Portage 3.0.38.1 (python 3.10.8-final-0, default/linux/amd64/17.1/systemd,
> gcc-11.3.0, glibc-2.35-r8, 5.15.75-gentoo x86_64)

Could you please update to the newest glibc-2.35 available and try again?
i.e. sys-libs/glibc-2.35-r11

There were some code path selection fixes...
Comment 9 Ionen Wolkens gentoo-dev 2022-11-09 19:56:16 UTC
(In reply to JohnLM from comment #0)
> And rustup-installed compiler seems to work when running another distro
> (Linux Mint) on the same machine.
fwiw what glibc version does your linux mint install have? If it's glibc-2.33 or older it likely wouldn't say much given it didn't use clone3.
Comment 10 JohnLM 2022-11-10 18:55:30 UTC
Created attachment 830299 [details]
emerge failure, glibc w/o clone3

OK, I rebuilt glibc with USE=-clone3. It didn't make the segfault go away. Attached log for `USE=-system-bootstrap MAKEOPTS=-j1 emerge -a1v rust`
The coredump did stack traces of 5 threads -- which is cool -- but I'm not sure if it's any more useful.

In attached log segfault was triggered in jemalloc code again, but it's not necessarily always so. I did a bunch more `cargo +stable build` runs. 2/3 segfaults were in LLVM code and  about half of the time the build simply froze without triggering a segfault. Hmm...
Comment 11 JohnLM 2022-11-10 19:44:30 UTC
(In reply to Ionen Wolkens from comment #9)
> (In reply to JohnLM from comment #0)
> > And rustup-installed compiler seems to work when running another distro
> > (Linux Mint) on the same machine.
> fwiw what glibc version does your linux mint install have? If it's
> glibc-2.33 or older it likely wouldn't say much given it didn't use clone3.

I used Linux Mint 20.3 LiveUSB. It runs glibc-2.31

(In reply to Andreas K. Hüttel from comment #8)
> Could you please update to the newest glibc-2.35 available and try again?
> i.e. sys-libs/glibc-2.35-r11
> 
> There were some code path selection fixes...

I unkeyworded and updated to sys-libs/glibc-2.35-r11, still with USE=-clone3
No changes, unfortunately.
Comment 12 Georgy Yakovlev archtester gentoo-dev 2022-11-11 02:54:09 UTC
can you attach backtrace failure without clone3?
Comment 13 Georgy Yakovlev archtester gentoo-dev 2022-11-11 02:55:06 UTC
I see it still fails in pthread_create, but since clone3 is disabled - stack should look differently.
Comment 14 JohnLM 2022-11-11 07:13:34 UTC
(In reply to Georgy Yakovlev from comment #12)
> can you attach backtrace failure without clone3?

(In reply to Georgy Yakovlev from comment #13)
> I see it still fails in pthread_create, but since clone3 is disabled - stack
> should look differently.

Yes, I already did. That's attachment 830299 [details] -- it's different that it now uses clone() instead of clone3()
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-11-11 07:20:12 UTC
I think clone* is a redherring here, it's just that clone3 was a problem with seccomp before. It's something glibc uses a lot (which is why it was a problem with seccomp+clone3) internally.
Comment 16 Andreas K. Hüttel archtester gentoo-dev 2022-11-12 18:05:17 UTC
(In reply to Sam James from comment #15)
> I think clone* is a redherring here, it's just that clone3 was a problem
> with seccomp before. It's something glibc uses a lot (which is why it was a
> problem with seccomp+clone3) internally.

++
Comment 17 JohnLM 2022-11-23 19:07:47 UTC
Thanks for the help so far! Do you have any more ideas for me to try? Seems like a dead end at the moment.
Comment 18 JohnLM 2022-11-25 18:56:39 UTC
At least out of curiosity I tried grepping for AVX instructions. I tried some commands like this:

$ objdump -d ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-a21dfa8672cc0cdd.so | awk '/^[0-9a-f]/ { l = $0 ; x = 1 }; /^ .*v(broad|insert|mask|per|zero)/ { if (x) { print l ; x = 0 } }'

and -- sure enough -- libLLVM and librustc_driver (but not rustc itself) found some.

So far so nice, but I also found some AVX in my system's librustc_driver (the working one)! Given this _and_ the fact that all the function names had "avx" somewhere in their names I'm completely sure the AVX code is used only if a runtime check passes. So _not_ the culprit. Oh well.