dev-util/promu-0.15.0, when built by portage, on x86, segfaults. dev-util/promu-0.14.0, when built by portage, on x86, works. dev-util/promu-0.15.0, when OUTSIDE of portage, on x86, works. dev-util/promu-0.14.0, when OUTSIDE of portage, on x86, works (I copied the binary here so I could try and compare) $ strace -ff /var/tmp/promu-0.15.0-gentoo execve("/var/tmp/promu-0.15.0-gentoo", ["/var/tmp/promu-0.15.0-gentoo"], 0xffcf5c04 /* 32 vars */) = 0 brk(NULL) = 0xa6c6000 brk(0xa6c6880) = 0xa6c6880 set_thread_area({entry_number=-1, base_addr=0xa6c6380, limit=0x0fffff, seg_32bit=1, contents=0, read_exec_only=0, limit_in_pages=1, seg_not_present=0, useable=1}) = 0 (entry_number=12) set_tid_address(0xa6c63e8) = 32078 set_robust_list(0xa6c63ec, 12) = 0 rseq(0xa6c6820, 0x20, 0, 0x53053053) = -1 ENOSYS (Function not implemented) ugetrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 readlinkat(AT_FDCWD, "/proc/self/exe", "/var/tmp/promu-0.15.0-gentoo", 4096) = 28 getrandom("\x7f\xe8\xa3\x05", 4, GRND_NONBLOCK) = 4 brk(NULL) = 0xa6c6880 brk(0xa6e7880) = 0xa6e7880 brk(0xa6e8000) = 0xa6e8000 mprotect(0x8636000, 3637248, PROT_READ) = 0 set_thread_area({entry_number=-1, base_addr=0x89eecb4, limit=0x0fffff, seg_32bit=1, contents=0, read_exec_only=0, limit_in_pages=1, seg_not_present=0, useable=1}) = 0 (entry_number=13) --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} --- +++ killed by SIGSEGV +++ Segmentation fault There are no GO* env vars anywhere in /etc/portage. Reproduced with: dev-lang/go-1.21.4 dev-lang/go-1.20.8 I think it's something to do w/ promu's static building. /var/tmp/promu-0.14.0-gentoo: linux-gate.so.1 (0xf7f41000) libresolv.so.2 => /lib/libresolv.so.2 (0xf75a4000) libc.so.6 => /lib/libc.so.6 (0xf7387000) /lib/ld-linux.so.2 (0xf7f43000) /var/tmp/promu-0.15.0-gentoo: not a dynamic executable /var/tmp/promu-0.15.0-upstream: linux-gate.so.1 (0xf7f19000) libresolv.so.2 => /lib/libresolv.so.2 (0xf7ef6000) libc.so.6 => /lib/libc.so.6 (0xf7cd9000) /lib/ld-linux.so.2 (0xf7f1b000) Affected systems: woodpecker.gentoo.org pigeon.gentoo.org AMD64 works fine, it's only these remaining x86 systems.
node_exporter has similar issues as well. Common pattern: built via portage -> segfault built outside portage -> works node_exporter-1.3.1 w/ goversion="go1.18.3" previously worked (known because of metrics). Tested: ------- node_exporter-1.7.0 (via Rahil's PR) node_exporter-1.5.0 node_exporter-1.4.0 X promu-0.14.0 X go 1.21.4 go 1.21.3 go 1.20.8 All segfaults. Other portage-built golang works, e.g. brand new build of goawk (which can get marked as ~x86 now, I tested it).
It seems that the trouble begins when the Makefile uses promu to rebuild itself. It builds a working, dynamically linked promu first via go install, which installs it to /var/tmp/portage/dev-util/promu-0.15.0/homedir/go/bin/promu. It uses that to build the statically linked /var/tmp/portage/dev-util/promu-0.15.0/work/promu-0.15.0/promu-0.15.0 which segfaults. The promu-0.14.0 ebuild used ego build instead of the Makefile, so it is not affected by this problem. I wonder if there is some previous version that doesn't segfault when built via the Makefile.
I patched the Makefile to add --cgo to the promu build options, and that resulted in a working, statically linked binary that doesn't segfault: --- a/Makefile +++ b/Makefile @@ -18,4 +18,4 @@ @echo ">> installing promu" GO111MODULE=$(GO111MODULE) GOOS= GOARCH= $(GO) install github.com/prometheus/promu @echo ">> rebuilding binaries using promu" - GO111MODULE=$(GO111MODULE) $(PROMU) build --prefix $(PREFIX) + GO111MODULE=$(GO111MODULE) $(PROMU) build --cgo --prefix $(PREFIX)
I'll look into this a bit more and then open an upstream issue if I can reproduce it in a container with a different distro and toolchain. As an alternative to patching the Makefile, this is another way to enable cgo and prevent the segfault issue: --- a/.promu.yml +++ b/.promu.yml @@ -2,6 +2,7 @@ # Whenever the Go version is updated here, # .circle/config.yml should also be updated. version: 1.20 + cgo: true repository: path: github.com/prometheus/promu build:
I was not able to reproduce this issue using the 386 version of the debian:unstable-20231030 docker image and debian's go1.21.4 package.
It's something about GOFLAGS and/or GOMODCACHE variables (or the dependency tarball), since the problem goes away if I unset those variables.
Removing -buildmode=pie from GOFLAGS like this fixed it: --- a/dev-util/promu/promu-0.15.0.ebuild +++ b/dev-util/promu/promu-0.15.0.ebuild @@ -34,6 +34,8 @@ src_unpack() { } src_compile() { + # https://bugs.gentoo.org/917577 + export GOFLAGS=${GOFLAGS//-buildmode=pie} emake build }
zmedico: can we ship that patch for now on x86?
Yeah, how does https://github.com/gentoo/gentoo/pull/33937 look?
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6015a94c17c9d066ba3c14380aec8cb176de7aa5 commit 6015a94c17c9d066ba3c14380aec8cb176de7aa5 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2023-11-22 16:57:39 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2023-11-25 00:59:14 +0000 dev-util/promu: Disable pie for x86 Bug: https://bugs.gentoo.org/917577 Signed-off-by: Zac Medico <zmedico@gentoo.org> dev-util/promu/promu-0.15.0.ebuild | 4 ++++ dev-util/promu/promu-9999.ebuild | 4 ++++ 2 files changed, 8 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9bd2a2d610a131178d15bc55f1c5ef2c7bd63f5f commit 9bd2a2d610a131178d15bc55f1c5ef2c7bd63f5f Author: Robin H. Johnson <robbat2@gentoo.org> AuthorDate: 2023-11-26 21:47:01 +0000 Commit: Robin H. Johnson <robbat2@gentoo.org> CommitDate: 2023-11-26 21:47:36 +0000 app-metrics/node_exporter: workaround bug #917577 for x86 builds Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Bug: https://bugs.gentoo.org/917577 app-metrics/node_exporter/node_exporter-1.7.0.ebuild | 4 ++++ 1 file changed, 4 insertions(+)
zmedico: The GOFLAGS tweak is ALSO needed on node_exporter, so I put it there.
@robbat, if change is needed in per-package basis (like you made in node_exporter) then could you test other packages that uses promu? "qgrep promu" shows me alertmanager, smartctl_exporter, app-metrics/bind_exporter, app-metrics/pushgateway etc uses promu.
Everything that builds a binary with "promu build" in the ebuild seems to blow up. == /var/tmp/portage/app-metrics/blackbox_exporter-0.24.0/image/usr/bin/blackbox_exporter Segmentation fault == /var/tmp/portage/app-metrics/memcached_exporter-0.10.0/image/usr/bin/memcached_exporter Segmentation fault == /var/tmp/portage/app-metrics/uwsgi_exporter-1.1.0/image/usr/bin/uwsgi_exporter Segmentation fault == /var/tmp/portage/app-metrics/bind_exporter-0.6.1/image/usr/bin/bind_exporter Segmentation fault == /var/tmp/portage/app-metrics/alertmanager-0.26.0/image/usr/bin/alertmanager Segmentation fault == /var/tmp/portage/app-metrics/alertmanager-0.26.0/image/usr/bin/amtool Segmentation fault == /var/tmp/portage/app-metrics/mysqld_exporter-0.14.0_p20230328/image/usr/bin/mysqld_exporter Segmentation fault == /var/tmp/portage/app-metrics/consul_exporter-0.7.1/image/usr/bin/consul_exporter Segmentation fault == /var/tmp/portage/app-metrics/pushgateway-1.5.1/image/usr/bin/pushgateway Segmentation fault == /var/tmp/portage/app-metrics/snmp_exporter-0.24.1/image/usr/bin/snmp_exporter Segmentation fault == /var/tmp/portage/app-metrics/prom2json-1.3.0/image/usr/bin/prom2json Segmentation fault
FWIW, I found that a go build -ldflags argument as suggested in https://stackoverflow.com/questions/64019336/go-compile-to-static-binary-with-pie will result in a running pie executable on x86, though the build emits warnings that it "requires at runtime the shared libraries from the glibc version used for linking": > # go build -ldflags '-linkmode external -s -w -extldflags "--static-pie"' -buildmode=pie -tags netgo,static_build > # github.com/prometheus/promu > /usr/bin/ld: /tmp/go-link-1249323874/000002.o: in function `mygetgrouplist': > /_/GOROOT/src/os/user/getgrouplist_unix.go:15:(.text+0x29): warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking > /usr/bin/ld: /tmp/go-link-1249323874/000001.o: in function `mygetgrgid_r': > /_/GOROOT/src/os/user/cgo_lookup_cgo.go:45:(.text+0x5d): warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking > /usr/bin/ld: /tmp/go-link-1249323874/000001.o: in function `mygetgrnam_r': > /_/GOROOT/src/os/user/cgo_lookup_cgo.go:54:(.text+0x11d): warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking > /usr/bin/ld: /tmp/go-link-1249323874/000001.o: in function `mygetpwnam_r': > /_/GOROOT/src/os/user/cgo_lookup_cgo.go:36:(.text+0x1e9): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking > /usr/bin/ld: /tmp/go-link-1249323874/000001.o: in function `mygetpwuid_r': > /_/GOROOT/src/os/user/cgo_lookup_cgo.go:27:(.text+0x2e9): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking > # file ./promu > ./promu: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), static-pie linked, BuildID[sha1]=4ec10f758ebc9658e18aba52616051c270a8b191, for GNU/Linux 3.2.0, stripped Repeating the same build with CGO_ENABLED=0 results in a segfaulting binary like the one that promu creates.
