Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 589378

Summary: www-servers/varnish-4.1.3 - build phase stuck on "varnishd -x dumprstparam"
Product: Gentoo Linux Reporter: Ștefan Talpalaru <stefantalpalaru>
Component: Current packagesAssignee: Anthony Basile <blueness>
Status: RESOLVED FIXED    
Severity: normal CC: idl0r, mozilla, sandbox, whissi
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 592420    
Bug Blocks:    
Attachments: varnish-4.1.3_build.log
emerge --info www-servers/varnish

Description Ștefan Talpalaru 2016-07-22 14:47:02 UTC
Created attachment 441370 [details]
varnish-4.1.3_build.log

When trying to build varnish, it gets stuck for ever while running this command:

/var/tmp/portage/www-servers/varnish-4.1.3/work/varnish-4.1.3_build/bin/varnishd/.libs/varnishd -x dumprstparam

which is executed like this: "/bin/sh -c ../../bin/varnishd/varnishd -x dumprstparam > include/params.rst"

The weird thing is that running it manually succeeds quickly.

I'm attaching the build log obtained after killing the varnishd process.
Comment 1 Anthony Basile gentoo-dev 2016-07-22 15:41:34 UTC
Can you get me `emerge --info =www-servers/varnish-4.1.3`  Also, can attach strace -p to see what its doing while it "stuck".  For what its worth, I don't hit this, but I've suspect this isn't "just your system" and that others will hit this too,
Comment 2 Ștefan Talpalaru 2016-07-22 16:17:06 UTC
Created attachment 441372 [details]
emerge --info www-servers/varnish

"emerge -pv varnish":

[ebuild     U  ] www-servers/varnish-4.1.3::gentoo [4.1.2::gentoo] USE="jemalloc -jit -static-libs" PYTHON_TARGETS="python2_7 -pypy -python3_3 -python3_4" 0 KiB

"strace -p...":

futex(0x7fa2506a01c0, FUTEX_WAIT_PRIVATE, 2, NULL
Comment 3 Anthony Basile gentoo-dev 2016-07-24 15:36:07 UTC
(In reply to Stefan Talpalaru from comment #2)
> Created attachment 441372 [details]
> emerge --info www-servers/varnish
> 
> "emerge -pv varnish":
> 
> [ebuild     U  ] www-servers/varnish-4.1.3::gentoo [4.1.2::gentoo]
> USE="jemalloc -jit -static-libs" PYTHON_TARGETS="python2_7 -pypy -python3_3
> -python3_4" 0 KiB
> 
> "strace -p...":
> 
> futex(0x7fa2506a01c0, FUTEX_WAIT_PRIVATE, 2, NULL

This gives me an idea of what's going on, its waiting on a futex lock, but its not enough for me to debug.  It looks like you're running a system which is currently quite a bit from stable so it makes it hard for me to reproduce on a similar system.

I think for now we leave this bug open and see if anyone else hits this and reports it, but as of now, I just don't have enough info.  The only other thing I can thing of is to kill it with SIGINT and get it to dump core (man core if you need details).
Comment 4 Ștefan Talpalaru 2016-07-31 20:40:46 UTC
I narrowed it down to an interaction between jemalloc-4.2.x and sys-apps/sandbox. Somebody with similar strace logs identified the guilty jemalloc commit: https://github.com/jemalloc/jemalloc/issues/430

It all boils down to "/proc/sys/vm/overcommit_memory" access looking like this outside the sandbox:

open("/proc/sys/vm/overcommit_memory", O_RDONLY) = 3
read(3, "0", 1)                         = 1

and like this inside the sandbox:

stat("/proc/sys/vm/overcommit_memory", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
futex(0x7f29c76320c8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7f29c7afb1c0, FUTEX_WAIT_PRIVATE, 2, NULL

More details in a gdb backtrace:

#0  0x00007ffadac6eb6c in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007ffadac68bad in pthread_mutex_lock () from /lib64/libpthread.so.0
#2  0x00007ffadb5fc562 in je_malloc_mutex_lock (tsdn=0x0, mutex=0x7ffadb8371c0 <init_lock>) at include/jemalloc/internal/mutex.h:94
#3  malloc_init_hard () at src/jemalloc.c:1401
#4  0x00007ffadb6002a5 in malloc_init () at src/jemalloc.c:302
#5  ialloc_body (slow_path=true, usize=<synthetic pointer>, tsdn=<synthetic pointer>, zero=true, size=<optimized out>) at src/jemalloc.c:1498
#6  calloc (num=<optimized out>, size=<optimized out>) at src/jemalloc.c:1739
#7  0x00007ffadb178647 in ?? () from /lib64/libdl.so.2
#8  0x00007ffadb17812a in dlsym () from /lib64/libdl.so.2
#9  0x00007ffadbeaded8 in get_dlsym (symname=symname@entry=0x7ffadbeb4c21 "mmap", symver=symver@entry=0x0)
    at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/wrappers.c:51
#10 0x00007ffadbeacdfe in sb_mmap (addr=0x0, prot=3, flags=34, fd=-1, offset=0, length=32)
    at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/memory.c:28
#11 malloc (size=32) at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/memory.c:50
#12 0x00007ffadbeb2d6b in __xmalloc (size=16, 
    file=file@entry=0x7ffadbeb47b0 "/var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/libsandbox.c", 
    func=func@entry=0x7ffadbeb4c00 <__func__.8560> "init_env_entries", line=line@entry=548)
    at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsbutil/sb_memory.c:34
#13 0x00007ffadbeab876 in init_env_entries (warn=1, prefixes_env=0x7ffdceaf3704 "/", env=0x7ffadbeb4b69 "SANDBOX_READ", prefixes_num=<optimized out>, 
    prefixes_array=<optimized out>) at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/libsandbox.c:548
#14 sb_process_env_settings () at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/libsandbox.c:620
#15 0x00007ffadbeaaa46 in libsb_init () at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/libsandbox.c:92
#16 0x00007ffadbeabf05 in before_syscall (dirfd=<optimized out>, sb_nr=-3, func=0x7ffadbeb4b13 "open_rd", 
    file=0x7ffadb62be90 "/proc/sys/vm/overcommit_memory", flags=0)
    at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/libsandbox.c:1059
#17 0x00007ffadbeb094c in open_DEFAULT (pathname=pathname@entry=0x7ffadb62be90 "/proc/sys/vm/overcommit_memory", flags=flags@entry=0)
    at /var/tmp/portage/sys-apps/sandbox-2.10-r2/work/sandbox-2.10/libsandbox/wrapper-funcs/__wrapper_simple.c:52
#18 0x00007ffadb6218cc in open (__oflag=0, __path=0x7ffadb62be90 "/proc/sys/vm/overcommit_memory") at /usr/include/bits/fcntl2.h:53
#19 os_overcommits_proc () at src/pages.c:217
Comment 5 Thomas Deutschmann (RETIRED) gentoo-dev 2016-08-01 11:44:24 UTC
Stefan, thank you for the ping. I am the one who reported this upstream.

I run into the problem while working on app-admin/rsyslog, a package in Gentoo I am proxy-maintaining.

This is very easy to reproduce: Just run "cd /usr/portage/app-admin/rsyslog && USE="jemalloc" ebuild rsyslog-8.19.0.ebuild clean test". It works until you are using >=jemaloc-4.2.0.

And yes, it only happens when running through ebuild/emerge utilities, i.e. sandbox is involved. When I run the same test suite as limited user (su portage --shell /bin/bash && cd /var/tmp/portage/app-admin/rsyslog-8.19.0/work/rsyslog-8.19.0 && make check) everything works.

I tried to add "addpredict", "addread", "addwrite" for /proc/sys/vm/overcommit_memory in src_test() without any change (well, I didn't expect a change because no sandbox violation was reported; it just get stucked).

CC'ing sandbox and jemalloc maintainer.
Comment 6 Thomas Deutschmann (RETIRED) gentoo-dev 2016-11-01 15:27:22 UTC
Should be fixed with >=dev-libs/jemalloc-4.2.1-r1. Please re-test!

> commit 7565f43fc1c544625d153f510b950f9f9bf6e848
> Author: Thomas Deutschmann
> Date:   Tue Nov 1 16:22:38 2016 +0100
> 
>     dev-libs/jemalloc: Patch added to fix an issue with sys-apps/sandbox
> 
>     Cherry picked commit c443b67561 [Link 1] which fixes an issue which
>     prevented any jemalloc-enabled application from running through
>     sys-apps/sandbox (typical use case is emerge with FEATURES=sandbox and
>     FEATURES=test).
> 
>     Cherry picked commit 3c8c3e9e9b [Link 2] which fixes a leaked file
>     descriptor.
> 
>     Link 1: https://github.com/jemalloc/jemalloc/commit/c443b67561891ae68d688daf5f8ce37820cdba2b
>     Link 2: https://github.com/jemalloc/jemalloc/commit/3c8c3e9e9b59b6e34a222816a05f0a01a68919b3
>     Gentoo-Bug: https://bugs.gentoo.org/592420
> 
>     Package-Manager: portage-2.3.2
Comment 7 Ștefan Talpalaru 2016-11-01 18:39:51 UTC
Yes, jemalloc-4.2.1-r1 fixes the problem and varnish builds fine with it.
Comment 8 SpanKY gentoo-dev 2016-11-15 04:44:18 UTC
you could try sandbox-2.11 ... it has some changes to try and better support custom allocators like jemalloc
Comment 9 Ștefan Talpalaru 2016-11-15 16:34:15 UTC
No, an unpatched jemalloc-4.2.1 doesn't work inside sandbox-2.11-r2.