Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 553918 - dev-perl/IO-AIO-4.310.0 (and 320) with perl-5.22: tests segfault in libsandbox due to small pthread stacks which sandbox blows
Summary: dev-perl/IO-AIO-4.310.0 (and 320) with perl-5.22: tests segfault in libsandbo...
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Sandbox Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-07-04 13:17 UTC by Andreas K. Hüttel
Modified: 2015-12-25 17:20 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas K. Hüttel gentoo-dev 2015-07-04 13:17:39 UTC
>>> Test phase: dev-perl/IO-AIO-4.310.0
make -j5 test TEST_VERBOSE=0 
Running Mkbootstrap for IO::AIO ()
chmod 644 "AIO.bs"
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00_load.t ..... ok   
t/01_stat.t ..... Failed 10/10 subtests 
t/02_read.t ..... Failed 2/2 subtests 
t/03_errors.t ... Failed 12/12 subtests 
t/04_fork.t ..... ok     
t/05_readdir.t .. Failed 11/12 subtests 
t/06_group.t .... ok     
t/07_feeder.t ... ok   

Test Summary Report
-------------------
t/01_stat.t   (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: Bad plan.  You planned 10 tests but ran 0.
t/02_read.t   (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: Bad plan.  You planned 2 tests but ran 0.
t/03_errors.t (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: Bad plan.  You planned 12 tests but ran 0.
t/05_readdir.t (Wstat: 139 Tests: 1 Failed: 0)
  Non-zero wait status: 139
  Parse errors: Bad plan.  You planned 12 tests but ran 1.
Files=8, Tests=32,  2 wallclock secs ( 0.05 usr  0.01 sys +  0.17 cusr  0.04 csys =  0.27 CPU)
Result: FAIL
Failed 4/8 test programs. 0/32 subtests failed.
Makefile:1006: recipe for target 'test_dynamic' failed
make: *** [test_dynamic] Error 255
 * ERROR: dev-perl/IO-AIO-4.310.0::gentoo failed (test phase):
 *   emake failed






(gdb) bt
#0  resolve_path (path=path@entry=0xbc13a0 "AIO.xs", follow_link=0, follow_link@entry=4) at ../../sandbox-2.6/libsandbox/libsandbox.c:182
#1  0x00007f8d10bda984 in check_syscall (flags=0, file=0xbc13a0 "AIO.xs", func=0x7f8d10be273f "open_rd", sb_nr=-3, sbcontext=<optimized out>)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:814
#2  before_syscall (dirfd=<optimized out>, sb_nr=-3, func=0x7f8d10be273f "open_rd", file=0xbc13a0 "AIO.xs", flags=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:983
#3  0x00007f8d10bdff51 in openat64_DEFAULT (dirfd=dirfd@entry=-100, pathname=0xbc13a0 "AIO.xs", flags=0)
    at ../../sandbox-2.6/libsandbox/wrapper-funcs/__wrapper_simple.c:52
#4  0x00007f8d0f35d05e in openat (__oflag=<optimized out>, __path=<optimized out>, __fd=-100) at /usr/include/bits/fcntl2.h:135
#5  eio_execute (req=0xa4aa10, self=0xa80cb0) at libeio/eio.c:1938
#6  etp_proc (thr_arg=<optimized out>) at libeio/eio.c:1813
#7  0x00007f8d1004f2c4 in start_thread (arg=0x7f8d10f8b700) at pthread_create.c:310
#8  0x00007f8d105512dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) 



Tests disabled for now
Comment 1 SpanKY gentoo-dev 2015-09-11 20:54:45 UTC
please show the exact command you ran to get that gdb output.  the perl test runner will re-exec itself via /bin/sh which then runs perl (but with modified env).  it's a bit of a rats nest.
Comment 2 Andreas K. Hüttel gentoo-dev 2015-12-12 12:28:11 UTC
Well the tests dump a couple of core files... so loading these with f.ex.
gdb /usr/bin/perl -core '/corefiles/core.perl!eio.954'
and running backtrace. 

Here is the full set for all 4 core files:
["Datei oder Verzeichnis nicht gefunden" = "File or directory not found"]

1)

huettel@pinacolada ~ $ gdb /usr/bin/perl -core '/corefiles/core.perl!eio.954'
GNU gdb (Gentoo 7.9.1 vanilla) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/perl...Reading symbols from /usr/lib64/debug//usr/bin/perl.debug...done.
done.
[New LWP 982]
[New LWP 954]
                                                                                                                                     
warning: Could not load shared library symbols for linux-vdso.so.1.                                                                  
Do you need "set solib-search-path" or "set sysroot"?                                                                                
[Thread debugging using libthread_db enabled]                                                                                        
Using host libthread_db library "/lib64/libthread_db.so.1".                                                                          
Core was generated by `/usr/bin/perl t/05_readdir.t'.                                                                                
Program terminated with signal SIGSEGV, Segmentation fault.                                                                          
#0  resolve_path (path=path@entry=0x25a83c0 ".", follow_link=follow_link@entry=0) at ../../sandbox-2.6/libsandbox/libsandbox.c:182   
182     ../../sandbox-2.6/libsandbox/libsandbox.c: Datei oder Verzeichnis nicht gefunden.                                            
(gdb)                                                                                                                                
(gdb) bt                                                                                                                             
#0  resolve_path (path=path@entry=0x25a83c0 ".", follow_link=follow_link@entry=0) at ../../sandbox-2.6/libsandbox/libsandbox.c:182
#1  0x00007f05d7b617a2 in check_syscall (sbcontext=0x7f05d7d6e2c0 <sbcontext>, flags=0, file=0x25a83c0 ".", 
    func=0x7f05d7b69183 "opendir", sb_nr=16) at ../../sandbox-2.6/libsandbox/libsandbox.c:814
#2  before_syscall (dirfd=<optimized out>, sb_nr=16, func=0x7f05d7b69183 "opendir", file=0x25a83c0 ".", flags=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:983
#3  0x00007f05d7b62046 in before_syscall (dirfd=dirfd@entry=-100, sb_nr=sb_nr@entry=16, func=func@entry=0x7f05d7b69183 "opendir", 
    file=file@entry=0x25a83c0 ".", flags=flags@entry=0) at ../../sandbox-2.6/libsandbox/libsandbox.c:999
#4  0x00007f05d7b64159 in opendir_DEFAULT (name=0x25a83c0 ".") at ../../sandbox-2.6/libsandbox/wrapper-funcs/__wrapper_simple.c:52
#5  0x00007f05d64d9476 in eio__scandir (self=<optimized out>, req=0x23be160) at libeio/eio.c:1393
#6  eio_execute (req=0x23be160, self=0x259be70) at libeio/eio.c:2060
#7  etp_proc (thr_arg=<optimized out>) at libeio/eio.c:1813
#8  0x00007f05d6fc4384 in start_thread (arg=0x7f05d7f1e700) at pthread_create.c:333
#9  0x00007f05d74c55cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) 

2)

huettel@pinacolada ~ $ gdb /usr/bin/perl -core '/corefiles/core.perl!eio.31670'
GNU gdb (Gentoo 7.9.1 vanilla) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/perl...Reading symbols from /usr/lib64/debug//usr/bin/perl.debug...done.
done.
[New LWP 31729]
[New LWP 31670]

warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/perl t/03_errors.t'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  resolve_path (path=path@entry=0x1265340 "/var/tmp/portage/dev-perl/IO-AIO-4.320.0/temp/c8aHkF4Sbw/some_dir/some_file", 
    follow_link=follow_link@entry=0) at ../../sandbox-2.6/libsandbox/libsandbox.c:182
182     ../../sandbox-2.6/libsandbox/libsandbox.c: Datei oder Verzeichnis nicht gefunden.
(gdb) bt
#0  resolve_path (path=path@entry=0x1265340 "/var/tmp/portage/dev-perl/IO-AIO-4.320.0/temp/c8aHkF4Sbw/some_dir/some_file", 
    follow_link=follow_link@entry=0) at ../../sandbox-2.6/libsandbox/libsandbox.c:182
#1  0x00007fbf848927a2 in check_syscall (sbcontext=0x7fbf84a9f2c0 <sbcontext>, flags=578, 
    file=0x1265340 "/var/tmp/portage/dev-perl/IO-AIO-4.320.0/temp/c8aHkF4Sbw/some_dir/some_file", func=0x7fbf84899a67 "open_wr", 
    sb_nr=-4) at ../../sandbox-2.6/libsandbox/libsandbox.c:814
#2  before_syscall (dirfd=<optimized out>, sb_nr=-4, func=0x7fbf84899a67 "open_wr", 
    file=0x1265340 "/var/tmp/portage/dev-perl/IO-AIO-4.320.0/temp/c8aHkF4Sbw/some_dir/some_file", flags=578)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:983
#3  0x00007fbf84897bc9 in openat64_DEFAULT (dirfd=dirfd@entry=-100, 
    pathname=0x1265340 "/var/tmp/portage/dev-perl/IO-AIO-4.320.0/temp/c8aHkF4Sbw/some_dir/some_file", flags=578)
    at ../../sandbox-2.6/libsandbox/wrapper-funcs/__wrapper_simple.c:52
#4  0x00007fbf82be7e1f in openat (__oflag=<optimized out>, __path=<optimized out>, __fd=-100) at /usr/include/bits/fcntl2.h:135
#5  eio_execute (req=0x1264790, self=0x119a030) at libeio/eio.c:1938
#6  etp_proc (thr_arg=<optimized out>) at libeio/eio.c:1813
#7  0x00007fbf83cf5384 in start_thread (arg=0x7fbf84c4f700) at pthread_create.c:333
#8  0x00007fbf841f65cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) 

3)

huettel@pinacolada ~ $ gdb /usr/bin/perl -core '/corefiles/core.perl!eio.31637'
GNU gdb (Gentoo 7.9.1 vanilla) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/perl...Reading symbols from /usr/lib64/debug//usr/bin/perl.debug...done.
done.
[New LWP 31660]
[New LWP 31637]

warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/perl t/02_read.t'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  resolve_path (path=path@entry=0x1e42270 "AIO.xs", follow_link=follow_link@entry=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:182
182     ../../sandbox-2.6/libsandbox/libsandbox.c: Datei oder Verzeichnis nicht gefunden.
(gdb) bt
#0  resolve_path (path=path@entry=0x1e42270 "AIO.xs", follow_link=follow_link@entry=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:182
#1  0x00007ff2bc2ca7a2 in check_syscall (sbcontext=0x7ff2bc4d72c0 <sbcontext>, flags=0, file=0x1e42270 "AIO.xs", 
    func=0x7ff2bc2d1a6f "open_rd", sb_nr=-3) at ../../sandbox-2.6/libsandbox/libsandbox.c:814
#2  before_syscall (dirfd=<optimized out>, sb_nr=-3, func=0x7ff2bc2d1a6f "open_rd", file=0x1e42270 "AIO.xs", flags=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:983
#3  0x00007ff2bc2cfbc9 in openat64_DEFAULT (dirfd=dirfd@entry=-100, pathname=0x1e42270 "AIO.xs", flags=0)
    at ../../sandbox-2.6/libsandbox/wrapper-funcs/__wrapper_simple.c:52
#4  0x00007ff2baa3ce1f in openat (__oflag=<optimized out>, __path=<optimized out>, __fd=-100) at /usr/include/bits/fcntl2.h:135
#5  eio_execute (req=0x1cca790, self=0x1e6c400) at libeio/eio.c:1938
#6  etp_proc (thr_arg=<optimized out>) at libeio/eio.c:1813
#7  0x00007ff2bb72d384 in start_thread (arg=0x7ff2bc687700) at pthread_create.c:333
#8  0x00007ff2bbc2e5cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) 

4)

huettel@pinacolada ~ $ gdb /usr/bin/perl -core '/corefiles/core.perl!eio.31607'
GNU gdb (Gentoo 7.9.1 vanilla) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/perl...Reading symbols from /usr/lib64/debug//usr/bin/perl.debug...done.
done.
[New LWP 31629]
[New LWP 31607]

warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/perl t/01_stat.t'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  resolve_path (path=path@entry=0xe66220 "AIO.xs", follow_link=follow_link@entry=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:182
182     ../../sandbox-2.6/libsandbox/libsandbox.c: Datei oder Verzeichnis nicht gefunden.
(gdb) bt
#0  resolve_path (path=path@entry=0xe66220 "AIO.xs", follow_link=follow_link@entry=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:182
#1  0x00007f58a65a87a2 in check_syscall (sbcontext=0x7f58a67b52c0 <sbcontext>, flags=0, file=0xe66220 "AIO.xs", 
    func=0x7f58a65afa6f "open_rd", sb_nr=-3) at ../../sandbox-2.6/libsandbox/libsandbox.c:814
#2  before_syscall (dirfd=<optimized out>, sb_nr=-3, func=0x7f58a65afa6f "open_rd", file=0xe66220 "AIO.xs", flags=0)
    at ../../sandbox-2.6/libsandbox/libsandbox.c:983
#3  0x00007f58a65adbc9 in openat64_DEFAULT (dirfd=dirfd@entry=-100, pathname=0xe66220 "AIO.xs", flags=0)
    at ../../sandbox-2.6/libsandbox/wrapper-funcs/__wrapper_simple.c:52
#4  0x00007f58a4d1ae1f in openat (__oflag=<optimized out>, __path=<optimized out>, __fd=-100) at /usr/include/bits/fcntl2.h:135
#5  eio_execute (req=0xe8a100, self=0xe81ca0) at libeio/eio.c:1938
#6  etp_proc (thr_arg=<optimized out>) at libeio/eio.c:1813
#7  0x00007f58a5a0b384 in start_thread (arg=0x7f58a6965700) at pthread_create.c:333
#8  0x00007f58a5f0c5cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb)
Comment 3 SpanKY gentoo-dev 2015-12-23 05:37:04 UTC
this is because the libeio code creates threads with "small" stacks -- it defaults to 32k on 64bit systems and 16k on 32bit systems.  when the sandbox code runs, it often uses buffers on the stack to hold paths that it's resolving, and those tend to be ~1k or so per buffer.  unlike the main thread whose stack can grow dynamically (and "forever"), thread stacks are static ... once you blow those, you're dead.

in this case, sandbox is blowing the stack and the code segfaults.  if i build the IO-AIO module by hand (because perl is buggy -- see bug 569266) using CPPFLAGS like -DX_STACKSIZE=0x10000 (65k), then the tests pass fine inside the sandbox.

i would just leave the tests disabled and add a comment referring to this bug.  i could try to trim down the stack usage a bit, but in the end, sandbox is going to require a good amount of stack space to operate -- moving from stack buffers to malloc is not a silver bullet as now i need to add a lot of malloc/free calls which means a lot of memory thrashing because sandbox itself has to get memory directly from the kernel via mmap (since some apps like to override malloc).

an out-of-process sandbox would address this, and maybe someday we'll get there (via the ptrace logic i already have in place).  but that day isn't soon.
Comment 4 Andreas K. Hüttel gentoo-dev 2015-12-25 17:20:33 UTC
(In reply to SpanKY from comment #3)
> 
> in this case, sandbox is blowing the stack and the code segfaults.  if i
> build the IO-AIO module by hand (because perl is buggy -- see bug 569266)
> using CPPFLAGS like -DX_STACKSIZE=0x10000 (65k), then the tests pass fine
> inside the sandbox.
> 
> i would just leave the tests disabled and add a comment referring to this
> bug.  

Done and thank you for the analysis.