Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 590422 - dev-lang/python-2.7.12 attempts to map memory with PROT_WRITE|PROT_EXEC when creating new threads
Summary: dev-lang/python-2.7.12 attempts to map memory with PROT_WRITE|PROT_EXEC when ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: The Gentoo Linux Hardened Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-03 21:16 UTC by mvaenskae
Modified: 2016-11-30 11:13 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info '=www-client/chromium-53.0.2785.34::gentoo' (emerge-info,5.29 KB, text/plain)
2016-08-03 21:16 UTC, mvaenskae
Details
emerge -pqv '=www-client/chromium-53.0.2785.34::gentoo' (emerge-pqv,502 bytes, text/plain)
2016-08-03 21:17 UTC, mvaenskae
Details
build.log (build.log,5.40 KB, text/x-log)
2016-08-03 21:17 UTC, mvaenskae
Details
environment (environment,201.07 KB, text/plain)
2016-08-03 21:18 UTC, mvaenskae
Details
strace.log (strace.log,64.44 KB, text/x-log)
2016-08-03 21:43 UTC, mvaenskae
Details
strace of pthread-test (pthread-test-strace.log,3.34 KB, text/x-log)
2016-08-04 19:13 UTC, mvaenskae
Details
python2.7.12 build.log (python2.7-build.log.xz,57.18 KB, application/x-xz)
2016-08-07 22:13 UTC, mvaenskae
Details
python3.4.5 build.log (python3.4-build.log.xz,70.52 KB, application/x-xz)
2016-08-07 22:14 UTC, mvaenskae
Details
faulty python binary (python3.4m-wrong,9.79 KB, application/octet-stream)
2016-08-14 21:17 UTC, Attila Tóth
Details
correct python binary (python3.4m-right,9.79 KB, application/octet-stream)
2016-08-14 21:18 UTC, Attila Tóth
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mvaenskae 2016-08-03 21:16:18 UTC
Cheers,

I am posting this issue to better debug my system issues.

The setup is a hardened Gentoo desktop.

Reproducible:
1. Get current portage tree
2. Follow update recommendation of portage and let it emerge chromium
3. Wait for python to die during the configure phase

Actual Result:
Python fails during the configure phase of the package.

Expected Result:
New chromium version installed.
Comment 1 mvaenskae 2016-08-03 21:16:50 UTC
Created attachment 442434 [details]
emerge --info '=www-client/chromium-53.0.2785.34::gentoo'
Comment 2 mvaenskae 2016-08-03 21:17:26 UTC
Created attachment 442436 [details]
emerge -pqv '=www-client/chromium-53.0.2785.34::gentoo'
Comment 3 mvaenskae 2016-08-03 21:17:56 UTC
Created attachment 442438 [details]
build.log
Comment 4 mvaenskae 2016-08-03 21:18:14 UTC
Created attachment 442440 [details]
environment
Comment 5 Mike Gilbert gentoo-dev 2016-08-03 21:22:58 UTC
The chromium build system does not support python3, and your build log says python2.7.

It seems to be something related to broken threading support isolated to your system.
Comment 6 mvaenskae 2016-08-03 21:27:31 UTC
Good, that helps already considerably then to find this issue. Any idea where to look forward? Anything required will be uploaded asap.
Comment 7 Mike Gilbert gentoo-dev 2016-08-03 21:34:58 UTC
Maybe check dmesg for errors.

Also, try running the following test. Does it raise an error?

python2.7 -c "import threading; threading.Thread().start()"

If so, please repeat the test with strace and upload the log.

strace -o strace.log python2.7 -c "import threading; threading.Thread().start()"
Comment 8 mvaenskae 2016-08-03 21:43:26 UTC
Created attachment 442442 [details]
strace.log

There is an issue. Per request, upload of the strace.
Comment 9 wbrana 2016-08-03 23:12:44 UTC
python needs MPROTECT disabled
Comment 10 mvaenskae 2016-08-04 00:22:05 UTC
(In reply to wbrana from comment #9)
> python needs MPROTECT disabled

So this is an ebuild issue on dev-lang/python not receiving proper XATTR_PAX and PT_PAX flags?
Comment 11 Mike Gilbert gentoo-dev 2016-08-04 00:35:54 UTC
Re-assigning to hardened.
Comment 12 Mike Gilbert gentoo-dev 2016-08-04 00:42:46 UTC
commit b8cb87b5af9ab23e60864a4c66988c0f595f5b77
Author: Mike Gilbert <floppym@gentoo.org>
Date:   Wed Aug 3 20:41:29 2016 -0400

    dev-lang/python: disable MPROTECT
    
    The hardened team can figure this shit out.
    
    Package-Manager: portage-2.3.0_p16

 dev-lang/python/python-2.7.12.ebuild | 7 +------
 dev-lang/python/python-3.4.5.ebuild  | 7 +------
 dev-lang/python/python-3.5.2.ebuild  | 7 +------
 3 files changed, 3 insertions(+), 18 deletions(-)
Comment 13 mvaenskae 2016-08-04 00:49:24 UTC
Just so you all know. I did disallow MPROTECT on /usr/bin/python2.7 with the following command: `paxctl-ng -m /usr/bin/python2.7`

`paxctl-ng -v /usr/bin/python2.7` returns:
/usr/bin/python2.7:
        PT_PAX    : -Em--
        XATTR_PAX : -Em--

Executing `python2.7 -c "import threading; threading.Thread().start()"` returns me nothing when -m is set.

Prior to setting -m paxctl-ng returned:
/usr/bin/python2.7:
        PT_PAX    : -E---
        XATTR_PAX : -E---

I do believe for this to be an issue with XATTR/PT flags.
Out of interest, in what state is MPROTECT in this case? And why does EMUTRAP need to be set explicitly?
Comment 14 Mike Gilbert gentoo-dev 2016-08-04 04:23:25 UTC
I reverted my knee-jerk commit.

Please figure out what is going on here to prevent further confusing bug reports.
Comment 15 Mike Gilbert gentoo-dev 2016-08-04 15:21:33 UTC
Can you compile this test program and see if it exhibits the same behavior as python2.7?

http://dev.gentoo.org/~floppym/pthread-test.c

Compile it like this:

gcc -pthread -o pthread-test pthread-test.c

Run it like this:

 % strace -o strace.log ./pthread-test
140010949154560: in new thread
32482: created new thread with id 140010949154560
Comment 16 mvaenskae 2016-08-04 19:13:23 UTC
Created attachment 442540 [details]
strace of pthread-test

strace -o strace.log ./pthread-test
3764035127040: in new thread
4891: created new thread with id 3764035127040
Comment 17 Mike Gilbert gentoo-dev 2016-08-04 20:58:09 UTC
Thanks. That tells me the issue is somehow specific to dev-lang/python.
Comment 18 mvaenskae 2016-08-04 21:09:23 UTC
(In reply to Mike Gilbert from comment #17)
> Thanks. That tells me the issue is somehow specific to dev-lang/python.

As I already mentioned, would you recommend me to do a rebuild of all my python-programs via `emerge -av1 $(qlist -ISC dev-python)`? This obviously leaves out some programs like livestreamer but would be a start.
Comment 19 Mike Gilbert gentoo-dev 2016-08-04 21:47:48 UTC
(In reply to mvaenskae from comment #18)

It can't hurt, but I'm not confident it would fix this. It's basically a random guess.
Comment 20 mvaenskae 2016-08-07 22:09:37 UTC
An interesting obervation I made.

Running `ldd /usr/bin/python3.4`, `ldd /usr/bin/python3.4m`, and `ldd /usr/bin/python2.7` returns me always the following:
    not a dynamic executable

Running the following scanelf commands returns me something different though.

scanelf -a /usr/bin/python2.7
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN ---xE- 0755 LE --- R-- RW-    -      -   NOW /usr/bin/python2.7

scanelf -a /usr/bin/python3.4
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN ---xE- 0755 LE --- R-- RW-    -      -   NOW /usr/bin/python3.4

scanelf -a /usr/bin/python3.4m
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN ---xE- 0755 LE --- R-- RW-    -      -   NOW /usr/bin/python3.4m

Further, I for now disabled mprotect on python2.7 and python3.4 (XATTR, I don't use PT) just so you guys know what behaviour to expect.
Comment 21 mvaenskae 2016-08-07 22:13:52 UTC
Created attachment 442738 [details]
python2.7.12 build.log

I took the liberty after now deducing python not calling libpthread.so and my python not being a dynamic library to upload my python2.7 build.log.
Comment 22 mvaenskae 2016-08-07 22:14:51 UTC
Created attachment 442740 [details]
python3.4.5 build.log

I took the liberty after now deducing python not calling libpthread.so and my python not being a dynamic library to upload my python3.4 build.log.
Comment 23 mvaenskae 2016-08-07 22:58:14 UTC
Alright, this is very unexpected and seems to have solved the python problems.

I had in my make.conf: PAX_MARKINGS="XT PT"

Changing this to PAX_MARKINGS="XT" resulted in the following.

ldd python2.7
        linux-vdso.so.1 (0x00000358f4a6b000)
        libpython2.7.so.1.0 => /usr/lib64/libpython2.7.so.1.0 (0x00000358f4443000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000358f4226000)
        libc.so.6 => /lib64/libc.so.6 (0x00000358f3e83000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00000358f3c7f000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00000358f3a7c000)
        libm.so.6 => /lib64/libm.so.6 (0x00000358f377f000)
        /lib64/ld-linux-x86-64.so.2 (0x00000358f4849000)

ldd python3.4
        linux-vdso.so.1 (0x0000039d22d61000)
        libpython3.4m.so.1.0 => /usr/lib64/libpython3.4m.so.1.0 (0x0000039d22681000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000039d22464000)
        libc.so.6 => /lib64/libc.so.6 (0x0000039d220c1000)
        libdl.so.2 => /lib64/libdl.so.2 (0x0000039d21ebd000)
        libutil.so.1 => /lib64/libutil.so.1 (0x0000039d21cba000)
        libm.so.6 => /lib64/libm.so.6 (0x0000039d219bd000)
        /lib64/ld-linux-x86-64.so.2 (0x0000039d22b3f000)

ldd python3.4m
        linux-vdso.so.1 (0x0000033349ee7000)
        libpython3.4m.so.1.0 => /usr/lib64/libpython3.4m.so.1.0 (0x0000033349807000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000333495ea000)
        libc.so.6 => /lib64/libc.so.6 (0x0000033349247000)
        libdl.so.2 => /lib64/libdl.so.2 (0x0000033349043000)
        libutil.so.1 => /lib64/libutil.so.1 (0x0000033348e40000)
        libm.so.6 => /lib64/libm.so.6 (0x0000033348b43000)
        /lib64/ld-linux-x86-64.so.2 (0x0000033349cc5000)

scanelf -a python2.7
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN PeMRxS 0755 LE RW- R-- RW-    -      -   NOW python2.7

scanelf -a python3.4
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN PeMRxS 0755 LE RW- R-- RW-    -      -   NOW python3.4

scanelf -a python3.4m
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN PeMRxS 0755 LE RW- R-- RW-    -      -   NOW python3.4m

Further, I can now thread properly and the test code per comment #7 runs without errors!

I consider the fix therefore to explicitly check for the PT_PAX_MARKINGS and adjust the build system to that accordingly or remove MPROTECT support in that case.

Special thanks to Zorry from the #gentoo-hardened channel for helping me figuring it this out and teaching me about gentoo's ebuild command.
Otherwise thanks also to all you guys who helped me out on this issue and finding out the culprits and showing me where I should look in the future.
I hope to become more active in the future and help you with squashing more bugs as my thanks to you, but for that I need more of my own bugs to train on :)
Comment 24 Anthony Basile gentoo-dev 2016-08-08 01:37:59 UTC
(In reply to mvaenskae from comment #23)
> Alright, this is very unexpected and seems to have solved the python
> problems.
> 
> I had in my make.conf: PAX_MARKINGS="XT PT"
> 
> Changing this to PAX_MARKINGS="XT" resulted in the following.

I'm just coming to this bug, so help me understand, if you have PAX_MARKINGS="XT PT" in make.conf and emerge python2.7, then threading fails, but if you switch to just "XT" and repeat, then it works?
Comment 25 mvaenskae 2016-08-08 02:12:00 UTC
(In reply to Anthony Basile from comment #24)
> (In reply to mvaenskae from comment #23)
> > Alright, this is very unexpected and seems to have solved the python
> > problems.
> > 
> > I had in my make.conf: PAX_MARKINGS="XT PT"
> > 
> > Changing this to PAX_MARKINGS="XT" resulted in the following.
> 
> I'm just coming to this bug, so help me understand, if you have
> PAX_MARKINGS="XT PT" in make.conf and emerge python2.7, then threading
> fails, but if you switch to just "XT" and repeat, then it works?

Yes, my output of ldd on comment #20 was when i had PT XT set and where I had issues with threads in python.

In comment #23 I then switched to have only XT and everything seems to be working fine. I just have the curious scenario now where i cannot see python actually calling libpthreads.so on my hardened machine (The one where I posted all the logs from and which had this python issue), so I am uncertain what actually all changed. I could build python again and send you the resulting binaries with PT on and off respectivly.
Comment 26 Anthony Basile gentoo-dev 2016-08-08 02:51:02 UTC
(In reply to mvaenskae from comment #25)
 so I am uncertain
> what actually all changed. I could build python again and send you the
> resulting binaries with PT on and off respectivly.

I don't understand how you can get this result, so yes, try reproducing and let's see what's different in the resulting binaries.
Comment 27 mvaenskae 2016-08-08 02:54:56 UTC
(In reply to Anthony Basile from comment #26)
> (In reply to mvaenskae from comment #25)
>  so I am uncertain
> > what actually all changed. I could build python again and send you the
> > resulting binaries with PT on and off respectivly.
> 
> I don't understand how you can get this result, so yes, try reproducing and
> let's see what's different in the resulting binaries.

How can I find out the difference in the binaries? I don't know how to represent the diffs of two binaries in a readable format.
If you give me instructions then I will get you the respective results.
Comment 28 Attila Tóth 2016-08-11 15:35:06 UTC
I got into the same situation I didn't understand until some comment from PAX Team helped me.

There were some changes in the toolchain recently. It turns out at some point the system generated binaries without GNU_STACK header. But only those binaries were affected, which got a PAX marking during emerge. Like python itself, some llvm binaries and webkit-gtk binaries for e.g.
Since the new keyworded toolchain no longer generates PT_PAX header into ELF binaries, I finally disabled the support in the hardened kernels I use and altered PAX_MARKINGS accordingly (XT). Now my toolchain generates binaries with GNU_STACK header even for those gets marked during emerge and they behave as expected.

This is an output of a wrong python executable:
scanelf -a /usr/bin/python3.4m
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN ---xE- 0755 LE --- R-- RW-    -      -   NOW /usr/bin/python3.4m

I bet "readelf -a /usr/bin/python3.4m | grep STACK" would give an empty output.

This is an output of a correct executable:
scanelf -a python3.4m
 TYPE    PAX   PERM ENDIAN STK/REL/PTL TEXTREL RPATH BIND FILE
ET_DYN PeMRxS 0755 LE RW- R-- RW-    -      -   NOW python3.4m

I bet this binary has the required GNU_STACK header!

So I think the problem here is that in some combination of PAX marking settings, those binaries are about to get marked end up without a GNU_STACK header.

Also pypax doesn't support python3.5 currently, but I'm not sure if it's relevant here.
Comment 29 Attila Tóth 2016-08-11 15:38:49 UTC
Just like: https://bugs.gentoo.org/show_bug.cgi?id=589828
Comment 30 Anthony Basile gentoo-dev 2016-08-13 18:13:32 UTC
(In reply to Attila Tóth from comment #28)
> I got into the same situation I didn't understand until some comment from
> PAX Team helped me.
> 
> There were some changes in the toolchain recently. It turns out at some
> point the system generated binaries without GNU_STACK header. But only those
> binaries were affected, which got a PAX marking during emerge. Like python
> itself, some llvm binaries and webkit-gtk binaries for e.g.
> Since the new keyworded toolchain no longer generates PT_PAX header into ELF
> binaries, I finally disabled the support in the hardened kernels I use and
> altered PAX_MARKINGS accordingly (XT). Now my toolchain generates binaries
> with GNU_STACK header even for those gets marked during emerge and they
> behave as expected.

I haven't this this on any of my systems, so I don't know the upgrade path by which you hit the issue.  Can you tell me what you had in your kernel config and PAX_MARKINGS during the toolchain change.  Also am I right that all you have to do to fix the problem is turn off PT_PAX_FLAGS in the kernel and set PAX_MARKINGS="XT", or do you also have to rebuild @world?
Comment 31 Attila Tóth 2016-08-14 14:52:30 UTC
(In reply to Anthony Basile from comment #30)
> I haven't this this on any of my systems, so I don't know the upgrade path
> by which you hit the issue.  Can you tell me what you had in your kernel
> config and PAX_MARKINGS during the toolchain change.  Also am I right that
> all you have to do to fix the problem is turn off PT_PAX_FLAGS in the kernel
> and set PAX_MARKINGS="XT", or do you also have to rebuild @world?

I'm running Gentoo Hardened on a laptop and a server. Both systems showed the same symptoms. The configuration was/is pretty much the same - albeit some obvious hardware differences. Before the symptoms I had PT_PAX enabled in the kernel and PAX_MARKINGS was set to "PT XT". Note, that in this way the ebuild system produced binaries without any XATTR_PAX markings, while PT_PAX was set to "-e---". There were some strange behavior I saw previously in a valid PT_PAX-with-missing-XATTR combination, so I had a simple script issuing "paxctl-ng -F" on each ELF file missing an XATTR_PAX field. The systems were fine.
After I've noticed the symptoms, I wasn't sure about the exact time the problems started. There was a binutils upgrade from 2.25.1 to 2.26.1. There were some signs things changed regarding PT_PAX. So I decided to disable PT_PAX in kernel and remove PT from PAX_MARKINGS. I had problems with python, mpv and another package. Finally turned out the current python binaries on my system doesn't have any GNU_STACK header. Moreover: only those binaries were in lack of GNU_STACK, which were marked during the ebuild (there aren't so many nowdays: python, some binaries in llvm, webkit-gtk, firefox, thunderbird and some others). At this stage I also considered another cause: pypax doesn't support python 3.5. But at this point I could no longer reproduce the problem. I tried to ebuild python eselecting different version of binutils and python. I haven't re-enabled PT_PAX support in the kernel. These machines are in production. Nowdays I do the upgrades on a weekly basis and I didn't have enough motivation to go back and reboot again - instead of looking forward.
I don't keen on the idea of enabling EMUTRAMP by default system-wide. So now I altered my simple script to set missing XATTR_PAX fields explicitly to "-e---".
Comment 32 Attila Tóth 2016-08-14 15:04:18 UTC
(In reply to Anthony Basile from comment #30)
> I haven't this this on any of my systems, so I don't know the upgrade path
> by which you hit the issue.  Can you tell me what you had in your kernel
> config and PAX_MARKINGS during the toolchain change.  Also am I right that
> all you have to do to fix the problem is turn off PT_PAX_FLAGS in the kernel
> and set PAX_MARKINGS="XT", or do you also have to rebuild @world?

I didn't have to rebuild world. I only re-emerged those packages involved on my system (python*, llvm and webkit-gtk). Although I could not identify any symptoms related to llvm or webkit-gtk marked binaries missing GNU_STACK header - maybe because those were marked with "-m" to disable MPROTECT. In case of python marking the incorrect binary with "-m" workarounds the problem. So basically after re-emerging python everything was fine. I was re-emerged llvm and webkit-gtk just to be sure.
I think disabling PT_PAX in kernel and setting PAX_MARKINGS to XT only should solve the problem of some users having symptoms like mine. Only a couple of packages must be re-emerged. scanelf and readelf are two candidates to confirm the diagnosis.
Comment 33 Attila Tóth 2016-08-14 15:15:30 UTC
(In reply to Anthony Basile from comment #30)
> I haven't this this on any of my systems, so I don't know the upgrade path
> by which you hit the issue.  Can you tell me what you had in your kernel
> config and PAX_MARKINGS during the toolchain change.  Also am I right that
> all you have to do to fix the problem is turn off PT_PAX_FLAGS in the kernel
> and set PAX_MARKINGS="XT", or do you also have to rebuild @world?

I don't know if there's a chance any non-hardened systems could be affected. I don't remember the last time I've booted a vanilla kernel.
Comment 34 Anthony Basile gentoo-dev 2016-08-14 15:41:54 UTC
(In reply to mvaenskae from comment #27)
> (In reply to Anthony Basile from comment #26)
> > (In reply to mvaenskae from comment #25)
> >  so I am uncertain
> > > what actually all changed. I could build python again and send you the
> > > resulting binaries with PT on and off respectivly.
> > 
> > I don't understand how you can get this result, so yes, try reproducing and
> > let's see what's different in the resulting binaries.
> 
> How can I find out the difference in the binaries? I don't know how to
> represent the diffs of two binaries in a readable format.
> If you give me instructions then I will get you the respective results.

Can you compress and attach python3.4 build with "XT PT" and with just "XT" and I'll look at them at my end.
Comment 35 Attila Tóth 2016-08-14 21:15:51 UTC
(In reply to Anthony Basile from comment #34)
> (In reply to mvaenskae from comment #27)
> > (In reply to Anthony Basile from comment #26)
> > > (In reply to mvaenskae from comment #25)
> > >  so I am uncertain
> > > > what actually all changed. I could build python again and send you the
> > > > resulting binaries with PT on and off respectivly.
> > > 
> > > I don't understand how you can get this result, so yes, try reproducing and
> > > let's see what's different in the resulting binaries.
> > 
> > How can I find out the difference in the binaries? I don't know how to
> > represent the diffs of two binaries in a readable format.
> > If you give me instructions then I will get you the respective results.
> 
> Can you compress and attach python3.4 build with "XT PT" and with just "XT"
> and I'll look at them at my end.

I could build a faulty python binary with PAX_MARKINGS="PT XT". I attach a faulty and a correct binary.
Comment 36 Attila Tóth 2016-08-14 21:17:31 UTC
Created attachment 443324 [details]
faulty python binary

PAX_MARKINGS="PT XT"

# paxctl-ng -v /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m
/var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m:
        PT_PAX    : -E---
        XATTR_PAX : -E---
# scanelf -e /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m
 TYPE   STK/REL/PTL FILE
ET_DYN --- R-- RW- /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m
# readelf -a /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m | grep STACK
#
Comment 37 Attila Tóth 2016-08-14 21:18:32 UTC
Created attachment 443326 [details]
correct python binary

PAX_MARKINGS="XT"

# paxctl-ng -v /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m
/var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m:
        PT_PAX    : not found
        XATTR_PAX : -E---
# scanelf -e /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m
 TYPE   STK/REL/PTL FILE
ET_DYN RW- R-- RW- /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m
# readelf -a /var/tmp/portage/dev-lang/python-3.4.5/image/usr/bin/python3.4m | grep STACK
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
#
Comment 38 Anthony Basile gentoo-dev 2016-08-25 11:13:41 UTC
(In reply to mvaenskae from comment #3)
(In reply to Attila Tóth from comment #37)

Can you guys confirm whether you had the following combination when you triggered the problem:

1. binutils-2.26.1 installed
2. sys-apps/paxctl is installed
3. PAX_MARKINGS="PT XT"

What I think is happening is that buidling without the PT_PAX patch for binutils, paxctl munges up the GNU_STACK header and leaves behind a broken elf object.
Comment 39 Attila Tóth 2016-08-26 10:38:25 UTC
(In reply to Anthony Basile from comment #38)
> (In reply to mvaenskae from comment #3)
> (In reply to Attila Tóth from comment #37)
> 
> Can you guys confirm whether you had the following combination when you
> triggered the problem:
> 
> 1. binutils-2.26.1 installed
> 2. sys-apps/paxctl is installed
> 3. PAX_MARKINGS="PT XT"
> 
> What I think is happening is that buidling without the PT_PAX patch for
> binutils, paxctl munges up the GNU_STACK header and leaves behind a broken
> elf object.

Dear Blueness,

I confirm, that I had the combo you've listed in my case.
I noticed myself, that the latest binutils no longer generates PT_PAX headers - as it was predicted before this will happen sometimes. I thought it was intentional. Things got screwed up only for those binaries needed marking. After removing PT from PAX_MARKINGS things got straight.

If you open up the binaries attached and compare the correct to the faulty with some bindiff util (like vbindiff), it'll show, that they differ just a little bit. At 0000 01C8 the correct binary has: 51 E5 74 64 06 00, while the wrong binary has: 80 15 04 65 00 18. I opt for your version, that paxctl can be the culprit factor here in case there's no PT_PAX header - although I haven't performed any specific tests to make sure.

BR: Dw.
Comment 40 Anthony Basile gentoo-dev 2016-08-26 23:10:12 UTC
(In reply to Attila Tóth from comment #39)
> 
> I confirm,

okay this is paxctl trying to steal the GNU_STACK program header and leaving the ELF in a bad state.  I've never liked that in paxctl and we I'm going to remove that functionality from the eclass.  The patch will be:

diff --git a/eclass/pax-utils.eclass b/eclass/pax-utils.eclass
index 9ed1170..7468216 100644
--- a/eclass/pax-utils.eclass
+++ b/eclass/pax-utils.eclass
@@ -82,11 +82,6 @@ pax-mark() {
                                einfo "PT_PAX marking -${flags} ${f} with paxctl"
                                # First, try modifying the existing PAX_FLAGS header.
                                paxctl -q${flags} "${f}" >/dev/null 2>&1 && continue
-                               # Second, try creating a PT_PAX header (works on ET_EXEC).
-                               # Even though this is less safe, most exes need it. #463170
-                               paxctl -qC${flags} "${f}" >/dev/null 2>&1 && continue
-                               # Third, try stealing the (unused under PaX) PT_GNU_STACK header
-                               paxctl -qc${flags} "${f}" >/dev/null 2>&1 && continue
                        fi
 
                        # Next try paxctl-ng -> this will not create/convert any program headers.

I'll send a notice to the gentoo-dev@g.o list and commit in a few days.
Comment 41 Attila Tóth 2016-08-27 10:18:00 UTC
(In reply to Anthony Basile from comment #40)
> (In reply to Attila Tóth from comment #39)
> > 
> > I confirm,
> 
> okay this is paxctl trying to steal the GNU_STACK program header and leaving
> the ELF in a bad state.  I've never liked that in paxctl and we I'm going to
> remove that functionality from the eclass.  The patch will be:

>  
>                         # Next try paxctl-ng -> this will not create/convert
> any program headers.
> 
> I'll send a notice to the gentoo-dev@g.o list and commit in a few days.

I also opt for paxctl-ng just like it is written in the eclass comment.
Comment 42 Anthony Basile gentoo-dev 2016-08-28 05:36:08 UTC
i've pushed this.  please test and reopen this bug if this is still a problem.
Comment 43 Tony Vroon (RETIRED) gentoo-dev 2016-11-09 09:31:37 UTC
This has broken dev-java/oracle-jre-bin-1.8.0.112; PaX flags do not set:
Nov  9 09:19:28 bob kernel: [1029629.796235] grsec: From x.x.x.x: denied RWX mmap of <anonymous mapping> by /var/tmp/portage/dev-java/oracle-jre-bin-1.8.0.112/image/opt/oracle-jre-bin-1.8.0.112/bin/java[java:30653] uid/euid:0/0 gid/egid:0/0, parent /usr/lib64/portage/python2.7/ebuild.sh[ebuild.sh:30376] uid/euid:0/0 gid/egid:0/0

Undoing this exact change (the two commented out lines marked with this bug number) allows the ebuild to install normally. Considering this a regression caused by the commit, and thus reopening the bug.
Let me know if you disagree, in which case I will file a new bug instead.
Comment 44 Attila Tóth 2016-11-09 12:35:27 UTC
(In reply to Tony Vroon from comment #43)
> This has broken dev-java/oracle-jre-bin-1.8.0.112; PaX flags do not set:
> Nov  9 09:19:28 bob kernel: [1029629.796235] grsec: From x.x.x.x: denied RWX
> mmap of <anonymous mapping> by
> /var/tmp/portage/dev-java/oracle-jre-bin-1.8.0.112/image/opt/oracle-jre-bin-
> 1.8.0.112/bin/java[java:30653] uid/euid:0/0 gid/egid:0/0, parent
> /usr/lib64/portage/python2.7/ebuild.sh[ebuild.sh:30376] uid/euid:0/0
> gid/egid:0/0
> 
> Undoing this exact change (the two commented out lines marked with this bug
> number) allows the ebuild to install normally. Considering this a regression
> caused by the commit, and thus reopening the bug.
> Let me know if you disagree, in which case I will file a new bug instead.

According to my interpretation the base problem was caused by the build systems creating erroneus headers (screwing up) during the build process in some combinations of PAX_MARKINGS and kernel PAX marking related configuration settings.
Do you have such a python executable?
What is the output of "paxctl-ng -v <binary>", "scanelf -e <binary>" and "readelf -a <binary> | grep STACK" output of your python2.7 executable?
In the mean time it is important to know your PAX_MARKINGS in your make.conf and how you configured your kernel regarding PAX markings?
Since oracle-jre-bin requires pax markings to be able to work, those markings should be also checked - whether the intended flags are actually applied or not. I don't see how those eclass changes may affect oracle-jre-bin executable markings, but it may happen.
I'm sorry if all this is trivial to you and you've already checked these.
Comment 45 Tony Vroon (RETIRED) gentoo-dev 2016-11-29 14:47:48 UTC
(In reply to Attila Tóth from comment #44)
> What is the output of "paxctl-ng -v <binary>", "scanelf -e <binary>" and
> "readelf -a <binary> | grep STACK"

bob ~ # paxctl-ng -v /usr/bin/python2.7
/usr/bin/python2.7:
	PT_PAX    : -E---
	XATTR_PAX : -E---

bob ~ # scanelf -e /usr/bin/python2.7
 TYPE   STK/REL/PTL FILE 
ET_DYN RW- R-- RW- /usr/bin/python2.7 

bob ~ # readelf -a /usr/bin/python2.7 | grep STACK
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000

> In the mean time it is important to know your PAX_MARKINGS in your make.conf

bob ~ # grep PAX /etc/make.conf
bob ~ # 

> and how you configured your kernel regarding PAX markings?

bob ~ # zgrep -i pax /proc/config.gz | grep -i flags
CONFIG_PAX_PT_PAX_FLAGS=y
# CONFIG_PAX_XATTR_PAX_FLAGS is not set
# CONFIG_PAX_NO_ACL_FLAGS is not set
CONFIG_PAX_HAVE_ACL_FLAGS=y
# CONFIG_PAX_HOOK_ACL_FLAGS is not set

> Since oracle-jre-bin requires pax markings to be able to work, those
> markings should be also checked - whether the intended flags are actually
> applied or not. I don't see how those eclass changes may affect
> oracle-jre-bin executable markings, but it may happen.

They seem to, as I said, undoing the change allows it to build.
Comment 46 Attila Tóth 2016-11-30 10:20:15 UTC
(In reply to Tony Vroon from comment #45)
> (In reply to Attila Tóth from comment #44)
> > What is the output of "paxctl-ng -v <binary>", "scanelf -e <binary>" and
> > "readelf -a <binary> | grep STACK"
> 
> bob ~ # paxctl-ng -v /usr/bin/python2.7
> /usr/bin/python2.7:
> 	PT_PAX    : -E---
> 	XATTR_PAX : -E---
> 
> bob ~ # scanelf -e /usr/bin/python2.7
>  TYPE   STK/REL/PTL FILE 
> ET_DYN RW- R-- RW- /usr/bin/python2.7 
> 
> bob ~ # readelf -a /usr/bin/python2.7 | grep STACK
>   GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
> 
> > In the mean time it is important to know your PAX_MARKINGS in your make.conf
> 
> bob ~ # grep PAX /etc/make.conf
> bob ~ # 
> 
> > and how you configured your kernel regarding PAX markings?
> 
> bob ~ # zgrep -i pax /proc/config.gz | grep -i flags
> CONFIG_PAX_PT_PAX_FLAGS=y
> # CONFIG_PAX_XATTR_PAX_FLAGS is not set
> # CONFIG_PAX_NO_ACL_FLAGS is not set
> CONFIG_PAX_HAVE_ACL_FLAGS=y
> # CONFIG_PAX_HOOK_ACL_FLAGS is not set
> 
> > Since oracle-jre-bin requires pax markings to be able to work, those
> > markings should be also checked - whether the intended flags are actually
> > applied or not. I don't see how those eclass changes may affect
> > oracle-jre-bin executable markings, but it may happen.
> 
> They seem to, as I said, undoing the change allows it to build.

Your python binary seems to be valid. However you keep both PT_PAX and XATTR_PAX flags mechanism enabled.
On my setup I could solve the issues I've faced by disabling the PT_PAX mechanism and moving to XATTR_PAX only. The mixture of the two seem to cause problems. Theoretically PT-XATTR combo should work, but it was always glitchy in my experience. In the mean time I suggest adding PAX_MARKINGS="XT" to your make.conf. I suspect re-emerging oracle-jre-bin after these changes and a reboot may restore the expected pax markings behavior. BR: Dw.
Comment 47 Mike Gilbert gentoo-dev 2016-11-30 11:13:15 UTC
(In reply to Tony Vroon from comment #43)

This failure has nothing to do with the problem reported in this bug report. File a new bug against pax-utils.eclass.