Cheers, Attached you will see 'emerge --info', 'emerge --info =media-video/mpv-0.18.1', 'emerge -pqv =media-video/mpv-0.18.1', and mpv's build.log. The setup is a hardened Gentoo desktop. Reproducible: 1. Get current portage tree 2. Follow update recommendation of portage and let it emerge mpv 3. Wait for python to die during the configure phase Actual Result: Python fails during the configure phase of the package. Expected Result: New mpv version installed.
Created attachment 442316 [details] emerge --info
Created attachment 442318 [details] emerge --info =media-video/mpv-0.18.1
Created attachment 442320 [details] emerge --pqv =media-video/mpv-0.18.1
Created attachment 442322 [details] build.log
Created attachment 442324 [details] environment
Please show emerge --info dev-lang/python:3.4. Also, rebuilding dev-lang/python:3.4 can't hurt.
Created attachment 442342 [details] emerge --info dev-lang/python:3.4 Just reemerged dev-lang/python:3.4 and tried reemerging mpv. Still fails. Here is what emerge --info dev-lang/python:3.4 reports.
Missing PYTHON_REQ_USE?
(In reply to Michał Górny from comment #8) > Missing PYTHON_REQ_USE? Hmm, nah. Flags look fine ;-f. Maybe it's waf using some obsolete/too new interfaces?
Not reproducible on my regular system. I am currently building hardened chroot and will try to reproduce there.
Please attach "/var/tmp/portage/media-video/mpv-0.18.1/work/mpv-0.18.1/build/config.log" as well.
(In reply to Michał Górny from comment #9) > Maybe it's waf using some obsolete/too new interfaces? This waf version was tested to work properly with a wide range of python versions from 2.7 to 3.5. Unless smth broke between python-3.4.3-r1 and python-3.4.5 it should be fine.
Also not reproducible on ~amd64 system with python-3.4.5. Must be a hardened issue.
Created attachment 442432 [details] config.log Per request my config.log of mpv. On a sidenote, I also experience build issues on www-client/chromium and trying to rebuild dev-libs/glib. I can open new bug reports for them as well. In that case and if my logs grow beyonf the 1000KB limit, may you be so kind and recommend me a way to compress my logs (meaning the command name, I can read up the usage via manpages). Last time I used tar and you didn't like it. I have a hunch in that something corrupted my FS and broke some of my system libs.
(In reply to mvaenskae from comment #14) > On a sidenote, I also experience build issues on www-client/chromium and > trying to rebuild dev-libs/glib. I can open new bug reports for them as > well. In that case and if my logs grow beyonf the 1000KB limit, may you be > so kind and recommend me a way to compress my logs (meaning the command > name, I can read up the usage via manpages). Last time I used tar and you > didn't like it. Any of xz, bzip2, xz are acceptable. They all work the same way -- they compress a single file without adding an unnecessary wrapper around it. Usage: xz build.log
Also, you should file separate bugs for separate packages. You can link them together using the See Also field if you think the failures are related.
(In reply to Mike Gilbert from comment #15) > Any of xz, bzip2, xz are acceptable. That should say xz, bzip2, gzip.
This is likely the same MPROTECT problem as bug 590422.
commit b8cb87b5af9ab23e60864a4c66988c0f595f5b77 Author: Mike Gilbert <floppym@gentoo.org> Date: Wed Aug 3 20:41:29 2016 -0400 dev-lang/python: disable MPROTECT The hardened team can figure this shit out. Package-Manager: portage-2.3.0_p16 dev-lang/python/python-2.7.12.ebuild | 7 +------ dev-lang/python/python-3.4.5.ebuild | 7 +------ dev-lang/python/python-3.5.2.ebuild | 7 +------ 3 files changed, 3 insertions(+), 18 deletions(-)
(In reply to Mike Gilbert from comment #19) I reverted this.
(In reply to mvaenskae from comment #14) > Created attachment 442432 [details] > config.log > > Per request my config.log of mpv. You've attached "temp/build.log" again. Please attach "work/mpv-0.18.1/build/config.log" instead.
Did you ever install any Python modules manually or via pip?
(In reply to Coacher from comment #21) > (In reply to mvaenskae from comment #14) > > Created attachment 442432 [details] > > config.log > > > > Per request my config.log of mpv. > You've attached "temp/build.log" again. > Please attach "work/mpv-0.18.1/build/config.log" instead. Oh, that is stupid of me and I apologize. I will have it uploaded later today in roughly 8 to 10 hours when I am back home.
(In reply to Michał Górny from comment #22) > Did you ever install any Python modules manually or via pip? I actually did that for one package, namely tensorflow for dev-lang/python:3.4. Confusingly this error had never appeared before I removed the package that was then not needed anymore. I did remove it now though and checked with the following command what was installed by pip and what wasn't: `pip list | while read l a; do loc=$(pip show "$l" | fgrep Location | awk '{ print $2 }') && find "${loc}" '(' -name "${l}*" -o -name "${l//-/_}*" ')' | head -n 1 | xargs qfile >/dev/null || echo "$l"; done` (Reference: https://forums.gentoo.org/viewtopic-t-1040730-start-0.html) I at least learned my lesson to never again use pip, but back then I didn't know and was more pressured into the quick and dirty solution due to time constraints. I would think doing a python-rebuild would fix this issue of the python packages having incorrect pax-flags (with MPROTECT turned off) and then recompile all leftover python packages (i.e. dev-lang/python) for getting their flags again correct.
Well, the problem with Python is that packages can influence many issues, often in an completely unpredictable way. This also includes leftover files after packages. I would suggest checking system Python directories for orphan files. I can suggest a tool for it when I get home, of possibly another team member would be able to suggest one earlier.
Here's a good command (you need app-portage/portage-utils): find /usr/lib64/python* -print0 | xargs -0 qfile -o
The strace log attached to bug 590422 shows python2.7 trying to mmap some memory with PROT_WRITE|PROT_EXEC just before it would call clone(2) to create a new thread. Can you try the same strace test with python3.4 for this bug?
(In reply to Mike Gilbert from comment #27) > The strace log attached to bug 590422 shows python2.7 trying to mmap some > memory with PROT_WRITE|PROT_EXEC just before it would call clone(2) to > create a new thread. > > Can you try the same strace test with python3.4 for this bug? I will do so later when I get home. I assume you want the default pax-flags set per default on it.
(In reply to Michał Górny from comment #25) > Well, the problem with Python is that packages can influence many issues, > often in an completely unpredictable way. This also includes leftover files > after packages. > > I would suggest checking system Python directories for orphan files. I can > suggest a tool for it when I get home, of possibly another team member would > be able to suggest one earlier. Thanks for the command in comment #26. What should I be doing in the case of finding some?
(In reply to mvaenskae from comment #28) > I will do so later when I get home. I assume you want the default pax-flags > set per default on it. Well, I'm really just looking for the mmap call, and whether it gets passed PROT_WRITE|PROT_EXEC. If it does, we are looking at the same problem with both python2.7 and python3.4. The PaX flags will simply determine whether the mmap call gets treated as an error or not.
Created attachment 442542 [details] config.log Hopefully this time the correct config.log file
Created attachment 442544 [details] strace of python:3.4 Per comment #30 I have uploaded my strace of the same execution done prior with python2.7. It does the same mmap with READ|WRITE|EXEC from the looks before the clone.
Your problem is caused by python binary without GNU_STACK header. The same problem causes your other bug: https://bugs.gentoo.org/show_bug.cgi?id=590422 Workaround: paxctl-ng -m the python binary. Solution: figure out how your toolchain's problem (PAX_MARKINGS, PT_PAX support: I suggest to disable PT and keep XT) and re-emerge python. I also suggest to eselect python 3.4 or 2.7 but not 3.5 (pypax).
mvaenskae, I see that bug 590422 is now resolved. Does this bug is still valid or it was resolved as well?
(In reply to Coacher from comment #34) > mvaenskae, I see that bug 590422 is now resolved. > Does this bug is still valid or it was resolved as well? mvaenskae, after more than 2 weeks of your silence I consider this bug as resolved. I tend to agree with comment 33 that this bug is indeed a dupe of bug 590422, but I cannot reproduce so I cannot say for sure. Closing as FIXED similarly to bug 590422.
