Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 599122 - dev-lang/python-3.4, dev-lang/python-3.5 hang in install phase test_buffer.py
Summary: dev-lang/python-3.4, dev-lang/python-3.5 hang in install phase test_buffer.py
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: ARM Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-07 12:09 UTC by Sven Müller
Modified: 2023-05-06 10:18 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,659.63 KB, text/plain)
2016-11-07 12:09 UTC, Sven Müller
Details
emerge --info (emerge_info.txt,5.44 KB, text/plain)
2016-11-07 13:12 UTC, Sven Müller
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sven Müller 2016-11-07 12:09:26 UTC
Created attachment 452582 [details]
build.log

The problem occurs on both python-3.4.5 and python-3.5.2. Also it's the same behaviour within a qemu-user environment and a native armv7-a environment. 

emerge python works fine until the install stage. It comes to the point: 

Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_audioop.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_augassign.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_base64.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_bigaddrspace.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_bigmem.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_binascii.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_binhex.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_binop.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_bisect.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_bool.py'...
Compiling '/var/tmp/portage/dev-lang/python-3.5.2/image//usr/lib/python3.5/test/test_buffer.py'...

And here it stops. The cpu runs a almost 100%. The cpu last isn't constant, hence python is doing something. Ctrl+C doesn't break the install process, you have to kill it with kill -9. I left it running for more than 5 hours. Nothing changes.
Comment 1 Sven Müller 2016-11-07 12:26:27 UTC
Found a workaround:

At the point it hangs, I can kill the install process. Then I remove the following files:

/var/tmp/portage/dev-lang/python-3.5.2/work/Python-3.5.2/Lib/test/test_math.py
/var/tmp/portage/dev-lang/python-3.5.2/work/Python-3.5.2/Lib/test/test_fractions.py
/var/tmp/portage/dev-lang/python-3.5.2/work/Python-3.5.2/Lib/test/test_float.py
/var/tmp/portage/dev-lang/python-3.5.2/work/Python-3.5.2/Lib/test/test_complex.py
/var/tmp/portage/dev-lang/python-3.5.2/work/Python-3.5.2/Lib/test/test_buffer.py

The the install phase succeeds. With: 

ebuild python-3.5.2.ebuild install
ebuild python-3.5.2.ebuild qmerge
 
I get it installed.
Comment 2 Sven Müller 2016-11-07 13:12:09 UTC
Created attachment 452616 [details]
emerge --info
Comment 3 Sven Müller 2016-11-09 18:52:28 UTC
Some more information:

The install process hangs, when I'm compiling python3.4/python3.5 native or in a qemu-user environment. If I compile python via cross-compiler on my host machine (amd64), then python builds successfully. 

Nevertheless, there are some files missing:

/usr/lib/python3.4/lib2to3/PatternGrammar3.4.5.final.0.pickle
/usr/lib/python3.4/lib2to3/Grammar3.4.5.final.0.pickle

which causes a access violation, when building other python packages. See here: 

https://bugs.gentoo.org/show_bug.cgi?id=599066
Comment 4 Firas Khalil Khana 2017-12-26 12:33:42 UTC
I encountered this issue on my system as well around a month ago on all installed python packages (2.7, 3.4, 3.5, 3.6) after I'd switched my GCC optimization CFLAGS from "-O2" to "-Ofast".

The build process would hang at test_buffer.py (on all python packages) and the CPU usage would remain 100% and the whole thing required a manual "SIGKILL" as I wasn't able to terminate the process...

I created a "package.env" that applies "-O3" instead of "-Ofast" to all my python packages and that solved the problem for me temporarly (that is until they all work with "-Ofast".

As of today (26/12/2017), python:2.7 seems to be working (compiled with "-Ofast"). On the other hand, it seems that the 3.x python versions (or at least 3.4, 3.5, 3.6) are still hanging at "test_buffer.py" (when being compiled with "-Ofast" and not "-O3", at least on my system).
Comment 5 Sven Müller 2018-03-17 09:13:32 UTC
Tried it today. 

Python 3.5 hangs with -Ofast and -O2. It worked with -O3.
Comment 6 Will Simoneau 2018-04-11 02:59:41 UTC
On i686:
-ffast-math => hang
no -ffast-math => OK

sys-devel/binutils-2.30, sys-devel/gcc-6.4.0-r1 here. I'm pretty much using -O2 plus a handful idiotic CFLAGS:
CFLAGS="-O2 -fomit-frame-pointer -fprefetch-loop-arrays -fno-stack-protector -fstack-protector-explicit -pipe -march=pentium3 -mfpmath=sse,387 -fno-float-store -ffast-math"
... but -ffast-math is definitely the key offender.