With python 3.9 and 3.10, and USE=full-support in both cases : $ time ebuild pandas-1.3.5.ebuild clean install real 11m22.591s user 11m9.010s sys 0m17.659s $ time ebuild pandas-1.4.0.ebuild clean install real 86m35.641s user 86m4.450s sys 0m24.960s Reproducible: Always
Created attachment 763387 [details] emerge --info
Please attach the build log.
Created attachment 763400 [details] build log
This is really weird. My first thought was that wheel build rebuilds stuff for you but it doesn't look like it.
Did you happen to spot visually which part of the build process took the longest? i.e. was it "Compiling X because it changed", or the C compilation, or...?
It seems to be the steps like "creating build/temp.linux-x86_64-3.9/pandas/_libs", with a cc1 instance compiling very slowly.
I got it while inspecting the slow cc1 instances : my python interpreters were compiled with -fipa-pta for a performance test, and it seems interpreters CFLAGS are leaking to python packages when building, so pandas was using this flag as well... I deleted this unwanted flag and rebuilded my python interpreters, and now pandas build times are back to normal : $ time ebuild pandas-1.3.5.ebuild clean install real 10m52.294s user 10m38.187s sys 0m17.444s $ time ebuild pandas-1.4.0.ebuild clean install real 11m20.357s user 11m6.101s sys 0m16.395s Strange that pandas <1.4 was unaffected... anyway, sorry for the noise.
No problem. I'm glad you've figured it out. That said, interpreter flags shouldn't really be leaking since we're (or at least should be) exporting CFLAGS. I wonder if pep517 is affecting that.
I have another flag that is only enabled for python interpreters (namely, -fno-semantic-interposition). If you give me a python package not using pep517, I can check if this flag is used during the build.
You can try dev-python/typed-ast.
OK, tried previous pandas too : typed-ast-1.5.1.ebuild: correct CFLAGS pandas-1.3.5.ebuild: correct CFLAGS pandas-1.4.0.ebuild: -fno-semantic-interposition leaked
Thank you for testing. I'm going to look at it later today.
This is regression in distutils vendored with setuptools (i.e. SETUPTOOLS_USE_DISTUTILS=local). I need to check whether this is something that Gentoo patchset to dev-lang/python fixed, or if upstream explicitly broke it.
Ok, it turns out that the undocumented distutils patch we're carrying not only changes some stuff for C++ but also sneakily unsets cflags gotten from sysconfig. I haven't ported the patch to the vendored distutils in setuptools, nor to Python 3.11, and both are affected now. FWICS, there's no way around this -- we either have to patch distutils, or to strip flags from sysconfig. The latter would imply that setup.py called outside Portage has non-standard behavior (well, our old patches are guilty as well) which I don't like. IMO the behavior should be changed from unconditionally stripping Python CFLAGS to overriding these CFLAGS when CFLAGS are explicitly defined in the environment. I wonder what are the chances of pushing this upstream.
Even better, we're building CPython wrong. Apparently we're supposed to use CFLAGS_NODIST, not CFLAGS.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1f8b1370d91c1f1c2322cdedde542c63869a654b commit 1f8b1370d91c1f1c2322cdedde542c63869a654b Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-01-24 14:29:02 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-01-24 22:09:36 +0000 dev-lang/python: Fix hardcoding C/FLAGS in Python 3.6 Closes: https://bugs.gentoo.org/831901 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/python-3.6.15-r1.ebuild | 342 ++++++++++++++++++++++++++++++++ 1 file changed, 342 insertions(+) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e18fd71cd28e0ef8842386cb1729cd1c7ade684c commit e18fd71cd28e0ef8842386cb1729cd1c7ade684c Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-01-24 14:28:44 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-01-24 22:09:35 +0000 dev-lang/python: Fix hardcoding C/FLAGS in Python 3.7 Closes: https://bugs.gentoo.org/831901 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/python-3.7.12_p1-r1.ebuild | 335 +++++++++++++++++++++++++++++ 1 file changed, 335 insertions(+) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0b3320be043d2cf67ca36bcda16fbe5485d8a35f commit 0b3320be043d2cf67ca36bcda16fbe5485d8a35f Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-01-24 14:28:08 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-01-24 22:09:33 +0000 dev-lang/python: Fix hardcoding C/FLAGS in Python 3.8 Closes: https://bugs.gentoo.org/831901 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/python-3.8.12_p1-r2.ebuild | 345 +++++++++++++++++++++++++++++ 1 file changed, 345 insertions(+) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6e1a528e5eacf3c0c05bcabed88fd15c0daa90f8 commit 6e1a528e5eacf3c0c05bcabed88fd15c0daa90f8 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-01-24 14:27:37 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-01-24 22:09:33 +0000 dev-lang/python: Fix hardcoding C/FLAGS in Python 3.9 Closes: https://bugs.gentoo.org/831901 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/python-3.9.10-r1.ebuild | 380 ++++++++++++++++++++++++++++++++ 1 file changed, 380 insertions(+) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c3608b131051020b3e895863a1336f2baad9ef77 commit c3608b131051020b3e895863a1336f2baad9ef77 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-01-24 14:26:16 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-01-24 22:09:32 +0000 dev-lang/python: Fix hardcoding C/LDFLAGS in Python 3.10 Closes: https://bugs.gentoo.org/831901 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/python-3.10.2-r1.ebuild | 386 ++++++++++++++++++++++++++++++++ 1 file changed, 386 insertions(+) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d635364ccb2ede25419c4a6066fa58893d1d5a93 commit d635364ccb2ede25419c4a6066fa58893d1d5a93 Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2022-01-24 14:25:54 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2022-01-24 22:09:31 +0000 dev-lang/python: Fix hardcoding C/LDFLAGS in Python 3.11 Closes: https://bugs.gentoo.org/831901 Signed-off-by: Michał Górny <mgorny@gentoo.org> dev-lang/python/python-3.11.0_alpha4.ebuild | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
