The issue seems to appear whenever an ebuild requires an extension to be built. I have attached terminal output building dev-python/memory_allocator-0.1.1 from the https://github.com/cschwan/sage-on-gentoo overlay up through the compile phase. One file is for compiling on Gentoo and the other on Prefix. Typical output is Prefix: running build_ext >>> Source compiled. Gentoo: running build_ext building 'memory_allocator.memory_allocator' extension creating /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.linux-x86_64-3.9 creating /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.linux-x86_64-3.9/memory_allocator x86_64-pc-linux-gnu-gcc -march=native -O2 -pipe -fPIC -I/usr/include/python3.9 -c memory_allocator/memory_allocator.c -o /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.linux-x86_64-3.9/memory_allocator/memory_allocator.o building 'memory_allocator.test' extension x86_64-pc-linux-gnu-gcc -march=native -O2 -pipe -fPIC -I/usr/include/python3.9 -c memory_allocator/test.c -o /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.linux-x86_64-3.9/memory_allocator/test.o x86_64-pc-linux-gnu-gcc -shared -Wl,-O1 -Wl,--as-needed -march=native -O2 -pipe /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.linux-x86_64-3.9/memory_allocator/memory_allocator.o -L/usr/lib64 -o /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/lib/memory_allocator/memory_allocator.cpython-39-x86_64-linux-gnu.so x86_64-pc-linux-gnu-gcc -shared -Wl,-O1 -Wl,--as-needed -march=native -O2 -pipe /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.linux-x86_64-3.9/memory_allocator/test.o -L/usr/lib64 -o /var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/lib/memory_allocator/test.cpython-39-x86_64-linux-gnu.so >>> Source compiled. Even though compile output is missing for extensions in Prefix the extensions are built.
Created attachment 744834 [details] compiling on Prefix
Created attachment 744837 [details] compiling on Gentoo
Created attachment 744840 [details] emerge --info on Prefix
This is probably not related to the distutils-r1.eclass but to something python on Prefix. On Prefix, relative to https://github.com/cschwan/sage-on-gentoo/blob/master/dev-python/memory_allocator/memory_allocator-0.1.1.ebuild after ebuild memory_allocator-0.1.1.ebuild unpack there is at the beginning of setup.py in the work folder from setuptools import setup, find_packages from setuptools.extension import Extension from distutils.command.build_ext import build_ext as du_build_ext class build_ext(du_build_ext): def run(self): from Cython.Build.Dependencies import cythonize self.distribution.ext_modules[:] = cythonize( self.distribution.ext_modules, language_level=3) du_build_ext.run(self) which appears to be correct. However, from the work folder python setup.py build_ext returns just `running build_ext` without any compiling info. And then `python setup.py build_ext --force` returns again just the same `running build_ext`. On Gentoo python setup.py build_ext returns running build_ext building 'memory_allocator.memory_allocator' extension x86_64-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python3.9 -c memory_allocator/memory_allocator.c -o build/temp.linux-x86_64-3.9/memory_allocator/memory_allocator.o x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed build/temp.linux-x86_64-3.9/memory_allocator/memory_allocator.o -L/usr/lib64 -o build/lib.linux-x86_64-3.9/memory_allocator/memory_allocator.cpython-39-x86_64-linux-gnu.so building 'memory_allocator.test' extension x86_64-pc-linux-gnu-gcc -pthread -fPIC -I/usr/include/python3.9 -c memory_allocator/test.c -o build/temp.linux-x86_64-3.9/memory_allocator/test.o x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed build/temp.linux-x86_64-3.9/memory_allocator/test.o -L/usr/lib64 -o build/lib.linux-x86_64-3.9/memory_allocator/test.cpython-39-x86_64-linux-gnu.so
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=ce4687a8c9a175ac2b08b42881efead4218a7f58 commit ce4687a8c9a175ac2b08b42881efead4218a7f58 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2021-10-21 06:39:04 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2021-10-21 06:39:04 +0000 python-utils-r1: sync with gx86, bug #818043 Bug: https://bugs.gentoo.org/818043 Signed-off-by: Fabian Groffen <grobian@gentoo.org> eclass/python-utils-r1.eclass | 117 ++++++++++++++++++++++-------------------- 1 file changed, 62 insertions(+), 55 deletions(-)
can you try using above commit, it did enable some compileall phases for later python versions, which is possibly this issue
I could not directly patch but used https://gitweb.gentoo.org/repo/proj/prefix.git/plain/eclass/python-utils-r1.eclass?id=ce4687a8c9a175ac2b08b42881efead4218a7f58 from the git tree I see no difference in output. After `ebuild memory_allocator-0.1.1.ebuild clean compile` I have ... running build_ext >>> Source compiled.
What platforms have you seen this on?
For me: running build_ext building 'memory_allocator.memory_allocator' extension building 'memory_allocator.test' extension creating /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9 creating /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9/memory_allocator creating /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9/memory_allocator x86_64-apple-darwin20-gcc -march=native -O2 -pipe -I/Users/sam/Gentoo/usr/include/python3.9 -c memory_allocator/memory_allocator.c -o /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9/memory_allocator/memory_allocator.o x86_64-apple-darwin20-gcc -march=native -O2 -pipe -I/Users/sam/Gentoo/usr/include/python3.9 -c memory_allocator/test.c -o /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9/memory_allocator/test.o x86_64-apple-darwin20-gcc -bundle -undefined dynamic_lookup -Wl,-dead_strip_dylibs -march=native -O2 -pipe /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9/memory_allocator/memory_allocator.o -L/Users/sam/Gentoo/usr/lib -o /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/lib/memory_allocator/memory_allocator.cpython-39-darwin.bundle x86_64-apple-darwin20-gcc -bundle -undefined dynamic_lookup -Wl,-dead_strip_dylibs -march=native -O2 -pipe /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/temp.macosx-11.0-x86_64-3.9/memory_allocator/test.o -L/Users/sam/Gentoo/usr/lib -o /Users/sam/Gentoo/var/tmp/portage/dev-python/memory_allocator-0.1.1/work/memory_allocator-0.1.1-python3_9/lib/memory_allocator/test.cpython-39-darwin.bundle >>> Source compiled. >>> Test phase [not enabled]: dev-python/memory_allocator-0.1.1 Are you sure they're actually being built?
Yes, they are built for both python3.9 and python3.10. From the work folder $ ls memory_allocator-0.1.1-python3_9/lib/memory_allocator/ __init__.pxd memory.pxd __init__.py signals.pxd memory_allocator.cpython-39-x86_64-linux-gnu.so test.cpython-39-x86_64-linux-gnu.so memory_allocator.pxd $ ls memory_allocator-0.1.1-python3_10/lib/memory_allocator/ __init__.pxd memory.pxd __init__.py signals.pxd memory_allocator.cpython-310-x86_64-linux-gnu.so test.cpython-310-x86_64-linux-gnu.so memory_allocator.pxd The host is $ uname -a Linux vaio-home 3.13.0-170-generic #220-Ubuntu SMP Thu May 9 12:40:49 UTC 2019 x86_64 Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz GenuineIntel GNU/Linux with OS $ lsb_release -a LSB Version: core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch:core-4.1-amd64:core-4.1-noarch:security-4.0-amd64:security-4.0-noarch:security-4.1-amd64:security-4.1-noarch Distributor ID: Ubuntu Description: Ubuntu 14.04.6 LTS Release: 14.04 Codename: trusty
I now do see slightly different output: python3.9 running build_ext Compiling memory_allocator/memory_allocator.pyx because it depends on /storage/strogdon/gentoo-rap/usr/lib/python3.9/site-packages/Cython/Includes/libc/string.pxd. Compiling memory_allocator/test.pyx because it depends on /storage/strogdon/gentoo-rap/usr/lib/python3.9/site-packages/Cython/Includes/libc/stdint.pxd. [1/2] Cythonizing memory_allocator/memory_allocator.pyx [2/2] Cythonizing memory_allocator/test.pyx >>> Source compiled. python3.10 running build_ext Compiling memory_allocator/memory_allocator.pyx because it depends on /storage/strogdon/gentoo-rap/usr/lib/python3.10/site-packages/Cython/Includes/libc/string.pxd. Compiling memory_allocator/test.pyx because it depends on /storage/strogdon/gentoo-rap/usr/lib/python3.10/site-packages/Cython/Includes/libc/stdint.pxd. [1/2] Cythonizing memory_allocator/memory_allocator.pyx [2/2] Cythonizing memory_allocator/test.pyx >>> Source compiled.
This output is now in sync with what I see on Gentoo. I have no idea what has changed?
Perhaps cython is the culprit. I now have dev-python/cython-0.29.26 installed everywhere.
Perhaps the change that Sam made to python-3.9.6 for distutils? In any case happy it seems resolved now. Please reopen if it comes back again.
