This package fails to build on an unstable system. Attached relevant logs and emerge --info
Created attachment 723853 [details] emerge_info emerge --info
Created attachment 723856 [details] ceph-16.2.5:20210714-080216.log.xz build log (compressed because it exceeds attachment limit, use 'xzless' to read it)
What's happening here? There is a ## 0 in the arguments of gcc [1/1] Cythonizing rgw.pyx running build running build_ext building 'rgw' extension C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc -iquote/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Os -pipe -march=native -fdi agnostics-color=always -fexceptions -Wformat -fstack-clash-protection -Wimplicit-function-declaration -Wno-error=implicit-function-declaration -D_GLIBCXX_ASSERTIONS -fPIC creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9 creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5 creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind creating /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw compile options: '-I/usr/include/python3.9 -I/usr/include/python3.9 -c' extra options: '-iquote/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Os -pipe -march=native -fdiagnostics-color=always -fexce ptions -Wformat -fstack-clash-protection -Wimplicit-function-declaration -Wno-error=implicit-function-declaration -D_GLIBCXX_ASSERTIONS -Wno-unused-result -Wsign-compare -DNDEBUG -Os -pipe -march=native -fdiagnostics-color=always -fexce ptions -Wformat -fstack-clash-protection -Wimplicit-function-declaration -Wno-error=implicit-function-declaration -fwrapv' x86_64-pc-linux-gnu-gcc: /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw/rgw.c x86_64-pc-linux-gnu-gcc: error: ##: No such file or directory x86_64-pc-linux-gnu-gcc: error: 0: No such file or directory error: Command "/usr/bin/x86_64-pc-linux-gnu-gcc -iquote/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Os -pipe -march=native -fdiagnostics-color=always -fexceptions -Wformat -fstack-clash-protection -Wimplicit-function-declaration -Wno-error=implicit-function-declaration -D_GLIBCXX_ASSERTIONS -fPIC -I/usr/include/python3.9 -I/usr/include/python3.9 -c /var/tmp /notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw/rgw.c -o /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/lib/cython_modules/temp.linux-x86_64-3.9/var/tmp/notmpfs/portage/sys-cluster /ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw/rgw.o -iquote/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Os -pipe -march =native -fdiagnostics-color=always -fexceptions -Wformat -fstack-clash-protection -Wimplicit-function-declaration -Wno-error=implicit-function-declaration -D_GLIBCXX_ASSERTIONS -Wno-unused-result -Wsign-compare -DNDEBUG -Os -pipe -march =native -fdiagnostics-color=always -fexceptions -Wformat -fstack-clash-protection -Wimplicit-function-declaration -Wno-error=implicit-function-declaration -fwrapv" failed with exit status 1 src/pybind/rgw/CMakeFiles/cython_rgw.dir/build.make:73: *** [lib/cython_modules/lib.3/rgw.cpython-39-x86_64-linux-gnu.so] error 1 #0 lib/cython_modules/lib.3/rgw.cpython-39-x86_64-linux-gnu.so at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw/CMakeFiles/cython_rgw.dir/build.make:72 #1 cython_rgw at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw/CMakeFiles/cython_rgw.dir/build.make:78 #2 src/pybind/rgw/CMakeFiles/cython_rgw.dir/build at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/src/pybind/rgw/CMakeFiles/cython_rgw.dir/build.make:82 remake[2]: Leaving directory '/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build' CMakeFiles/Makefile2:5240: *** [src/pybind/rgw/CMakeFiles/cython_rgw.dir/all] error 2 #0 src/pybind/rgw/CMakeFiles/cython_rgw.dir/all at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/CMakeFiles/Makefile2:5238 #1 src/pybind/CMakeFiles/cython_modules.dir/all at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/CMakeFiles/Makefile2:5134 #2 src/pybind/all at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/CMakeFiles/Makefile2:1284 #3 src/all at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/CMakeFiles/Makefile2:278 #4 all at /var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build/CMakeFiles/Makefile2:75 remake[1]: Leaving directory '/var/tmp/notmpfs/portage/sys-cluster/ceph-16.2.5/work/ceph-16.2.5_build' Makefile:146: *** [all] error 2
Same is happening with 16.2.6 with recently updated packages, since 2-3 weeks. Try to build in a minimal ~amd64 container, with [ebuild R ] sys-cluster/ceph-16.2.6::gentoo USE="cephfs fuse kerberos mgr numa radosgw sqlite ssl systemd tcmalloc -babeltrace -custom-cflags -diskprediction (-dpdk) -grafana -jaeger -jemalloc -kafka -ldap -lttng -pmdk -rabbitmq -rbd-rwl -rbd-ssd -rgw-lua -spdk (-system-boost) -test -uring -xfs -zbd -zfs" CPU_FLAGS_X86="sse sse2 sse3 -avx2 -avx512f -pclmul -sse4_1 -sse4_2 -ssse3" LUA_SINGLE_TARGET="lua5-3" PYTHON_TARGETS="python3_9 -python3_8" 0 KiB one of the recent dependent packages must be causing this issue. Cheers, Andrej
Well this is a funny one. Turns out the Ceph build is ending up pulling in distutils customizations from numpy, and those customizations are broken when the C compiler has arguments with spaces in them, which Cython uses. Here's the call stack where that happens: Traceback (most recent call last): File "/var/tmp/portage/sys-cluster/ceph-16.2.6/work/ceph-16.2.6/src/pybind/rados/setup.py", line 194, in <module> ext_modules=cythonize( File "/usr/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 963, in cythonize ctx = c_options.create_context() File "/usr/lib/python3.9/site-packages/Cython/Compiler/Main.py", line 596, in create_context return Context(self.include_path, self.compiler_directives, File "/usr/lib/python3.9/site-packages/Cython/Compiler/Main.py", line 80, in __init__ from . import Builtin, CythonScope File "/usr/lib/python3.9/site-packages/Cython/Compiler/CythonScope.py", line 5, in <module> from .UtilityCode import CythonUtilityCode File "/usr/lib/python3.9/site-packages/Cython/Compiler/UtilityCode.py", line 3, in <module> from .TreeFragment import parse_from_strings, StringParseContext File "/usr/lib/python3.9/site-packages/Cython/Compiler/TreeFragment.py", line 17, in <module> from .Visitor import VisitorTransform File "Cython/Compiler/Visitor.py", line 16, in init Cython.Compiler.Visitor File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 31, in <module> from .Pythran import has_np_pythran, pythran_type, is_pythran_buffer File "/usr/lib/python3.9/site-packages/Cython/Compiler/Pythran.py", line 10, in <module> import pythran File "/usr/lib/python3.9/site-packages/pythran/__init__.py", line 40, in <module> from pythran.config import get_include File "/usr/lib/python3.9/site-packages/pythran/config.py", line 8, in <module> import numpy.distutils.system_info as numpy_sys File "/usr/lib/python3.9/site-packages/numpy/distutils/__init__.py", line 24, in <module> from . import ccompiler File "/usr/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 35, in <module> So this is ceph -> Cython -> pythran -> numpy. The bug is in /usr/lib/python3.9/site-packages/numpy/distutils/unixccompiler.py: ccomp_s = " ".join(self.compiler_so) if opt not in ccomp_s: ccomp_s = ccomp_s.replace(gcv_opt, opt) self.compiler_so = ccomp_s.split() This breaks when self.compiler_so has arguments with spaces. I submitted a fix to numpy here: https://github.com/numpy/numpy/pull/20116 . Applying that patch to dev-python/numpy fixes the ceph build.
(In reply to Hector Martin from comment #5) > I submitted a fix to numpy here: https://github.com/numpy/numpy/pull/20116 . > Applying that patch to dev-python/numpy fixes the ceph build. Big thanks. I'll wait a wee bit to see if anyone comments upstream but pull it in soon anyway.
*** Bug 821580 has been marked as a duplicate of this bug. ***
*** Bug 823032 has been marked as a duplicate of this bug. ***
I'd missed (somehow) that the patch was merged a few days ago. On it.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=aeb4bf89ff38ff18c17f51d8c9d0d19cae1c349d commit aeb4bf89ff38ff18c17f51d8c9d0d19cae1c349d Author: Sam James <sam@gentoo.org> AuthorDate: 2021-11-17 23:12:39 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2021-11-17 23:13:04 +0000 dev-python/numpy: add upstream patch for compiler args (fix Ceph build failure) Bug: https://bugs.gentoo.org/802150 Thanks-to: Hector Martin <marcan@marcan.st> Signed-off-by: Sam James <sam@gentoo.org> .../numpy-1.21.4-build-compiler-args-ceph.patch | 49 +++++++ dev-python/numpy/numpy-1.21.4-r1.ebuild | 144 +++++++++++++++++++++ 2 files changed, 193 insertions(+)
Fixed version is stabled.