https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/ Issue: dev-lang/swift-5.10.1 fails to compile. Discovered on: amd64 (internal ref: guru_ci) Info about the issue: https://wiki.gentoo.org/wiki/Project:Tinderbox/Common_Issues_Helper#CF0014
Created attachment 903006 [details] build.log.xz build log and emerge --info (compressed because it exceeds attachment limit, use 'xzless' to read it)
Error(s) that match a know pattern: open class var standardError: FileHandle { but the path does not exist. -- Could NOT find CURL (missing: CURL_DIR) -- Could NOT find Clang (missing: Clang_DIR) -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) -- Could NOT find Foundation (missing: Foundation_DIR) -- Could NOT find LLVM (missing: LLVM_DIR) -- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) -- Could NOT find Python module pygments -- Could NOT find Python module pygments.lexers.c_cpp -- Could NOT find Python module yaml -- Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR) (Required is at least version "3") -- Could NOT find XCTest (missing: XCTest_DIR) -- Could NOT find dispatch (missing: dispatch_DIR) -- Could not find ParallelSTL, libc++abi will not attempt to use it but the build may fail if the libc++ in use needs it to be available. ERROR: command terminated with a non-zero exit status 1, aborting FAILED: bindings/python/LLDBWrapPython.cpp bindings/python/lldb.py /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python/LLDBWrapPython.cpp /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python/lldb.py ModuleNotFoundError: No module named 'distutils' stream <<< "error: "
Hmm. I'm able to compile swift-5.10.1 from scratch in a stage3 Docker container with $ emerge --sync $ emerge --ask dev-vcs/git app-eselect/eselect-repository $ eselect repository enable guru $ emerge --sync guru $ emerge --ask dev-lang/swift The stage3 container has dev-python/setuptools just like tinderbox, though the version is different: * stage3: dev-python/setuptools-73.0.1:0 -python_targets_pypy3 -python_targets_python3_10 -python_targets_python3_11 python_targets_python3_12 -python_targets_python3_13 -test * tinderbox: dev-python/setuptools-74.1.2:0 python_targets_pypy3 python_targets_python3_10 python_targets_python3_11 python_targets_python3_12 -python_targets_python3_13 -test I'll see if updating makes a difference.
I just noticed that CMake is somehow picking up and using Python 3.13 here, despite PYTHON_SINGLE_TARGET being 3.12: cd /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python && /usr/bin/python3.13 /var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb/bindings/prepare_bindings.py --srcRoot=/var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb --targetDir=/var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python --cfgBldDir=/var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64/bindings/python --prefix=/var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/lldb-linux-x86_64 --use-static-binding Given that setuptools is indeed not being built for Python 3.13 (-python_targets_python3_13), this failure would make sense — but I don't know why CMake would be picking up 3.13 despite the Python environment being set up with `python_setup`. I'll try to install 3.13 in this container and see if I can repro.
Looks like CMake defaults to searching for individual Python versions from 3.13 down, and uses the first one it finds. See also: https://bugs.gentoo.org/835799 Looks like I need to pass in `-DPython3_EXECUTABLE=${PYTHON}`
Actually, it looks like `-DPython3_FIND_UNVERSIONED_NAMES:STRING="FIRST"` might be a better option, because LLDB is searching for the Python3 Development module via CMake, and just setting the executable doesn't pick it up (whereas finding the unversioned `python` libs does). Confirmed this in the `gentoo/python` docker image with Python 3.13 installed (where not setting it picks up the Python 3.13 libs instead).
Okay, confirmed that without `-DPython3_FIND_UNVERSIONED_NAMES`, installation fails in a clean `gentoo/python` container, which has Python 3.13 (and 3.12 and below) installed with the same exact error; now testing the fix.
Hmm, this does not appear to be sufficient: $ env /usr/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/var/tmp/portage/dev-lang/swift-5.10.1/work/stage1/usr/bin/clang <SNIP> -Xlinker /var/tmp/portage/dev-lang/swift-5.10.1/work/build/Ninja-ReleaseAssert/foundation-linux-x86_64/lib' -DPython3_FIND_UNVERSIONED_NAMES=FIRST -DSWIFT_USE_LINKER=lld -DBUILD_TESTING:BOOL=NO -DSWIFT_INCLUDE_TESTS:BOOL=NO <SNIP> /var/tmp/portage/dev-lang/swift-5.10.1/work/llvm-project/lldb ... -- Found Python3: /usr/bin/python3.13 (found version "3.13.0") found components: Interpreter Development Development.Module Development.Embed -- Found PythonAndSwig: /usr/lib64/libpython3.13.so I wonder if this is getting overridden somehow?
Argh, it's because `Python3_FIND_UNVERSIONED_NAMES` was introduced in CMake 3.20, and the components of Swift 5.10.1 use an older version of CMake. lldb, specifically, uses CMake 3.13.4, which rules out pretty much all modern hints I can give to CMake. I might, however, be able to use `Python3_ROOT_DIR` instead.
Okay, setting `PYTHON_HOME` (which `llvm-project/lldb/cmake/modules/FindPythonAndSwig.cmake` assigns to `Python3_ROOT_DIR`) does appear to allow the right version of Python to be found. Will try setting this.
Oh, interesting. It appears that `Python3_FIND_UNVERSIONED_NAMES` _is_ actually necessary, but I need to _also_ supply `PYTHON_HOME` along with it for it to take effect.
> I need to _also_ supply `PYTHON_HOME` along with it for it to take effect. This assessment was incorrect. When manually testing, I had forgotten to set my PATH to include the temp dir containing Python. > It appears that `Python3_FIND_UNVERSIONED_NAMES` _is_ actually necessary What I had missed when testing was that I temporarily set the `cmake_minimum_version` for LLDB to 3.20, so `Python3_FIND_UNVERSIONED_NAMES` was actually taking effect; with LLDB's default 3.13.4, it does not. So—fingers crossed I have it right this time—it looks like I'll need to include a patch to update `cmake_minimum_version` for LLDB, then add `Python3_FIND_UNVERSIONED_NAMES`.
Confirmed the fix with a patch to LLDB's `cmake_minimum_version` while passing in `-DPython3_FIND_UNVERSIONED_NAMES=FIRST`.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/proj/guru.git/commit/?id=c9dfaf59e30a6e3d6bb138eb47d1fa8d03c19d51 commit c9dfaf59e30a6e3d6bb138eb47d1fa8d03c19d51 Author: Itai Ferber <itai@itaiferber.net> AuthorDate: 2024-09-15 23:49:12 +0000 Commit: Itai Ferber <itai@itaiferber.net> CommitDate: 2024-09-15 23:50:13 +0000 dev-lang/swift: set LLDB to respect PYTHON_SINGLE_TARGET Closes: https://bugs.gentoo.org/939685 Signed-off-by: Itai Ferber <itai@itaiferber.net> .../files/swift-5.10.1-lldb-cmake-minimum-version.patch | 15 +++++++++++++++ dev-lang/swift/swift-5.10.1.ebuild | 17 +++++++++++++++-- 2 files changed, 30 insertions(+), 2 deletions(-)
