As far as I can see since 3.11 minimal host python is built during crossbuild. But seems that it can't find necessary modules: checking for --with-build-python... Python path configuration: PYTHONHOME = (not set) PYTHONPATH = (not set) program name = '/build/arm64-usr/var/tmp/portage/dev-lang/python-3.11.1/work/python-3.11.1-aarch64-unknown-linux-gnu/python' isolated = 0 environment = 1 user site = 1 safe_path = 0 import site = 1 is in build tree = 1 stdlib dir = '/build/arm64-usr/var/tmp/portage/dev-lang/python-3.11.1/work/python-3.11.1-aarch64-unknown-linux-gnu/Lib' sys._base_executable = '/build/arm64-usr/var/tmp/portage/dev-lang/python-3.11.1/work/python-3.11.1-aarch64-unknown-linux-gnu/python' sys.base_prefix = '/usr' sys.base_exec_prefix = '/usr' sys.platlibdir = 'lib' sys.executable = '/build/arm64-usr/var/tmp/portage/dev-lang/python-3.11.1/work/python-3.11.1-aarch64-unknown-linux-gnu/python' sys.prefix = '/usr' sys.exec_prefix = '/usr' sys.path = [ '/usr/lib/python311.zip', '/build/arm64-usr/var/tmp/portage/dev-lang/python-3.11.1/work/python-3.11.1-aarch64-unknown-linux-gnu/Lib', '/build/arm64-usr/var/tmp/portage/dev-lang/python-3.11.1/work/python-3.11.1-aarch64-unknown-linux-gnu/build/lib.linux-aarch64-3.11', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings'
I have seen this one several times but haven't figured it out yet. 3.10 works fine.
Please always include the full build.log and emerge --info.
I've figured it out. Although we build a separate Python for the build host, it was still relying on the same version actually being installed. It was looking for libpython3.11.so, but we hadn't added the build directory to LD_LIBRARY_PATH, so it was finding it under /usr instead. The installed Python looks for modules under /usr rather than ${S}. I'm not sure why 3.11 failed here, even when it was already installed, but it doesn't really matter. If you examine libpython3.11.so, you'll see it differs between our temporary build and a regular build. The former has ${S} embedded as the stdlib directory, whereas the latter does not. How come? This took quite a while to figure out, but the build sets the VPATH macro to this directory only when you're building outside the source tree. There's some ugly code in configure to unset VPATH when srcdir is "." because you're building inside the source tree. Strange but true. The simple solution here is to add --disable-shared to the temporary build. That way, there is no libpython3.11.so and everything just works. It's late now, so I'll double check which versions I should apply this to tomorrow.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=708460e50bca1d251a826bf74c00a3021ea76f10 commit 708460e50bca1d251a826bf74c00a3021ea76f10 Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2023-01-23 22:30:37 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2023-01-23 22:30:37 +0000 dev-lang/python: Fix cross-compiling, particularly 3.11 Despite efforts to avoid needing the same version of Python installed on the build host, this was still required due to libpython.so not being found. Even when it was found, it was erroneously looking under /usr rather than ${S} for the Python modules. For some unknown reason, this caused 3.11 to fail. The easiest solution is to just build a static Python with no libpython.so. This also ensures modules are loaded from ${S}. Closes: https://bugs.gentoo.org/889874 Signed-off-by: James Le Cuirot <chewi@gentoo.org> dev-lang/python/python-3.10.9.ebuild | 3 +++ dev-lang/python/python-3.11.1.ebuild | 3 +++ dev-lang/python/python-3.12.0_alpha4.ebuild | 3 +++ dev-lang/python/python-3.8.16-r3.ebuild | 3 +++ dev-lang/python/python-3.9.16.ebuild | 3 +++ 5 files changed, 15 insertions(+)
