In python_export() in python-utils-r1.eclass, the following command is used to set PYTHON_SITEDIR: PYTHON_SITEDIR=$("${PYTHON}" -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_lib())') || die This ends up setting PYTHON_SITEDIR to the correct path for the build environment not the host environment. When the build platform is multilib (amd64 for instance) and the host platform isn't (arm for instance) PYTHON_SITEDIR should be '/usr/lib/python3.5' but instead is '/usr/lib64/python3.5' and site-packages get installed to the wrong path.
Created attachment 433326 [details, diff] Patch for python-utils-r1.eclass Uses PYTHONPATH="${EROOT}usr/lib/${impl}" to set the correct python module path when calling: "${PYTHON} -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_lib())'" This works whether or not we're cross-compiling. For python3.5 on am amd64 multilib, while not cross-compiling, the command is: PYTHONPATH="/usr/lib/python3.5" /usr/bin/python3.5 -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_lib())' which results in: /usr/lib64/python3.5/site-packages While cross-compiling arm, the command is: PYTHONPATH="/usr/armv7a-hardfloat-linux-gnueabi/usr/lib/python3.5" /usr/bin/python3.5 -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_lib())' which results in: /usr/lib/python3.5/site-packages
(In reply to Peter Levine from comment #1) Thanks for the patch, but I don't like what you are doing there. > For python3.5 on am amd64 multilib, while not cross-compiling, the command > is: > > PYTHONPATH="/usr/lib/python3.5" /usr/bin/python3.5 -c 'import > distutils.sysconfig; print(distutils.sysconfig.get_python_lib())' > > which results in: > > /usr/lib64/python3.5/site-packages This works by accident; if SYMLINK_LIB=no, /usr/lib/python3.5 will not exist on amd64. > While cross-compiling arm, the command is: > PYTHONPATH="/usr/armv7a-hardfloat-linux-gnueabi/usr/lib/python3.5" > /usr/bin/python3.5 -c 'import distutils.sysconfig; > print(distutils.sysconfig.get_python_lib())' > > which results in: > > /usr/lib/python3.5/site-packages Setting PYTHONPATH to a python install using a different ABI sounds rather dangerous. I think an easier solution here would be to revert the eclass back to using a hard-coded path: ${EPREFIX}/usr/$(get_libdir)/${EPYTHON}/site-packages
(In reply to Mike Gilbert from comment #2) > (In reply to Peter Levine from comment #1) > > Thanks for the patch, but I don't like what you are doing there. > > > For python3.5 on am amd64 multilib, while not cross-compiling, the command > > is: > > > > PYTHONPATH="/usr/lib/python3.5" /usr/bin/python3.5 -c 'import > > distutils.sysconfig; print(distutils.sysconfig.get_python_lib())' > > > > which results in: > > > > /usr/lib64/python3.5/site-packages > > This works by accident; if SYMLINK_LIB=no, /usr/lib/python3.5 will not exist > on amd64. > > > While cross-compiling arm, the command is: > > PYTHONPATH="/usr/armv7a-hardfloat-linux-gnueabi/usr/lib/python3.5" > > /usr/bin/python3.5 -c 'import distutils.sysconfig; > > print(distutils.sysconfig.get_python_lib())' > > > > which results in: > > > > /usr/lib/python3.5/site-packages > > Setting PYTHONPATH to a python install using a different ABI sounds rather > dangerous. I agree. I don't have much experience with python build settings and environment variables but this was the hack that made it work on my end. > I think an easier solution here would be to revert the eclass back to using > a hard-coded path: > > ${EPREFIX}/usr/$(get_libdir)/${EPYTHON}/site-packages That would be the most elegant solution if it's sufficient.
Thinking on this a bit more, even if we hard-code the path in the eclass, anything using distutils (setup.py) to install things will still get the path wrong when the CHOST libdir is not equal to the CBUILD libdir. Getting this right would require a broader change, and I'm not sure it is worth the effort.
*** Bug 591878 has been marked as a duplicate of this bug. ***
I am not sure if this fix completely solves the problem, but we ran into this in Chrome OS and were considering the following patch, but it might not cover some of the edge cases previously mentioned in this bug: diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass index e3cf82b4..368b2688 100644 --- a/eclass/python-utils-r1.eclass +++ b/eclass/python-utils-r1.eclass @@ -918,7 +918,7 @@ python_domodule() { local PYTHON_SITEDIR=${PYTHON_SITEDIR} [[ ${PYTHON_SITEDIR} ]] || python_export PYTHON_SITEDIR - d=${PYTHON_SITEDIR#${EPREFIX}}/${python_moduleroot//.//} + d=${PYTHON_SITEDIR#${SYSROOT}}/${python_moduleroot//.//} fi ( @@ -954,7 +954,7 @@ python_doheader() { local d PYTHON_INCLUDEDIR=${PYTHON_INCLUDEDIR} [[ ${PYTHON_INCLUDEDIR} ]] || python_export PYTHON_INCLUDEDIR - d=${PYTHON_INCLUDEDIR#${EPREFIX}} + d=${PYTHON_INCLUDEDIR#${SYSROOT}} ( insopts -m 0644
Created attachment 562694 [details, diff] Replace EPREFIX with SYSROOT when stripping paths.
I met this problem twice when cross compiling from amd64 prefix to an arm64 prefix. The first time is dev-lang/python, where PYTHON_SITEDIR is determined by execute ${PYTHON}. However, ${PYTHON} is pointed towards outside prefix, which result in a worsen case -- my host system Debian's python says that PYTHON_SITEDIR=/usr/local/lib/python3.11/dist-packages The second time is sys-apps/portage, where wrong PYTHON_SITEDIR causes incorrect installation. Is anyone currently working on this? I will try to take a look if there's no existing solution
(In reply to Yiyang Wu from comment #8) > The first time is dev-lang/python Sorry, both two problems occurs when cross-compile portage -- they are actually the same one.
