Trying to update to dev-python/xcffib-0.11.1, compilation for python3_8 target goes well with python3_8: running distutils-r1_run_phase distutils-r1_python_compile but for target python3_9 fails with python3_9: running distutils-r1_run_phase distutils-r1_python_compile python3.9 setup.py build -j 6 /usr/lib/python3.9/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer. warnings.warn( /usr/bin/python3.9: No module named pip Reproducible: Always Steps to Reproduce: 1. emerge -1av dev-python/xcffib 2. 3. Actual Results: * python3_9: running distutils-r1_run_phase distutils-r1_python_compile python3.9 setup.py build -j 6 /usr/lib/python3.9/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer. warnings.warn( /usr/bin/python3.9: No module named pip Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/setuptools/installer.py", line 82, in fetch_build_egg subprocess.check_call(cmd) File "/usr/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/usr/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps' , '-w', '/XXXXXX/portage/dev-python/xcffib-0.11.1/temp/tmpi02zo7dx', '--quiet', 'pycparser']' returned non-zero exit status 1. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/XXXXXX/portage/dev-python/xcffib-0.11.1/work/xcffib-0.11.1/setup.py", line 60, in <module> setup( File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 152, in setup _install_setup_requires(attrs) File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 812, in fetch_build_eggs resolved_dists = pkg_resources.working_set.resolve( File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 771, in resolve dist = best[req.key] = env.best_match( File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1056, in best_match return self.obtain(req, installer) File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1068, in obtain return installer(requirement) File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 883, in fetch_build_egg return fetch_build_egg(self, req) File "/usr/lib/python3.9/site-packages/setuptools/installer.py", line 84, in fetch_build_egg raise DistutilsError(str(e)) from e distutils.errors.DistutilsError: Command '['/usr/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/XXXXXX/portage/dev-python/xcffib-0.11.1/temp/tmpi02zo7dx', '--quiet', 'pycparser']' returned non-zero exit status 1. * ERROR: dev-python/xcffib-0.11.1::gentoo failed (compile phase): * (no error message) * * Call stack: * ebuild.sh, line 127: Called src_compile * environment, line 3064: Called distutils-r1_src_compile * environment, line 1356: Called _distutils-r1_run_foreach_impl 'distutils-r1_python_compile' * environment, line 561: Called python_foreach_impl 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 2743: Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 2284: Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 2282: Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 881: Called distutils-r1_run_phase 'distutils-r1_python_compile' * environment, line 1347: Called distutils-r1_python_compile * environment, line 1128: Called esetup.py 'build' '-j' '6' * environment, line 1823: Called die * The specific snippet of code: * "${@}" || die -n; * Expected Results: No error Reason ------ I first thought that dev-python/xcffib-0.11.1 *needs* pip. But, during preparation of this bug report, I saw pycparser being mentioned in the error output. As it turned out, the true reason of the above was that dev-python/pycparser was installed in target python3_8, but not in python3_9: eix pycparser [U] dev-python/pycparser Available versions: 2.21^t {test PYTHON_TARGETS="pypy3 python3_10 python3_8 python3_9"} Installed versions: 2.20(07:04:09 pm 19/01/2022)(PYTHON_TARGETS="python2_7 python3_6 python3_7 python3_8 -pypy3") Homepage: https://github.com/eliben/pycparser Description: C parser and AST generator written in Python It was also in a lower version than the current one, but I don't think this was the problem, since compilation for Python 3.8 went smoothly. Solution -------- Upgrading dev-python/pycparser (which also installed it in the python3_9 target) resolved the issue. You might want to add some check in the ebuild: dev-python/pycparser is a build dependency and must be present for all python targets where this package should be installed. You may close this as resolved. Some info: Portage 3.0.28 (python 3.8.12-final-0, default/linux/amd64/17.1/hardened, gcc-11.2.0, glibc-2.30-r8, 5.4.168-gentoo x86_64) ================================================================= System uname: Linux-5.4.168-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700HQ_CPU_@_2.60GHz-with-glibc2.2.5 Timestamp of repository gentoo: Sat, 22 Jan 2022 sh bash 5.1_p8 ld GNU ld (Gentoo 2.37_p1 p0) 2.37 app-misc/pax-utils: 1.3.3::gentoo app-shells/bash: 5.1_p8::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.34.0-r6::gentoo dev-lang/python: 2.7.18_p13::gentoo, 3.6.10-r2::gentoo, 3.7.7-r2::gentoo, 3.8.12_p1-r1::gentoo, 3.9.9-r1::gentoo, 3.10.0_p1-r1::gentoo dev-lang/rust: 1.53.0::gentoo dev-lang/rust-bin: 1.53.0::gentoo dev-util/cmake: 3.21.4::gentoo dev-util/meson: 0.60.3::gentoo sys-apps/baselayout: 2.7-r3::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.25::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.11.6-r3::gentoo, 1.12.6::gentoo, 1.13.4-r2::gentoo, 1.14.1::gentoo, 1.15.1-r2::gentoo, 1.16.4::gentoo sys-devel/binutils: 2.33.1-r1::gentoo, 2.37_p1::gentoo sys-devel/binutils-config: 5.4::gentoo sys-devel/clang: 7.0.0::gentoo, 8.0.1::gentoo, 9.0.1::gentoo, 10.0.0::gentoo, 12.0.1::gentoo, 13.0.0::gentoo sys-devel/gcc: 7.5.0::gentoo, 8.3.0-r1::gentoo, 8.4.0::gentoo, 9.3.0::gentoo, 11.2.0::gentoo sys-devel/gcc-config: 2.5-r1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/lld: 13.0.0::gentoo sys-devel/llvm: 7.0.0-r1::gentoo, 8.0.1::gentoo, 9.0.1::gentoo, 10.0.0::gentoo, 12.0.1::gentoo, 13.0.0::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.30-r8::gentoo
Please attach the full build.log in future but yeah, looks like missing pycparser.
There's not a single "pycparser" occurrence in the source. On the other hand, the package requires cffi and cffi depends on pycparser. Did you have also have wrong targets on dev-python/cffi? Did you do some hacking after building cffi?
Created attachment 764535 [details] xcffib-0.11.1 build log
(In reply to Michał Górny from comment #2) > There's not a single "pycparser" occurrence in the source. On the other > hand, the package requires cffi and cffi depends on pycparser. Did you have > also have wrong targets on dev-python/cffi? Did you do some hacking after > building cffi? Maybe not in the source - but if you look at the output above - and in the build.log that I now posted, you will see: subprocess.CalledProcessError: Command '['/usr/bin/python3.9', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps' , '-w', '/XXXXXX/portage/dev-python/xcffib-0.11.1/temp/tmpi02zo7dx', '--quiet', 'pycparser']' returned non-zero exit status 1. which does contain 'pycparser'. It was this that tipped me off to look at pycparser.
(In reply to Michał Górny from comment #2) > Did you have > also have wrong targets on dev-python/cffi? No. Python targets for dev-python/cffi must have been correct. I can see this now from the dates when the two packages were merged: dev-python/cffi-1.15.0 was merged on 24th Jan., 2022, in the right targets: [I] dev-python/cffi Available versions: 1.14.6:0/1.14.6^t 1.15.0:0/1.15.0^t {doc test PYTHON_TARGETS="python3_10 python3_8 python3_9"} Installed versions: 1.15.0:0/1.15.0^t(01:40:34 PM 01/24/2022)(-doc -test PYTHON_TARGETS="python3_8 python3_9 -python3_10") while dev-python/xcffib-0.11.1 was merged on 4th Feb., 2022: [I] dev-python/xcffib Available versions: 0.11.1^t {test PYTHON_TARGETS="python3_10 python3_8 python3_9"} Installed versions: 0.11.1^t(11:16:20 AM 02/04/2022)(-test PYTHON_TARGETS="python3_8 python3_9 -python3_10") > Did you do some hacking after > building cffi? No.
Did you perhaps rebuild pycparser after emerging cffi
(In reply to Michał Górny from comment #6) > Did you perhaps rebuild pycparser after emerging cffi Yes, because, as of today: [I] dev-python/pycparser Available versions: 2.21^t {test PYTHON_TARGETS="pypy3 python3_10 python3_8 python3_9"} Installed versions: 2.21^t(11:08:34 AM 02/04/2022)(-test PYTHON_TARGETS="python3_8 python3_9 -pypy3 -python3_10") therefore pycparser was (re-)compiled on 02/04/2022, while cffi still (even today) has a date of 01/24/2022: [I] dev-python/cffi Available versions: 1.14.6:0/1.14.6^t 1.15.0:0/1.15.0^t {doc test PYTHON_TARGETS="python3_10 python3_8 python3_9"} Installed versions: 1.15.0:0/1.15.0^t(01:40:34 PM 01/24/2022)(-doc -test PYTHON_TARGETS="python3_8 python3_9 -python3_10") so, indeed, pycparser was (re-)compiled on a later date than dev-python/cffi. FWIW dev-python/xcffib, dev-python/cairocffi and virtual/python-cffi also have a 02/04/2022 date on them. That is, all of them were (re-)merged on a later date than dev-python/cffi. That's because I am in the process of *incrementally* upgrading my system, since I cannot do it at once. I don't want to bother you with the details (they are irrelevant and you have enough to do :-)), but you can read about it here: https://forums.gentoo.org/viewtopic-t-1146987.html Be warned that you need time, patience - and good nerves. :-)))
Ok, so the problem roughly is: normally Portage only checks for immediate dependencies. So if you emerge xcffib, it checks whether all packages directly required by xcffib (i.e. cffi here) are installed but it doesn't check if cffi's dependencies are still satisfied. There are options like --deep or --complete-graph that can change this behavior and make Portage compute the complete dependency tree every time, at the cost of slower processing. However, since you're still working on upgrading your system, they could cause more trouble than help. Ideally, get to the point when you can --deep upgrade @world, then depclean and everything should work fine afterwards.
(In reply to Michał Górny from comment #8) > Ok, so the problem roughly is: normally Portage only checks for immediate > dependencies. So if you emerge xcffib, it checks whether all packages > directly required by xcffib (i.e. cffi here) are installed but it doesn't > check if cffi's dependencies are still satisfied. > > There are options like --deep or --complete-graph that can change this > behavior and make Portage compute the complete dependency tree every time, > at the cost of slower processing. Just of curiosity, I wanted to see if --deep would catch it. So let's take pycparser out of the python3_9 target and see if --deep will catch it: /etc/portage/package.use: # TEMPORARILY take pycparser out of python3_9. =dev-python/pycparser-2.21 -python_targets_python3_9 emerge -1av dev-python/pycparser ...oops, this did complain about the installed dev-python/cffi. :-) So let's do: emerge -1av --nodeps dev-python/pycparser Now pycparser is missing from python3_9. Let's see if --deep will catch it when we merge dev-python/xcffib: emerge -1Dav dev-python/xcffib These are the packages that would be merged, in reverse order: Calculating dependencies... done! [ebuild R ] dev-python/xcffib-0.11.1::gentoo USE="-test" PYTHON_TARGETS="python3_8 python3_9 -python3_10" 0 KiB [nomerge ] dev-python/cffi-1.15.0:0/1.15.0::gentoo USE="-doc -test" PYTHON_TARGETS="python3_8 python3_9 -python3_10" [ebuild R ] dev-python/pycparser-2.21::gentoo USE="-test" PYTHON_TARGETS="python3_8 python3_9* (-pypy3) -python3_10" 0 KiB Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB The following USE changes are necessary to proceed: (see "package.use" in the portage(5) man page for more details) # required by dev-python/cffi-1.15.0::gentoo[-test] # required by dev-python/xcffib-0.11.1::gentoo[python_targets_python3_9,python_targets_python3_8,-python_targets_python3_10] # required by dev-python/xcffib (argument) >=dev-python/pycparser-2.21 python_targets_python3_9 Would you like to add these changes to your config files? [Yes/No] So yes, it does! The key is to use -D (--deep) if one wants to catch such things. Now I learned something, again. :-)
