| Summary: | media-libs/opencv-3.4.1-r5 with dev-util/cmake-3.14.0 - Found PythonInterp: /tmp/portage/media-libs/opencv-3.4.1-r5/temp/python2.7/bin/python3 (Required is at least version "3.4") CMake Error at cmake/OpenCVDetectPython.cmake:59 (if): | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Georgy Yakovlev <gyakovlev> |
| Component: | Current packages | Assignee: | Amy Liffey <amynka> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | anton.gubarkov, anton.kochkov, atoth, b.buschinski, bob.mt.wya, creideiki+gentoo-bugzilla, dschridde+gentoobugs, email, gentoo.bugzilla, gentoo, gmt, grozin, josef64, kajanos, kdvgent, klaus818, krinpaus, mark+gentoobugs, me, n.sevchenco, nrndda, phantom4, rabbe, stefantalpalaru, steils, steven, vmatare+gbug, zeekec |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
opencv cmake python failure
0001-media-libs-opencv-fix-python-build-with-cmake-3.14.patch |
||
|
Description
Georgy Yakovlev
2019-03-18 04:16:05 UTC
quickly glancing at logs it looks like it comes from cmake/OpenCVDetectPython.cmake:225 (find_python) Same problem for me. Build log and emerge info at https://bpaste.net/show/dbb9426ff951 . However, I do have another Gentoo system where the exact same version build successfully. Not sure what's wrong. (In reply to Pistos from comment #2) > Same problem for me. Build log and emerge info at > https://bpaste.net/show/dbb9426ff951 . However, I do have another Gentoo > system where the exact same version build successfully. Not sure what's > wrong. can you check if you have opencl-related packages installed and useflag enabled on that other system? I've started seeing this problem after I globally enabled opencl useflag and got related packages installed. I haven't had any time to find the source of the problem yet. Same issue here with installed =dev-util/cmake-3.14.0 With =cmake-3.13.4 builds opencv-3.4.1-r5 here successfully. so it blows up on
if(${_preferred_version_major} EQUAL ${PYTHON_VERSION_MAJOR})
basically PYTHON_VERSION_MAJOR is passed as empty.
it's derived from CMake's FindPythonInterp() which was modified in 3.14
https://github.com/Kitware/CMake/commit/08dee696b5831a8cba0a70df5e6b072f91d0c6f8#diff-9646870e455c818224d924fdec5d174a
FindPythonInterp: Do not assume any version if test script fails
The Python version is retrieved by executing a small python script. If,
for any reason, script fails to execute, leave the `PYTHON_VERSION_*`
variables undefined instead of assuming version 1.4.
it tries to run
python -c "try: import sys; sys.stdout.write(sys.version)\nexcept: sys.stdout.write(\"1.4.0\")"
so something happens here and PYTHON_VERSION_MAJOR ends up empty and passed to if which blows up.
minimal cmake file is CMakeLists.txt cmake_minimum_required(VERSION 3.14) execute_process(COMMAND "python" -c "try: import sys; sys.stdout.write(sys.version)\nexcept: sys.stdout.write(\"1.4.0\")") but it works fine. cmake . -- The C compiler identification is GNU 8.3.0 -- The CXX compiler identification is GNU 8.3.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done 3.6.8 (default, Mar 4 2019, 13:11:46) [GCC 8.3.0]-- Configuring done -- Generating done -- Build files have been written to: /home/ya/ctest I'll trace actual build process later to see what is actually outputs. continuing
so for python3 it tries to run
/usr/share/cmake/Modules/FindPythonInterp.cmake(141): execute_process(COMMAND /tmp/portage/media-libs/opencv-3.4.1-r5/temp/python2.7/bin/python3 -c try: import sys; sys.stdout.write(sys.version)
except: sys.stdout.write("1.4.0") OUTPUT_VARIABLE _VERSION RESULT_VARIABLE _PYTHON_VERSION_RESULT ERROR_QUIET )
/usr/share/cmake/Modules/FindPythonInterp.cmake(145): if(NOT _PYTHON_VERSION_RESULT )
/usr/share/cmake/Modules/FindPythonInterp.cmake(154): else()
/usr/share/cmake/Modules/FindPythonInterp.cmake(155): unset(PYTHON_VERSION_STRING )
/usr/share/cmake/Modules/FindPythonInterp.cmake(156): unset(PYTHON_VERSION_MAJOR )
/usr/share/cmake/Modules/FindPythonInterp.cmake(157): unset(PYTHON_VERSION_MINOR )
/usr/share/cmake/Modules/FindPythonInterp.cmake(158): unset(PYTHON_VERSION_PATCH )
/usr/share/cmake/Modules/FindPythonInterp.cmake(161): unset(_PYTHON_VERSION_RESULT )
/usr/share/cmake/Modules/FindPythonInterp.cmake(162): unset(_VERSION )
and the reason it fails is because of hack in the ebuild:
# cheap trick: python_setup sets one of them as a symlink
# to the correct interpreter, and the other to fail-wrapper
-DPYTHON2_EXECUTABLE=$(type -P python2)
-DPYTHON3_EXECUTABLE=$(type -P python3)
-DPYTHON2_EXECUTABLE=/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python2.7/bin/python2
-DPYTHON3_EXECUTABLE=/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python2.7/bin/python3
/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python2.7/bin/python3
: python_wrapper_setup: python3 is not supported by python2.7 (PYTHON_COMPAT)
with following contents
#!/bin/sh
echo ": python_wrapper_setup: python3 is not supported by python2.7 (PYTHON_COMPAT)" >&2
exit 127
OpenCVDetectPython.cmake tries to run python3 wrapper while building for python2 in, but above wrapper created by eclass func python_wrapper_setup() exits and cmake no longer sets PYTHON_VERSION_MAJOR
now need to figure out how to fix this mess without breaking python =)
instead of passing
-DPYTHON2_EXECUTABLE=$(type -P python2)
-DPYTHON3_EXECUTABLE=$(type -P python3)
it's possible to pass
-DPYTHON_DEFAULT_EXECUTABLE="$(type -P ${EPYTHON%.*})"
or even simple
-DPYTHON_DEFAULT_EXECUTABLE="${EPYTHON%.*}"
or even
-DPYTHON_DEFAULT_EXECUTABLE=python
because
type -P python
/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python3.6/bin/python
PATH /tmp/portage/media-libs/opencv-3.4.1-r5/temp/python3.6/bin:/usr/lib/portage/python3.6/ebuild-helpers/xattr:/usr/lib/portage/python3.6/ebuild-helpers:/usr/lib/llvm/8/bin:/usr/lib/llvm/7/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
cat /tmp/portage/media-libs/opencv-3.4.1-r5/temp/python3.6/bin/python
#!/bin/sh
exec "/usr/bin/python3.6" "${@}"
this will stop cmake module from guessing python and just accept given implementation at current ebuild phase.
if(PYTHON_DEFAULT_EXECUTABLE)
set(PYTHON_DEFAULT_AVAILABLE "TRUE") < - this branch is selected
elseif(PYTHON2INTERP_FOUND) # Use Python 2 as default Python interpreter
set(PYTHON_DEFAULT_AVAILABLE "TRUE")
set(PYTHON_DEFAULT_EXECUTABLE "${PYTHON2_EXECUTABLE}")
elseif(PYTHON3INTERP_FOUND) # Use Python 3 as fallback Python interpreter (if there is no Python 2)
set(PYTHON_DEFAULT_AVAILABLE "TRUE")
set(PYTHON_DEFAULT_EXECUTABLE "${PYTHON3_EXECUTABLE}")
endif()
since PATH is mangled by python eclasses ebuild will find a correct wrapper for current implementation and will use that for build, it prepends correct path for current python implementation.
I haven't tested it for multiple python3 implementations, but I guess it should work.
(In reply to Georgy Yakovlev from comment #8) > instead of passing > > -DPYTHON2_EXECUTABLE=$(type -P python2) > -DPYTHON3_EXECUTABLE=$(type -P python3) Why do those lines even need to be in the ebuild? Everything works just fine if they are removed (and nothing else added) and simply adding: -DPYTHON_DEFAULT_EXECUTABLE=python is not sufficient; the magic is deleting the both the PYTHON2 and PYTHON3 lines. Just deleting the PYTHON3 line ends up with this: -- Found PythonInterp: /var/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python3.6/bin/python2 (Required is at least version "2.7") CMake Error at cmake/OpenCVDetectPython.cmake:59 (if): if given arguments: "2" "EQUAL" I don't understand why this ever worked; apparently both supported python versions have to be there, working, in order to get the major version numbers and that only happens if the -DPYTHON[23]_EXECUTABLE does not point to the "fail" wrapper, but at some point this wasn't required (with the current 3.4.1-r5 ebuild). (In reply to John Bowler from comment #9) > (In reply to Georgy Yakovlev from comment #8) > > instead of passing > > > > -DPYTHON2_EXECUTABLE=$(type -P python2) > > -DPYTHON3_EXECUTABLE=$(type -P python3) > > Why do those lines even need to be in the ebuild? W4M: spenciver /usr # diff -up {portage,overlay/gmt}/media-libs/opencv/opencv-3.4.1-r5.ebuild --- portage/media-libs/opencv/opencv-3.4.1-r5.ebuild 2019-01-24 01:39:29.000000000 -0800 +++ overlay/gmt/media-libs/opencv/opencv-3.4.1-r5.ebuild 2019-03-23 15:23:00.792356668 -0700 @@ -474,8 +474,6 @@ python_module_compile() { mycmakeargs+=( # cheap trick: python_setup sets one of them as a symlink # to the correct interpreter, and the other to fail-wrapper - -DPYTHON2_EXECUTABLE=$(type -P python2) - -DPYTHON3_EXECUTABLE=$(type -P python3) -DINSTALL_PYTHON_EXAMPLES=$(usex examples) -DLIBPY_SUFFIX=64 ) spenciver /usr # ebuild overlay/gmt/media-libs/opencv/opencv-3.4.1-r5.ebuild clean install >/dev/null || echo nope spenciver /usr # egrep '^--[[:space:]]*Python|^[[:space:]]\*[[:space:]][a-zF].*:' /var/tmp/portage/media-libs/opencv-3.4.1-r5/temp/build.log * FEATURES: compressdebug installsources network-sandbox preserve-libs sandbox splitdebug userpriv usersandbox * FEATURES: compressdebug installsources network-sandbox preserve-libs sandbox splitdebug userpriv usersandbox * abi_x86_64.amd64: running multilib-minimal_abi_src_configure -- Python (for build): NO * abi_x86_64.amd64: running multilib-minimal_abi_src_compile * abi_x86_64.amd64: running multilib-minimal_abi_src_install * python2_7: running python_module_compile -- Python 2: -- Python (for build): /var/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python2.7/bin/python * python3_6: running python_module_compile -- Python 3: -- Python (for build): /var/tmp/portage/media-libs/opencv-3.4.1-r5/temp/python3.6/bin/python * Final size of build directory: 886232 KiB (865.4 MiB) * Final size of installed tree: 265588 KiB (259.3 MiB) spenciver /usr # Could you please adjust the summary to contain the error message? There is already a duplicate: bug #681594 *** Bug 681594 has been marked as a duplicate of this bug. *** Yeah, not sure how that "python3" executable is created within the python2.7/bin subdir. For what it's worth, it's not specific to the "opencl" USE flag, as I don't have that flag enabled; yet I, too, experience this issue. I don't think this is due to changes in the media-libs/opencv package, by the way. I have successfully built this package before without problems. I do recall a large number of python packages being updated recently. I wonder if this issue is related to that. [21:04][2078]# lsl /var/tmp/portage/media-libs/opencv-3.4.1-r100/temp/python2.7/bin/ total 24K lrwxrwxrwx. 1 root 17 Mar 24 20:49 2to3 -> /usr/bin/2to3-2.7 -rwxr-xr-x. 1 root 43 Mar 24 20:49 python -rwxr-xr-x. 1 root 43 Mar 24 20:49 python2 -rwxr-xr-x. 1 root 50 Mar 24 20:49 python2-config -rwxr-xr-x. 1 root 108 Mar 24 20:49 python3 -rwxr-xr-x. 1 root 115 Mar 24 20:49 python3-config -rwxr-xr-x. 1 root 50 Mar 24 20:49 python-config [21:04][2079]# /var/tmp/portage/media-libs/opencv-3.4.1-r100/temp/python2.7/bin/python3 --version : python_wrapper_setup: python3 is not supported by python2.7 (PYTHON_COMPAT) [21:04][2080]# /var/tmp/portage/media-libs/opencv-3.4.1-r100/temp/python2.7/bin/python2 --version Python 2.7.16 [21:04][2081]# python2 --version Python 2.7.16 Note, the same issue holds for the media-libs/opencv-3.4.1-r5 version from the gentoo repos as for my media-libs/opencv-3.4.1-r100; in case someone decides to point out the difference in my previous output. (In reply to Greg Turner from comment #10) > spenciver /usr # diff -up > {portage,overlay/gmt}/media-libs/opencv/opencv-3.4.1-r5.ebuild > --- portage/media-libs/opencv/opencv-3.4.1-r5.ebuild 2019-01-24 > 01:39:29.000000000 -0800 > +++ overlay/gmt/media-libs/opencv/opencv-3.4.1-r5.ebuild 2019-03-23 > 15:23:00.792356668 -0700 > @@ -474,8 +474,6 @@ python_module_compile() { > mycmakeargs+=( > # cheap trick: python_setup sets one of them as a symlink > # to the correct interpreter, and the other to fail-wrapper > - -DPYTHON2_EXECUTABLE=$(type -P python2) > - -DPYTHON3_EXECUTABLE=$(type -P python3) > -DINSTALL_PYTHON_EXAMPLES=$(usex examples) > -DLIBPY_SUFFIX=64 > ) This also works for me. A patched opencv-3.4.1-r6 is available in my overlay: https://github.com/stefantalpalaru/gentoo-overlay Works for me, too. Created attachment 570892 [details, diff]
0001-media-libs-opencv-fix-python-build-with-cmake-3.14.patch
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a232a9da5922f62dfb73eba78b360bc5eb2e8a19 commit a232a9da5922f62dfb73eba78b360bc5eb2e8a19 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2019-03-26 18:49:25 +0000 Commit: Georgy Yakovlev <gyakovlev@gentoo.org> CommitDate: 2019-03-26 20:46:53 +0000 media-libs/opencv: fix python build with cmake 3.14 Closes: https://bugs.gentoo.org/680824 Package-Manager: Portage-2.3.62, Repoman-2.3.12 Acked-by: Amy Liffey <amynka@gentoo.org> Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> media-libs/opencv/opencv-3.4.1-r5.ebuild | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) |