| Summary: | dev-python/{shiboken,pyside}-1.1.0: add support for python3 | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Rafał Mużyło <galtgendo> |
| Component: | [OLD] Development | Assignee: | Qt Bug Alias <qt> |
| Status: | RESOLVED FIXED | ||
| Severity: | enhancement | CC: | arfrever.fta |
| Priority: | Normal | Keywords: | InVCS |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
ebuild patch
a patch1 for files/ ap atch 2 for /files work-in-progress ebuild works for me shiboken-1.1.0-r1.ebuild shiboken-1.1.0-r1.ebuild pyside-1.1.0-r1.ebuild |
||
OK, this will be a pure cmake question:
python3 gets detected if I make following change in cmake/Modules/FindPython3Libs.cmake
- IF(_CURRENT_VERSION GREATER 3.1)
- SET(_32FLAGS "m" "u" "mu" "")
- ELSE()
- SET(_32FLAGS "")
- ENDIF()
- FOREACH(_COMPILATION_FLAGS ${_32FLAGS})
+ FOREACH(_COMPILATION_FLAGS "m" "u" "mu" "")
This would suggest that the problem lies in not being able "" via a list of strings. Now, is that a problem with the macro or with cmake ?
my tests indicate that the package does not support python3.1. That aside, I have made a patch of these changes, and the ebuild manages to build with them. While python3 gets detected, it doesn't get included in the build and only python2.7 libs are linked and included in the /usr/lib64/cmake/Shiboken-1.1.0/. However, what of the unmentioned FindPython3Interp.cmake. FIND_PROGRAM(PYTHON3_EXECUTABLE NAMES python3.2mu python3.2m python3.2u python3.2 python3.1 python3.0 python3 PATHS [HKEY_LOCAL_MACHINE\\SOFTWARE\\Python\\PythonCore\\3.2\\InstallPath] This is a Windows registry entry, correct me if I'm wrong. Does the source code actually address the existence of a /usr/bin/python3.2 ??? @comment 2: :roll: While I've got only python slots 2.7 and 3.2 installed, I'm nearly sure even 3.0 would have worked. Did you pass -DUSE_PYTHON3=1 to cmake ? As for the path, something similar is in the default (that is one from *cmake* tarball) python2 cmake macro (remember, that the default macro should actually work even for python3, we modify it in the ebuild, so it fits slotting and python.eclass). no. localhost shiboken # e Available Python interpreters: [1] python2.5 [2] python2.6 [3] python2.7 [4] python3.1 [5] python3.2 * localhost shiboken # e4 python-3.1 globally localhost shiboken # USE=debug emerge =dev-python/shiboken-1.1.0 >>> Emerging (1 of 1) dev-python/shiboken-1.1.0 >>> Jobs: 0 of 1 complete, 1 running Load avg: 0.45, 0.32, 0.31openpty failed: 'out of pty devices' >>> Failed to emerge dev-python/shiboken-1.1.0, Log file: >>> '/mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/temp/build.log' localhost shiboken # e5 python-3.2 globally localhost shiboken # USE=debug emerge =dev-python/shiboken-1.1.0 * IMPORTANT: 1 news items need reading for repository 'progress'. * IMPORTANT: 9 news items need reading for repository 'gentoo'. * Use eselect news to read news items. Calculating dependencies... done! >>> Verifying ebuild manifests >>> Emerging (1 of 1) dev-python/shiboken-1.1.0 >>> Installing (1 of 1) dev-python/shiboken-1.1.0 I shall continue to tinker, but this cmake build simply isn't up to the task @comment 4: Do tell, where did I wrote "current ebuild works with python3" ? In fact, I wrote the exact opposite. Don't make out I'm trying to disprove your point. In fact, quite the opposite. I just demoed the package builds with python3.2 eselected. Current ebuild works with python3 as in it build time. Make the most of at least a minor victory. The task at hand is run time python3 support, correct me if I'm wrong. localhost shiboken # find /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/work/shiboken-1.1.0_build/ -name shiboken.pc /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/work/shiboken-1.1.0_build/data/shiboken.pc shiboken.pc is a generated file. I am guessing the answer still lies in adjusting cmake/Modules/FindPython3Libs.cmake. I can't find someone yet who is versed with cmake builds. Created attachment 303059 [details, diff]
ebuild patch
Look what I found
"Use python3 libraries to build shiboken." FALSE)
CMakeLists.txt
/usr/lib64/cmake/Shiboken-1.1.0/ShibokenConfig.cpython3.2.cmake
/usr/lib64/cmake/Shiboken-1.1.0/ShibokenConfigVersion.cmake
/usr/lib64/generatorrunner/shiboken_generator.so
/usr/lib64/libshiboken-python2.7.so.1.1
/usr/lib64/libshiboken-python2.7.so.1.1.0
/usr/lib64/libshiboken.cpython3.2.so.1.1.0
/usr/lib64/pkgconfig/shiboken.pc
Patch attached. There are a number of things to clean up, but this makes for a start
Created attachment 303061 [details, diff]
a patch1 for files/
Created attachment 303065 [details, diff]
ap atch 2 for /files
You should set SUPPORT_PYTHON_ABIS="1" near the top of the ebuild (before inherit) and then build and install the package for both python2 and python3. Have a look at http://www.gentoo.org/proj/en/Python/developersguide.xml#doc_chap2 with this I am familiar. Separate build directories must be used for different Python versions. Distutils by default uses "build" directory, which can be changed by "-b" option of "build" command of setup.py. localhost shiboken # find /var/tmp/portage/portage/dev-python/shiboken-1.1.0/work/shiboken-1.1.0 -name setup.py localhost shiboken # Note it yields blank. You should set near the top of the ebuild One of the final changes I made to the patch before submitting it was to take that line out. It was present for most of the changes. The line python_set_active_version 2 is incompatible with SUPPORT_PYTHON_ABIS="1". One has to go. Note I made the setting of python version 2 conditional. Up to the last version, python_set_active_version 2 was apt. Now that python3 comes into play, it isn't. I have demoed that python3.1 is not supported by the packages at build time. It may do so for run time. Going one step at a time, let's appease build time first. Note also the dependent line PYTHON_DEPEND= is adjusted to PYTHON_DEPEND="2:2.5 3:3.2" which refelcts this status. The only python that can't be used at build time is python3.1, hence the conditional statement. The alternate way is to set RESTRICT_PYTHON_ABIS=" " This is a topical alternative. You suggestions although welcome surprise me. The building of both python2 and python3 are conducted by setup.py, which I have long known. It doesn't exist in the source. It would need to be manufactured. This is doable by one such as Afrever, not yet I, though I haven't yet tried. You never know. The ebuild would accordingly need python_src_compile && python_src_install or dist_utils_src_compile && dist_utils_src_install and explicitly and unconditionally incorprated into the ebuild, and likely the sandard other in pkg_postinst and pkg_postrm. Created attachment 303199 [details]
work-in-progress ebuild
The main issue is the pkgconfig file now...
Created attachment 303895 [details]
works for me
Created attachment 303899 [details]
shiboken-1.1.0-r1.ebuild
Implemented src_test() too. All tests pass on my machine with both python 2.7 and python 3.2
I consider this ebuild complete. I just need to check if pyside still works and then I'll commit it. Further testing is obviously appreciated.
archtester shiboken # USE_PYTHON="2.7 3.2" ebuild shiboken-1.1.0.ebuild clean test 100% tests passed, 0 tests failed out of 129 Total Test time (real) = 11.45 sec * Tests succeeded. archtester shiboken # USE_PYTHON="2.7 3.2" ebuild shiboken-1.1.0.ebuild install -- Installing: /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image/usr/lib64/pkgconfig/shiboken.pc >>> Completed installing shiboken-1.1.0 into /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image/ archtester shiboken # ls /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image/usr/lib64//pkgconfig/ shiboken-2.7.pc shiboken-3.2.pc archtester shiboken # USE_PYTHON="2.7 3.2" ebuild shiboken-1.1.0.ebuild merge >>> Original instance of package unmerged safely. >>> dev-python/shiboken-1.1.0 merged. that would be a yes. The gap.
archtester shiboken # USE_PYTHON="2.6 2.7 3.1" ebuild shiboken-1.1.0.ebuild merge
/mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/work/shiboken-1.1.0-3.1/libshiboken/sbkenum.cpp:235:8: error: ‘Py_hash_t’ does not name a type
/mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/work/shiboken-1.1.0-3.1/libshiboken/sbkenum.cpp: In function ‘PyTypeObject* Shiboken::Enum::newTypeWithName(const char*, const char*)’:
/mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/work/shiboken-1.1.0-3.1/libshiboken/sbkenum.cpp:536:22: error: ‘enum_hash’ was not declared in this scope
make[2]: *** [libshiboken/CMakeFiles/libshiboken.dir/sbkenum.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [libshiboken/CMakeFiles/libshiboken.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
and the bonus
archtester shiboken # USE_PYTHON="2.7-pypy-1.7 3.2" ebuild shiboken-1.1.0.ebuild clean install
-- Installing: /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image/usr/lib64/pkgconfig/shiboken.pc
>>> Completed installing shiboken-1.1.0 into /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image/
archtester shiboken # ls /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image//usr/lib64/pkgconfig/
shiboken-2.7-pypy-1.7.pc shiboken-3.2.pc
archtester shiboken # ls /mnt/gen2/TmpDir/portage/dev-python/shiboken-1.1.0/image//usr/lib64/pkgconfig/
shiboken-2.7-pypy-1.8.pc shiboken-3.2.pc
The change; 3.1 in, pypy out
RESTRICT_PYTHON_ABIS="3.1 *-jython"
Thanks a lot for testing Ian! AFAIK shiboken requires python2 >= 2.6 and python3 >= 3.2, so the failure with 3.1 is expected. I'll restrict it. For pypy, I'm not sure it really works... does it actually build the shiboken library? pypy was merely an afterthought, but it if builds, hey why not. * Building of dev-python/shiboken-1.1.0 with PyPy 1.8 (Python 2.7)... [ 85%] Building CXX object libshiboken/CMakeFiles/libshiboken.dir/shibokenbuffer.cpp.o Linking CXX shared library libshiboken-python2.7.so * Building of dev-python/shiboken-1.1.0 with CPython 2.7... [ 85%] Building CXX object libshiboken/CMakeFiles/libshiboken.dir/shibokenbuffer.cpp.o Linking CXX shared library libshiboken-python2.7.so The chances of you wanting pypy to work are two fold. It also appears it will require some more tweaking of the source so as to not make two lib files of the same name (In reply to comment #18) > The chances of you wanting pypy to work are two fold. It also appears it > will require some more tweaking of the source so as to not make two lib > files of the same name Indeed, pypy support would require more patching.. Created attachment 304407 [details]
shiboken-1.1.0-r1.ebuild
Made the naming scheme consistent across python versions.
Created attachment 304431 [details]
pyside-1.1.0-r1.ebuild
Experimental ebuild for pyside with support for multiple python ABIs.
Right for this -r1 from comment 20; gentoo64 pyside # USE=script USE_PYTHON="2.6 2.7 2.7-pypy-1.8" emerge =dev-python/pyside-1.1.0::gentoo >>> Emerging (2 of 2) dev-python/pyside-1.1.0 >>> Installing (2 of 2) dev-python/pyside-1.1.0 all good. sorry to confuse that was actuualty the -r1 =dev-python/pyside-1.1.0-r1::gentoo dev-python/shiboken-1.1.0-r1 and dev-python/pyside-1.1.0-r1 ebuilds committed to CVS, p.masked for now. More testing is always appreciated, especially testing of reverse deps. I've just committed a rewritten dev-python/pyside-tools-0.2.13-r1 too. Everything unmasked. Thanks to everyone involved, especially to Ian Delaney for the great testing! |
While the tarball content suggest python3 is already supported, for some reason the macro from cmake/Modules/FindPython3Libs.cmake fails to detect it properly during manual build (as the ebuild doesn't support it yet anyway). It seems to be something triggered by ${_COMPILATION_FLAGS}, as simply removing that from both "FIND_LIBRARY(PYTHON3_LIBRARY..." and "FIND_PATH(PYTHON3_INCLUDE_DIR..." makes it work. Can't really say why as the macro itself seems to test the empty suffix - probably some cmake quirk. The other catch might be shiboken.pc, which seems to support only one python version at a time. Adding support here is essential to adding support in dev-python/pyside.