Created attachment 466634 [details] "emerge --info" output. Before we get this ebuild party started, I'd like to gratuitously thank all maintainers, contributors, and QA-ers of the Qt overlay for their tireless efforts. I'd particularly like to thank Pesa and kensington for their maintenance of the PySide2 and Shiboken2 ebuilds. This bug report's for you. (Literally.) Rather than split this report into multiple issues (which I probably should have done), I lazily condensed everything pertaining to PySide2 and Shiboken2 into a single monolithic issue (which I probably should not have done). I blame Larry Wall. In descending order of importance, PySide2 and Shiboken2 issues include: [[[ ISSUE 1 ]]] Packaging Nomenclature The PySide2 and Shiboken2 packages currently share the same namespace as the PySide1 and Shiboken1 packages: * "<dev-python/pyside-9999" provides PySide1. * "~dev-python/pyside-9999" provides PySide2. * "<dev-python/shiboken-9999" provides Shiboken1. * "~dev-python/shiboken-9999" provides Shiboken2. This is a problem waiting to burst like an infested blood bubble. PySide1 and PySide2 reside in distinct Python packages; Shiboken1 and Shiboken2 reside in distinct shared libraries. No filesystem or Python conflicts exist. All four are explicitly designed to be concurrently installable into the same Python environment. On my system, for example: * PySide1 resides at "/usr/lib64/python3.5/site-packages/PySide". * PySide2 resides at "/usr/lib64/python3.5/site-packages/PySide2". * Shiboken1 resides at "/usr/lib64/python3.5/site-packages/shiboken.so". * Shiboken2 resides at "/usr/lib64/python3.5/site-packages/shiboken2.so". The Qt overlay's oversimplified packaging erases these distinctions, preventing PySide1 and PySide2 (as well as Shiboken1 and Shiboken2) from cohabiting the same system-wide Python environment. Worryingly, this approach also silently breaks most applications requiring PySide1. Installing PySide2 and Shiboken2 from the Qt overlay silently uninstalls PySide1 and Shiboken1 installed from Portage. PySide1 and PySide2 are fundamentally incompatible, residing in distinct packages and exposing distinct APIs. Although transparently supporting both in the same application is technically feasible (e.g., with a compatibility layer like "Qt.py"), most applications requiring PySide1 will break at runtime in the absence of PySide1. PySide2 does not suffice. Ideally, this namespace collision should be resolved as soon as feasible – before downstream overlays begin authoring ebuilds depending on the PySide2 and Shiboken2 ebuilds provided by the Qt overlay. Like I sort-of intended to do. [[[ ISSUE 2 ]]] Dependency Versioning The PySide2 ebuild currently requires a minimum of Qt >= 4.8.5. (See the ${QT_PV} global defined by "pyside-9999.ebuild".) You see the problem, I trust. PySide2 is a Qt 5 framework. No version of Qt 4 (including Qt 4.8.5) suffices to satisfy PySide2 requirements. The minimum version of Qt required by PySide2 is specified on line 67 of PySide2's top-level "CMakeLists.txt" makefile, reading: find_package(Qt5 5.3.0 REQUIRED COMPONENTS Core) Ergo, line 37 of the PySide2 ebuild should probably be revised to read: QT_PV="5.3.0:5" [[[ ISSUE 3 ]]] Everything Is Broken, I Have No More Tears to Cry Some or all USE flags appear to be currently ignored. This is bad for all the obvious reasons. In my case, however, PySide2 is not currently installable on my system but would be if at least one USE flags was respected. Specifically, Shiboken2 fails to parse system-wide XML files at PySide2 installation time required only by the "webkit" USE flag: $ emerge pyside . . . make[2]: Entering directory '/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7' [ 94%] Running generator for QtWebKitWidgets... cd /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets && /usr/bin/shiboken2 --generator-set=shiboken --enable-parent-ctor-heuristic --enable-pyside-extensions --enable-return-value-heuristic --use-isnull-as-nb_nonzero /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/pyside2_global.h --include-paths=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2:/usr/include/qt5/ --typesystem-paths=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtCore:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtGui:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWidgets:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtCore:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtGui:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtWidgets:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtPrintSupport:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtNetwork:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtPrintSupport:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtNetwork:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets --output-directory=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtWebKitWidgets --license-file=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets/../licensecomment.txt /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets/typesystem_webkitwidgets.xml --api-version=5.6 --drop-type-entries="" qt.shiboken: Can't find typesystem_webkit.xml, typesystem paths: /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtCore, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtGui, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWidgets, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtCore, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtGui, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtWidgets, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtPrintSupport, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtNetwork, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtPrintSupport, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtNetwork, /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets Fatal error: line=46, column=62, message=Failed to parse: 'typesystem_webkit.xml' Cannot parse file: /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets/typesystem_webkitwidgets.xmlshiboken: Error running ApiExtractor. Command line: --generator-set=shiboken --enable-parent-ctor-heuristic --enable-pyside-extensions --enable-return-value-heuristic --use-isnull-as-nb_nonzero /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/pyside2_global.h --include-paths=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2:/usr/include/qt5/ --typesystem-paths=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtCore:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtGui:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWidgets:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtCore:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtGui:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtWidgets:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtPrintSupport:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtNetwork:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtPrintSupport:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtNetwork:/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets --output-directory=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/PySide2/QtWebKitWidgets --license-file=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets/../licensecomment.txt /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999/PySide2/QtWebKitWidgets/typesystem_webkitwidgets.xml --api-version=5.6 --drop-type-entries= make[2]: *** [PySide2/QtWebKitWidgets/CMakeFiles/QtWebKitWidgets.dir/build.make:79: PySide2/QtWebKitWidgets/PySide2/QtWebKitWidgets/qgraphicswebview_wrapper.cpp] Error 1 make[2]: Leaving directory '/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7' make[1]: *** [CMakeFiles/Makefile2:940: PySide2/QtWebKitWidgets/CMakeFiles/QtWebKitWidgets.dir/all] Error 2 make[1]: Leaving directory '/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7' make: *** [Makefile:128: all] Error 2 * ERROR: dev-python/pyside-9999::qt failed (compile phase): * emake failed * * If you need support, post the output of `emerge --info '=dev-python/pyside-9999::qt'`, * the complete build log and the output of `emerge -pqv '=dev-python/pyside-9999::qt'`. * The complete build log is located at '/var/tmp/portage/dev-python/pyside-9999/temp/build.log.gz'. * The ebuild environment file is located at '/var/tmp/portage/dev-python/pyside-9999/temp/environment'. * Working directory: '/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7' * S: '/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999' >>> Failed to emerge dev-python/pyside-9999, Log file: >>> '/var/tmp/portage/dev-python/pyside-9999/temp/build.log.gz' Naturally, PySide2 fails at 94% after compiling for an hour. *shakes fist* Naturally, I have explicitly disabled the "webkit" USE flag for "dev-python/pyside" – as confirmed by the following "build-python2_7.log" snippet: [32;01m*[0m python2_7: running configuration >>> Working in BUILD_DIR: "/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7" cmake -C /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_TESTS=no -DDISABLE_QtGui=no -DDISABLE_QtTest=no -DDISABLE_QtQml=yes -DDISABLE_QtQuick=yes -DDISABLE_QtQuickWidgets=yes -DDISABLE_QtUiTools=no -DDISABLE_QtHelp=no -DDISABLE_QtMultimedia=no -DDISABLE_QtOpenGL=no -DDISABLE_QtScript=yes -DDISABLE_QtScriptTools=yes -DDISABLE_QtSql=yes -DDISABLE_QtSvg=no -DDISABLE_QtWebKit=yes -DDISABLE_QtXmlPatterns=no -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_USER_MAKE_RULES_OVERRIDE=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/gentoo_rules.cmake -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/dev-python/pyside-9999/work/pyside-9999-python2_7/gentoo_toolchain.cmake /var/tmp/portage/dev-python/pyside-9999/work/pyside-9999 The "-DDISABLE_QtWebKit=yes" option implies the "webkit" USE flag to be disabled, which it is. Yet the WebKit-specific error above remains. So, what's going on here? Ignoring the upstream failure of Shiboken2 to parse a core XML file (not our fault, but aggravating nonetheless), PySide2's top-level "CMakeLists.txt" makefile ignores all attempts by our PySide2 ebuild to set command-line options derived from USE flags – as confirmed by the subsequent "build-python2_7.log" snippet: module Qt5WebKit found module Qt5WebKitWidgets found We told CMake to ignore WebKit, but somebody told CMake to ignore us. My dark suspicion is that PySide2's build system no longer respects the "cmake" command-line options passed by the src_configure() phase of our PySide2 ebuild, silently reducing this phase to a noop. By inspection, the only references to "WebKit" in "CMakeLists.txt" are lines 164 and 165, reading: find_package(Qt5WebKit) find_package(Qt5WebKitWidgets) My infantile hope is that this is fixable by us. Would patching "CMakeLists.txt" in the src_prepare() phase suffice? Specifically, would it be feasible to wrap the above lines with a conditional testing the truth of our "DISABLE_QtWebKit" option? My CMake-foo is weak, however; I'm not quite confident I know how to implement this, else I'd have attached a working patch. An initial stab into the darkness might resemble: if (NOT DISABLE_QtWebKit) find_package(Qt5WebKit) find_package(Qt5WebKitWidgets) endif() It can't be that easy, of course. I'll hack on this tomorrow and see if I can MacGyver up a general-purpose solution. Until then, thanks again for your herculean Qt efforts. Beers all around!
Created attachment 466636 [details] "build-python2_7.log" for "dev-python/pyside-9999".
Houston, we have worky. The key is this single obfuscatory sentence embedded deep within the bowels of the official CMake documentation for the find_package() function: "Every non-REQUIRED find_package call can be disabled by setting the CMAKE_DISABLE_FIND_PACKAGE_<PackageName> variable to TRUE." See also: https://cmake.org/cmake/help/v3.0/command/find_package.html Given this, any optional PySide2 dependency "Qt5Blah" may be disabled by passing the corresponding option "-DCMAKE_DISABLE_FIND_PACKAGE_Qt5Blah=1" at configuration time (e.g., "-DCMAKE_DISABLE_FIND_PACKAGE_Qt5Svg=1" to disable SVG). There are caveats, however. The PySide2 ebuild erroneously attempts to disable Qt4 rather than Qt5 dependencies (e.g., the Qt4-specific dependency "QtGui" rather than the Qt5-specific dependency "Qt5Gui"). Fixing this mostly reduces to replacing every prefix "Qt" by "Qt5", which is trivial. Mostly. Some dependencies that were optional under PySide are now mandatory under PySide2, including "Qt5Gui", "Qt5Sql", and "Qt5Test". Yes, that's right: PySide2 appears to require both X and SQL support. I'm inclined to believe that the SQL requirement at least is an upstream bug. The comparable PyQt5 framework does *NOT* require SQL, and there appears to be no demonstrable reason for PySide2 to do so either. An issue should probably be posted to the upstream PySide2 tracker at: https://bugreports.qt.io/browse/PYSIDE/fixforversion/15785 Moreover, new optional packages not supported by PySide are now supported by PySide2, including "Qt5Designer", "Qt5WebChannel", "Qt5WebEngineWidgets", and "Qt5WebSockets". Corresponding USE flags should probably be added -- particularly for "Qt5WebEngineWidgets", which requires Google Chrom[e|ium] and is thus *VERY* heavyweight. The complete list of optional and mandatory Qt5 dependencies for PySide2 is specified on lines 117 through of 142 of the "PySide2/CMakeLists.txt" file. Each call to CHECK_PACKAGE_FOUND() whose second option is "opt" indicates an optional dependency; each call to CHECK_PACKAGE_FOUND() passed no such option indicates a mandatory dependency. For example: * "CHECK_PACKAGE_FOUND(Qt5Gui)" indicates the "Qt5Gui" package to be mandatory. * "CHECK_PACKAGE_FOUND(Qt5WebSockets)" indicates the "Qt5WebSockets" package to be optional. Ignoring these new optional dependencies, I can confirm that the following procedure suffices to re-enable USE flag support for PySide2. In the src_configure() phase: * Comment out the following lines, which now appear to be mandatory: -DDISABLE_QtGui=$(usex !X) -DDISABLE_QtTest=$(usex !X) -DDISABLE_QtSql=$(usex !sql) * Rewrite all remaining lines: # From this form... -DDISABLE_QtGui=$(usex !X) # Into this form. -DCMAKE_DISABLE_FIND_PACKAGE_Qt5Gui=$(usex !X) No patches to any "CMakeLists.txt" file are needed, which is nice. Significant changes to our PySide2 ebuild will be required to fully support all available dependencies, which is not quite so nice. Welcome to Gentoo. For my part, I'd be happy to submit a GitHub pull request implementing the above changes -- ignoring the new optional and mandatory dependencies detailed above, which aren't particularly low-hanging fruit. My altruism only extends so far. Thanks again! Let's right this rickety PySide2 ship. EDIT: *UGH.* The src_install() phase fails with "!!! dodoc: ChangeLog does not exist". Fortunately, this should be trivial to fix. My pull request should address this as well.
The pull request is live: https://github.com/gentoo/qt/pull/142 There is hope for us yet.
First of all, thanks for your detailed report and the pull request! > [[[ ISSUE 1 ]]] Packaging Nomenclature So pyside1 and pyside2 can be installed side-by-side... that's great news! Any reason why you renamed the whole package (changed PN) instead of slotting it (e.g. SLOT=2)? > [[[ ISSUE 2 ]]] Dependency Versioning > The PySide2 ebuild currently requires a minimum of Qt >= 4.8.5. (See the ${QT_PV} global defined by "pyside-9999.ebuild".) > [...] > [[[ ISSUE 3 ]]] Everything Is Broken, I Have No More Tears to Cry > [...] Note that there haven't been any efforts towards packaging pyside2 (or updating the current packaging for pyside2) so far. So whatever is in {shiboken,pyside}-9999 is still there since the 1.x times. Nobody is expecting it to work. In fact, we should have removed those live ebuilds long ago.
> First of all, thanks for your detailed report and the pull request! Hey, thank you! I briefly contemplated just pushing these ebuilds to my own overlay (https://github.com/leycec/raiagent) before admitting to myself that PySide2 deserved a better home. What better home than... the Qt overlay? > So pyside1 and pyside2 can be installed side-by-side... that's great new! It is pretty great. PySide2 has come a long way since its humble GitHub beginnings, including backported support for Python 2.7 and a stable release only months away. Yay! > Any reason why you renamed the whole package (changed PN) instead of slotting it (e.g. SLOT=2)? Absolutely... none. You got me. I was unsure which packaging scheme is preferred by modern Gentoo developers. For orthogonality with "dev-python/PyQt4" and "dev-python/PyQt5" (and other ebuilds I'd contributed to in the past, including "media-libs/libsdl" and "media-libs/libsdl2"), I clumsily opted to rename rather than reslot the existing "dev-python/pyside" and "dev-python/shiboken" packages. You're right, though. That's exactly what slots are for. I'll get these packages renamed back to their original names with a new "SLOT=2". > So whatever is in {shiboken,pyside}-9999 is still there since the 1.x times. I thought so too, initially. But the "shiboken-9999" ebuild at least does appear to have been explicitly modified to support Shiboken2, including requiring Qt5 dependencies and installing a "shiboken2.pc" file. Which was great. That ebuild mostly worked as is without modification. The "pyside-9999" ebuild, however... *shakes head and fists* > In fact, we should have removed those live ebuilds long ago. Oh, noes! I'm really glad you kept them. Now we can build on all that hard work. This is the culmination of decades of dedication. It was all leading up to this. When a stable release of PySide2 finally lands, the Qt overlay will be ready.
https://gitweb.gentoo.org/proj/qt.git/commit/?id=a03f2e5c1d461e828e3b36b0f94a036695d04374
