Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 644748 - [Prefix/Cygwin] Qt5 (5.9.3) Cygwin support patches
Summary: [Prefix/Cygwin] Qt5 (5.9.3) Cygwin support patches
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Other
: Normal normal (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-16 11:50 UTC by Sven Eden
Modified: 2023-05-25 20:08 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Fix handling of Q_OS_CYGWIN (qtcore-5.9.3-fix_build_on_cygwin_prefix.patch,6.31 KB, patch)
2018-01-16 12:04 UTC, Sven Eden
Details | Diff
Fixed patch to make qtcore-5.9.3 compile on cygwin (qtcore-5.9.3-fix_build_on_cygwin_prefix.patch,6.31 KB, patch)
2018-01-16 14:50 UTC, Sven Eden
Details | Diff
qtcore-5.9.3-fix_build_on_cygwin_prefix.patch Version 3 (qtcore-5.9.3-fix_build_on_cygwin_prefix.patch,7.92 KB, patch)
2018-01-17 07:49 UTC, Sven Eden
Details | Diff
Update for the Qt-5.9.3 build system (qt-5.9.3-fix_build_system_on_cygwin_general.patch,9.41 KB, patch)
2018-01-19 11:53 UTC, Sven Eden
Details | Diff
QtCore specific patches to build and install on cygwin (qtcore-5.9.3-fix_build_on_cygwin_prefix.patch,11.90 KB, patch)
2018-01-19 11:54 UTC, Sven Eden
Details | Diff
Fix portage/eclass/qt5-build.eclass to support cygwin (fix_qt5-build.eclass_for_cygwin.patch,2.72 KB, patch)
2018-01-19 12:00 UTC, Sven Eden
Details | Diff
Fix portage/eclass/qt5-build.eclass to support cygwin v2 (fix_qt5-build.eclass_for_cygwin.patch,2.72 KB, patch)
2018-01-23 10:34 UTC, Sven Eden
Details | Diff
dev-qt/qtchooser-0_p20170803 cygwin support patch (qtchooser-0_p20170803-fix_build_on_cygwin_prefix.patch,287 bytes, patch)
2018-01-23 10:42 UTC, Sven Eden
Details | Diff
QtGui specific patches to build and install on cygwin (qtgui-5.9.3-fix_build_on_cygwin_prefix.patch,6.57 KB, patch)
2018-01-23 10:44 UTC, Sven Eden
Details | Diff
QtNetwork specific patches to build and install on cygwin (qtnetwork-5.9.3-fix_build_on_cygwin_prefix.patch,2.31 KB, patch)
2018-01-23 10:47 UTC, Sven Eden
Details | Diff
Fix portage/eclass/qt5-build.eclass to support cygwin v2 (the true patch) (fix_qt5-build.eclass_for_cygwin.patch,2.74 KB, patch)
2018-01-25 11:13 UTC, Sven Eden
Details | Diff
QtGui specific patch to handle situations where SHM is not supported. (qtgui-5.9.3-fix_shm_not_supported.patch,1.84 KB, patch)
2018-02-23 09:38 UTC, Sven Eden
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sven Eden 2018-01-16 11:50:28 UTC
Due to some "misunderstanding" within the qt5 sources about what cygwin is and what it isn't, qtcore-5.9.3 can not be compiled from within a gentoo prefix on cygwin.

Reproducible: Always

Steps to Reproduce:
1. emerge =dev-qt/qtcore-5.9.3

Actual Results:  
Portage fails in compile phase

Expected Results:  
Portage merges qtcore

I am not entirely sure that this is not a gentoo prefix specific problem, so I'll post my solution here.
 ( Yes. I have successfully merged qtcore-5.9.3 )

Cygwin is not windows. Unfortunately the qt devs handle it like it was, so the compilation for Q_OS_CYGWIN is prepared like it was Q_OS_WIN or Q_OS_WINRT.

The correct handling would be somewhere between Q_OS_LINUX and Q_OS_ANDROID. Or in other words, Q_OS_CYGWIN is like Q_OS_LINUX, but it lacks some linux specific features that depend on an actual linux kernel to run.
Comment 1 Sven Eden 2018-01-16 12:04:56 UTC
Created attachment 515030 [details, diff]
Fix handling of Q_OS_CYGWIN

This patch, put into $EPREFIX/etc/portage/patches/dev-qt/qtcore , allowed me to successfully merge qtcore-5.9.3

The patching, so far, consists of:
 - Do not handle Q_OS_CYGWIN where Q_OS_WIN* is handled
 - Define Q_OS_LINUX, too, when __CYGWIN__ is detected
 - Re-enable RTLD_LOCAL, this is supported by cygwin for a long time, now.
 - On Cygwin, features.h is (somehow and only sometimes?) pre-included. In
   such a case, __GNU_VISIBLE is falsly defined as 0. So this patch adds a
   block to mkspecs/cygwin-g++/qplatformdefs.h that either re-defines
   __GNU_VISIBLE to be 1, or includes features.h if it was not defined.
   (Note: features.h is included too late in cygwins unistd.h)
 - Add #include "qplatformdefs.h were neccessary and missing.
 - Do not use Linux mutexes on Cygwin (needs linux/futex.h and kernel
   support.)
 - Do not use prctl(), but pthread_setname_np() like on QNX.
 - Do not try to use statvfs64(), on cygwin64 with LARGEFILE support, this
   is still named statvfs().
 - Do not try to use inotify. (Needs linux kernel support)

That's it. With this I now have:

--------
 ~ $ uname -a
CYGWIN_NT-10.0 CHH0711 2.9.0-gentoo-r2 2018-01-15 15:57 x86_64 Cygwin
 ~ $ eix --nocolor -I -c qtcore
[I] dev-qt/qtcore (5.9.3(5/5.9)@16.01.2018): Cross-platform application development framework
--------
Comment 2 Sven Eden 2018-01-16 14:50:39 UTC
Created attachment 515034 [details, diff]
Fixed patch to make qtcore-5.9.3 compile on cygwin

I am very sorry, but when editing the patch (paths of the files changed), I seem to have deleted a seemingly empty line which was crucial for the patch to apply. Sorry about that!

This fixed version applies cleanly.
Comment 3 Sven Eden 2018-01-17 07:49:03 UTC
Created attachment 515138 [details, diff]
qtcore-5.9.3-fix_build_on_cygwin_prefix.patch Version 3

Although the previous patch already works, I have learned some more things about what is "not optimal" while working on getting qtdbus and qtgui to emerge on cygwin.

qtgui failed, because the cygwin-g++ configuration neglects all cpu feature flags.

I tried to include gcc-base.conf to add those flags (for SSE2 and beyond), but that adds other stuff that causes even the qmake bootstrap to fail.

So this newest patch adds those flags directly to the cygwin-g++ qmake.conf.

I have added those flags even for qtcore, because I found SSE2+ bits even within corelib. So while those flags might not improve things (not fully tested, yet), they definitely won't hurt.

A simple "emerge -1 qtcore" just succeeded with this patch applied.

Btw: As all qt5 packages must be patched at least a little, I'd like to know what is better:
 A) Make this a general qt5 bug with the appropriate patches, or
 B) Open a new bug for every single qt5 package, or
 C) Use B plus adding a block on a (new to create) "qt5-on-cygwin-prefix"
    tracker bug?

I am still testing my qtgui patchset, qtdbus already works. So, post the qtdbus patch here or in an individual new bug?
Comment 4 Sven Eden 2018-01-19 11:53:47 UTC
Created attachment 515302 [details, diff]
Update for the Qt-5.9.3 build system

As the patch grew larger and larger, and all qt5 packages share build system patches more or less completely, I decided to split the patches.

This is the general build system patch, based largely on the patching done by the cygwin developers themselves. All qt5 packages should apply this.

(Well, at least that is what I did, and I just successfully merged phonon.)
Comment 5 Sven Eden 2018-01-19 11:54:39 UTC
Created attachment 515304 [details, diff]
QtCore specific patches to build and install on cygwin

This is the qtcore specific part.
Comment 6 Sven Eden 2018-01-19 12:00:06 UTC
Created attachment 515306 [details, diff]
Fix portage/eclass/qt5-build.eclass to support cygwin

The qt5-build.eclass statically sets QMAKE_CFLAGS, QMAKE_CXXFLAGS and QMAKE_LFLAGS to {C,CXX,LD}FLAGS, as derived from the environment.

This is generally a good idea, so that user values set in make.conf and similar get heeded.

Unfortunately it is crucial, that the cygwin-g++ specs can set their own values. With this eclass as it is, the specs get ignored.

This patch changes the values to not being set, but the environment values to be added instead, if (and only if) it is used on Cygwin.

Another issue is, that the Qt5 build system is not organized to install .dll files somewhere else but in the library directory ($EPREFIX/usr/lib/), but on cygwin .dll files belong in $EPREFIX/usr/bin, so the eclass now moves them into /usr/bin/.

This is the last part needed to successfully build and use Qt5Core.
Comment 7 Michael Haubenwallner (RETIRED) gentoo-dev 2018-01-19 14:34:47 UTC
(In reply to Sven Eden from comment #0)
> Cygwin is not windows. Unfortunately the qt devs handle it like it was, so
> the compilation for Q_OS_CYGWIN is prepared like it was Q_OS_WIN or
> Q_OS_WINRT.
> 
> The correct handling would be somewhere between Q_OS_LINUX and Q_OS_ANDROID.
> Or in other words, Q_OS_CYGWIN is like Q_OS_LINUX, but it lacks some linux
> specific features that depend on an actual linux kernel to run.

Feels like a general problem with the Cygwin support in Qt:
Would you mind trying to sort this out with Qt upstream?

@qt: More ideas?
Comment 8 Davide Pesavento (RETIRED) gentoo-dev 2018-01-19 17:10:32 UTC
cygwin is not supported upstream, so don't expect any actions from them. Since Sven already produced several patches (great job btw!), I recommend he tries to push them upstream, against the dev branch (will probably require some rebasing). See http://wiki.qt.io/Qt_Contribution_Guidelines for more info.

(In reply to Sven Eden from comment #3)
> Btw: As all qt5 packages must be patched at least a little, I'd like to know
> what is better:
>  A) Make this a general qt5 bug with the appropriate patches, or
>  B) Open a new bug for every single qt5 package, or
>  C) Use B plus adding a block on a (new to create) "qt5-on-cygwin-prefix"
>     tracker bug?
> 
> I am still testing my qtgui patchset, qtdbus already works. So, post the
> qtdbus patch here or in an individual new bug?

I think the best course of action is to keep a single downstream (gentoo) bug to track the effort of getting your patches merged upstream. Once that's done, we can discuss the eclass-specific bits.
Comment 9 Sven Eden 2018-01-23 10:34:59 UTC
Created attachment 516034 [details, diff]
Fix portage/eclass/qt5-build.eclass to support cygwin v2

In the previous version I forgot to add $EPREFIX to the mv command. This time everything is moved like it should.

btw: This is exactly what cygwinports do, see:
https://github.com/cygwinports/qt5-base/blob/b4b3bb94c1c199739f938d44bb6c2c1a4aef84c7/qt5-base.cygport#L103
Comment 10 Sven Eden 2018-01-23 10:38:26 UTC
(In reply to Davide Pesavento from comment #8)
> cygwin is not supported upstream, so don't expect any actions from them.
> Since Sven already produced several patches (great job btw!), I recommend he
> tries to push them upstream, against the dev branch (will probably require
> some rebasing). See http://wiki.qt.io/Qt_Contribution_Guidelines for more
> info.


I will keep that in mind. I still have many packages to go, until I have my target applications running.

(In reply to Davide Pesavento from comment #8)
> I think the best course of action is to keep a single downstream (gentoo)
> bug to track the effort of getting your patches merged upstream. Once that's
> done, we can discuss the eclass-specific bits.


Alright. I have this bug renamed and will post my (final and working) patches here.

Once I am done (aas far as I can and need), I'll try to get them upstream.

Thank you very much for your answers everybody!
Comment 11 Sven Eden 2018-01-23 10:42:41 UTC
Created attachment 516036 [details, diff]
dev-qt/qtchooser-0_p20170803 cygwin support patch

This just adds -D_GNU_SOURCE to the CXXFLAGS. Updating the Qt5 build system for the rest of Qt5 made this necessary.
Comment 12 Sven Eden 2018-01-23 10:44:43 UTC
Created attachment 516040 [details, diff]
QtGui specific patches to build and install on cygwin

This patch adds some knowledge about Cygwin and fixes the "undefined reference" issues.
Comment 13 Sven Eden 2018-01-23 10:47:00 UTC
Created attachment 516042 [details, diff]
QtNetwork specific patches to build and install on cygwin

Mainly adds the usage of the cygwin1.dll (no other *libc present, right?) and fixes an "undefined reference" issue.
Comment 14 Sven Eden 2018-01-25 11:13:00 UTC
Created attachment 516538 [details, diff]
Fix portage/eclass/qt5-build.eclass to support cygwin v2 (the true patch)

Although I fixed the previous oversight of not using $EPREFIX when moving the cygQt5*.dll files, I forgot to update the patch. Silly me!

This time it is the right one. For sure.
Comment 15 Michael Haubenwallner (RETIRED) gentoo-dev 2018-01-25 11:41:20 UTC
Actually I'm wondering why it is necessary to move cyg*.dll files from lib/ to bin/ in the ebuild/eclass. Instead, in the end this should be done right in 'make install' already - like libtool does.
Comment 16 Sven Eden 2018-01-26 08:05:19 UTC
(In reply to Michael Haubenwallner from comment #15)
> Actually I'm wondering why it is necessary to move cyg*.dll files from lib/
> to bin/ in the ebuild/eclass. Instead, in the end this should be done right
> in 'make install' already - like libtool does.

tl;dr: Because it is not (easily) possible.

long: (Mainly for those not familiar with Windows shared libraries)
 A) DLLs are the Windows variant of shared libraries, and therefore handled as "SHLIB"s in the qt5 build system.

 B) In reality DLLs are executables, with a DllMain() (See https://tinyurl.com/hc3ayk8) entry point for initialization.

These two points do not fit. DLLs must have the executable bit set, or they can not be loaded. Thus they belong in [/usr]/[s]bin.
But being handled as libraries, the build system of Qt must put them, no matter how you configure what, and I guess that's rooted deep within the cmake entanglement, into [prefix]/lib.

---

Down the line I do not care, and would be happy if all DLLs resided in /usr/lib. 
However, libraries must be in the PATH to be found and executable, because we are on windows.
Although looking strange, the cygwin devs design decision to put shared libraries into /usr/bin is actually just consequent.
Comment 17 Michael Haubenwallner (RETIRED) gentoo-dev 2018-01-26 08:11:35 UTC
This is why I wished CMake had an option to use libtool for library handling - not only for Cygwin, but also for AIX...
Comment 18 Sven Eden 2018-02-23 09:38:05 UTC
Created attachment 520668 [details, diff]
QtGui specific patch to handle situations where SHM is not supported.

This situation can happen, if the CygServer is not running.

----

Sorry for not posting for so long. I had massive problems with my Prefix.
Although I managed to emerge both Konsole and KCalc, only the latter worked. And both spewed a lot of "Signal not found" messages out on the terminal.

This was paired with massive problems getting dbus to run properly. I eventually decided that I simply did things in the wrong order, and made my installation FUBAR. Then I started from scratch.

Now dbus is working nicely, I have Kerberos integration, Avahi, syslog-ng, Postfix and sshd running, and am rebuilding everything.

I am documenting my progress on
https://wiki.gentoo.org/wiki/Prefix/Cygwin/X ; So if anybody likes to help and to get this moved further, please do. ;-)
Comment 19 Fabian Groffen gentoo-dev 2023-05-25 20:08:47 UTC
support for Cygwin was dropped