Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 277711

Summary: dev-python/PyQt4 fails to build with multilib-portage
Product: Gentoo Linux Reporter: Thomas Sachau <tommy>
Component: New packagesAssignee: Qt Bug Alias <qt>
Status: RESOLVED WONTFIX    
Severity: normal CC: ftobin, python
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: build.log for PyQt4-4.5.1
multilib header file with inclusion depending on current ABI

Description Thomas Sachau gentoo-dev 2009-07-13 20:53:51 UTC
This may be related to my multilib setup, but since i am not familiar with this package, i hope, someone familiar with it can help me to fix this one.

emerge --info:
Portage 2.2_rc33-r2 (unavailable, gcc-4.4.0, glibc-2.9_p20081201-r5, 2.6.28-hardened-r7 x86_64)
=================================================================
System uname: Linux-2.6.28-hardened-r7-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q6600_@_2.40GHz-with-gentoo-2.0.1
Timestamp of tree: Mon Jul 6 07:44:38 CEST 2009
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p24
dev-java/java-config: 2.1.8-r1
dev-lang/python:     2.6.2-r1
dev-python/pycrypto: 2.0.1-r8
dev-util/ccache:     2.4-r8
dev-util/cmake:      2.6.4
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.4.3-r3
sys-apps/sandbox:    2.0
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.4_p6, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils:  2.18-r4
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.30
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=nocona -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-march=nocona -O2 -pipe"
DISTDIR="/home/thomas/daten/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going"
FEATURES="assume-digests autoconfig ccache collision-protect distlocks fixpackages metadata-transfer parallel-fetch preserve-libs protect-owned sandbox sfperms sign strict unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="ftp://mirror.cambrium.nl/pub/os/linux/gentoo/ http://mirror.muntinternet.net/pub/gentoo/"
INSTALL_MASK="/usr/share/gtk-doc"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,--as-needed"
LINGUAS="de"
MAKEOPTS="-j5 --load-average=8"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/toolchain-overlay /usr/local/portage/layman/toolchain-overlay-testing /usr/local/portage/layman/multilib-overlay /usr/local/portage/layman/sunrise /usr/local/portage/layman/java-overlay /usr/local/portage/layman/java-experimental /usr/local/portage/layman/enlightenment /usr/local/portage"
SYNC="cvs://tommy@cvs.gentoo.org:/var/cvsroot"
USE="3dnow X alsa amd64 berkdb cracklib crypt cups custom-cflags custom-cxxflags gpm hardened justify midi mmx multilib ncurses nls nptl nptlonly nsplugin ogg opengl openmp pam pic readline scanner sse sse2 ssl sysfs tcpd unicode urandom vorbis xorg zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de" USERLAND="GNU" VIDEO_CARDS="nvidia"
Unset:  CPPFLAGS, CTARGET, FFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Thomas Sachau gentoo-dev 2009-07-13 20:54:32 UTC
Created attachment 197840 [details]
build.log for PyQt4-4.5.1
Comment 2 Davide Pesavento gentoo-dev 2009-07-14 22:30:45 UTC
Are you sure your qt4 headers are correctly installed? In particular, do you have /usr/include/qt4/QtGui/QTextObjectInterface ?
Comment 3 Dror Levin (RETIRED) gentoo-dev 2009-07-16 18:10:12 UTC
Looks like bug 277985, no?
Comment 4 Thomas Sachau gentoo-dev 2009-07-16 18:43:21 UTC
(In reply to comment #3)
> Looks like bug 277985, no?
> 

No, that bug is only for crosscompile of distutils packages while this one is about normal compile of PyQt4 on amd64, no crosscompile.

How does PyQt4 access the header files? Since i found out that it only happens with a special header file construction for multilib for qt-gui, it has do to something with the way, how PyQt4 does include the header.
Comment 5 Thomas Sachau gentoo-dev 2009-07-16 18:46:09 UTC
Created attachment 198221 [details]
multilib header file with inclusion depending on current ABI
Comment 6 Davide Pesavento gentoo-dev 2009-07-16 21:13:00 UTC
The __i386__ and __x86_64__ macros, which are implicitly defined by gcc's preprocessor, are not defined by moc: thus both include directives are discarded and when moc later sees QTextObjectInterface it complains it cannot find its definition (because the header wasn't included).

Adding -D__i386__ or -D__x86_64__ (depending on the current ABI) to moc's command line should fix the issue.

Anyway, I don't know why QTextObject is treated as a multilib header, since I don't think it depends on the current ABI... Does multilib portage create a wrapper for every header file?
Comment 7 Thomas Sachau gentoo-dev 2009-07-17 14:26:08 UTC
(In reply to comment #6)
> The __i386__ and __x86_64__ macros, which are implicitly defined by gcc's
> preprocessor, are not defined by moc: thus both include directives are
> discarded and when moc later sees QTextObjectInterface it complains it cannot
> find its definition (because the header wasn't included).
> 
> Adding -D__i386__ or -D__x86_64__ (depending on the current ABI) to moc's
> command line should fix the issue.
> 
> Anyway, I don't know why QTextObject is treated as a multilib header, since I
> don't think it depends on the current ABI... Does multilib portage create a
> wrapper for every header file?
> 

multilib portage does save the headers for 32bit and 64bit phase and then does a diff between them and if they differ, it does install both headers and a wrapper for each header file of that package.
Comment 8 Markos Chandras (RETIRED) gentoo-dev 2009-07-17 14:43:41 UTC
Thomas I removed 4.5.1 from tree since there is 4.5.2 on tree already which is considered as a bug fix release. Did you try to build this one?

I am adjusting the title accordingly
Comment 9 Davide Pesavento gentoo-dev 2009-07-17 15:07:46 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > The __i386__ and __x86_64__ macros, which are implicitly defined by gcc's
> > preprocessor, are not defined by moc: thus both include directives are
> > discarded and when moc later sees QTextObjectInterface it complains it cannot
> > find its definition (because the header wasn't included).
> > 
> > Adding -D__i386__ or -D__x86_64__ (depending on the current ABI) to moc's
> > command line should fix the issue.
> > 
> > Anyway, I don't know why QTextObject is treated as a multilib header, since I
> > don't think it depends on the current ABI... Does multilib portage create a
> > wrapper for every header file?
> > 
> 
> multilib portage does save the headers for 32bit and 64bit phase and then does
> a diff between them and if they differ, it does install both headers and a
> wrapper for each header file of that package.
> 

So this means that if there exists at least one header file which is ABI-dependent, the whole set of headers (even ABI-independent ones) for that package gets wrapped by multilib portage?
Comment 10 Thomas Sachau gentoo-dev 2009-07-17 15:22:24 UTC
(In reply to comment #8)
> Thomas I removed 4.5.1 from tree since there is 4.5.2 on tree already which is
> considered as a bug fix release. Did you try to build this one?
> 
> I am adjusting the title accordingly
> 

same result with 4.5.2

(In reply to comment #9)
> (In reply to comment #7)
> > (In reply to comment #6)
> > > The __i386__ and __x86_64__ macros, which are implicitly defined by gcc's
> > > preprocessor, are not defined by moc: thus both include directives are
> > > discarded and when moc later sees QTextObjectInterface it complains it cannot
> > > find its definition (because the header wasn't included).
> > > 
> > > Adding -D__i386__ or -D__x86_64__ (depending on the current ABI) to moc's
> > > command line should fix the issue.
> > > 
> > > Anyway, I don't know why QTextObject is treated as a multilib header, since I
> > > don't think it depends on the current ABI... Does multilib portage create a
> > > wrapper for every header file?
> > > 
> > 
> > multilib portage does save the headers for 32bit and 64bit phase and then does
> > a diff between them and if they differ, it does install both headers and a
> > wrapper for each header file of that package.
> > 
> 
> So this means that if there exists at least one header file which is
> ABI-dependent, the whole set of headers (even ABI-independent ones) for that
> package gets wrapped by multilib portage?
> 

That is, how it currently works, yes.
Comment 11 Davide Pesavento gentoo-dev 2009-07-18 00:19:00 UTC
Another problem: qt version checks like

#if QT_VERSION (< or >=) 0xfoobar
    ...some stuff...
#endif

seem to be broken.

(btw, can someone change the summary to reflect that all these issues are caused by headers wrapping in multilib setups?)
Comment 12 Davide Pesavento gentoo-dev 2009-07-18 18:14:53 UTC
I believe that *every* Qt4-based app using moc is currently broken on multilib setups.
moc has very limited preprocessing capabilities, as noted in the docs, and the header wrapping performed by multilib portage is deliberately violating those restrictions.

I see 3 possible solutions:

(1) add -D$(get_abi_CDEFINE) to every moc invocation: this can be easy or very hard depending on the build system; eqmake4 should be changed to pass DEFINES+=$(get_abi_CDEFINE) to qmake, which will fix most qmake-based packages, but won't fix PyQt4 for example. (IMHO this solution sucks because it's not general)

(2) report the issue upstream and ask them to change moc to automatically define some platform macros like gcc does. (probably the best solution, but it could take a lot of time before upstream fixes it)

(3) declare this bug INVALID and design a better solution for multilib headers handling. (note: a better solution may not exist :P)
Comment 13 Davide Pesavento gentoo-dev 2009-09-02 13:22:23 UTC
I just noticed the following in PMS section 12.4:

"Each phase function must be called at most once during the build process for any given package."

It seems to me that multilib portage is violating this requirement. Please correct me if I'm wrong.
Comment 14 Thomas Sachau gentoo-dev 2009-09-02 15:14:44 UTC
(In reply to comment #13)
> I just noticed the following in PMS section 12.4:
> 
> "Each phase function must be called at most once during the build process for
> any given package."
> 
> It seems to me that multilib portage is violating this requirement. Please
> correct me if I'm wrong.
> 

This part may have to be modified for multilib support.

Basicly, multilib-portage does call each phase once, but for every ABI. If you want to enforce this part on multilib-aware package managers, you would have to introduce something different like another var, which behaves similar to the current SLOT.
Comment 15 Davide Pesavento gentoo-dev 2009-09-02 15:28:37 UTC
Thanks for the explanation.
I was just pointing out that the current multilib implementation can be very intrusive wrt. ebuilds and eclasses, because some of them rely upon that assumption, and fail when it isn't respected.
Comment 16 Markos Chandras (RETIRED) gentoo-dev 2009-10-16 14:19:13 UTC
Are there any recent updates on this bug?
Comment 17 Thomas Sachau gentoo-dev 2009-10-16 14:44:02 UTC
(In reply to comment #16)
> Are there any recent updates on this bug?
> 

The specific bug itself still exists, but does not show up with the current implementation. But there are other bugs related to multilib-portage like not respecting the current environment. Probably best to do a IRC session for debugging and fixing this mess.
Comment 18 Davide Pesavento gentoo-dev 2009-10-17 10:05:20 UTC
IMHO this bug cannot be fixed with the current multilib implementation.
The only thing to be fixed here is the handling of multilib headers, and that one is a multilib portage bug.
I'm afraid I was probably wrong in comment #12, solutions (1) and (2) wouldn't really solve anything :(
Comment 19 Davide Pesavento gentoo-dev 2010-06-25 12:20:55 UTC
Thomas, did you make any progress with this?
Comment 20 Thomas Sachau gentoo-dev 2010-08-19 16:51:48 UTC
(In reply to comment #19)
> Thomas, did you make any progress with this?
> 

From what i can see, this is no bug in multilib-portage itself, but related to the custom buildsystem of qt-related packages.

Since the headers PyQt4 currently checks are the same for amd64 and x86, the bug does not show up, but still exists there, if noone did add a workaround/fix in the background.
Comment 21 Davide Pesavento gentoo-dev 2012-06-16 13:03:20 UTC
Thomas, seeing that you proposed multilib portage for EAPI 5, it might be better to resurrect this bug and try to find a solution.

First of all, I suppose the situation hasn't changed since 2 years ago, right?
Comment 22 Thomas Sachau gentoo-dev 2012-06-17 10:24:27 UTC
As i wrote before: currently, this issue is hidden since the header files, that are used by PyQt4, dont differ between amd64 and x86, so those header files dont get wrapped.
Beside that, there is not much one can do from package manager side for custom build systems (and no, adding workarounds for issues in every custom build system found in the wild is no fix, it just hides the issues).
Comment 23 Michael Palimaka (kensington) gentoo-dev 2013-05-22 08:23:47 UTC
PyQt4 includes a new build system, which according to upstream[1], will in the future support cross-compilation.

[1]: http://www.riverbankcomputing.co.uk/news/pyqt-4101
Comment 24 Davide Pesavento gentoo-dev 2014-10-14 20:05:10 UTC
(In reply to Michael Palimaka (kensington) from comment #23)
> PyQt4 includes a new build system, which according to upstream[1], will in
> the future support cross-compilation.
> 
> [1]: http://www.riverbankcomputing.co.uk/news/pyqt-4101

>=PyQt4-4.11.2-r1 has been switched to using configure-ng.py (although I'm not sure how this is related to the original bug report)
Comment 25 Davide Pesavento gentoo-dev 2014-10-14 20:19:22 UTC
We're not going to support multilib-portage, sorry. Closing as WONTFIX (and given the moc limitations mentioned above, might even be CANTFIX).

The current approach using multilib-minimal.eclass seems to yield better results (we're only wrapping two header files for now).