net-libs/xulrunner-1.9.1 fails to build while calling the lex() function in ply. It doesn't appear to have a dependency on ply, even though it is obviously being used. The error occurs because the outputdir= argument was only added in ply-2.5, while ply-2.3 is the latest version in Portage. The latest version of ply is 3.1. I suggest a version bump of dev-python/ply and adding it as a dependency for xulrunner. On the other hand, I don't quite get how this package builds for anyone else, if at all, if it requires a version of a package that isn't even available in portage. Maybe there's some other issue that causes this dependency (something being rebuilt that should be supplied precompiled, maybe?) Build error: /usr/lib64/python2.6/site-packages/ply/yacc.py:70: DeprecationWarning: the md5 module is deprecated; use hashlib instead import re, types, sys, cStringIO, md5, os.path ./qsgen.py:135: DeprecationWarning: the sets module is deprecated import sets Traceback (most recent call last): File "./qsgen.py", line 1487, in <module> main() File "./qsgen.py", line 1469, in main traceable=options.enable_traceables) File "./qsgen.py", line 281, in readConfigFile parser = xpidl.IDLParser(cachedir) File "/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/work/mozilla-1.9.1/xpcom/idl-parser/xpidl.py", line 1217, in __init__ optimize=1) TypeError: lex() got an unexpected keyword argument 'outputdir' gmake[4]: *** [dom_quickstubs.h] Error 1 gmake[4]: *** Waiting for unfinished jobs.... gmake[4]: Leaving directory `/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/work/mozilla-1.9.1/js/src/xpconnect/src' gmake[3]: *** [libs] Error 2 gmake[3]: Leaving directory `/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/work/mozilla-1.9.1/js/src/xpconnect' gmake[2]: *** [libs_tier_gecko] Error 2 gmake[2]: Leaving directory `/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/work/mozilla-1.9.1' gmake[1]: *** [tier_gecko] Error 2 gmake[1]: Leaving directory `/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/work/mozilla-1.9.1' make: *** [default] Error 2 * * ERROR: net-libs/xulrunner-1.9.1 failed. * Call stack: * ebuild.sh, line 49: Called src_compile * environment, line 5306: Called _eapi2_src_compile * ebuild.sh, line 634: Called die * The specific snippet of code: * emake || die "emake failed" * The die message: * emake failed * * If you need support, post the topmost build error, and the call stack if relevant. * A complete build log is located at '/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/temp/build.log'. * The ebuild environment file is located at '/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/temp/environment'. * !!! When you file a bug report, please include the following information: GENTOO_VM=sun-jdk-1.6 CLASSPATH="" JAVA_HOME="/opt/sun-jdk-1.6.0.13" JAVACFLAGS="-source 1.4 -target 1.4" COMPILER="" emerge --info: Portage 2.2_rc33 (default/linux/amd64/2008.0/desktop, gcc-4.3.2, glibc-2.8_p20080602-r1, 2.6.29-gentoo-r5 x86_64) ================================================================= System uname: Linux-2.6.29-gentoo-r5-x86_64-Intel-R-_Core-TM-2_Duo_CPU_P8600_@_2.40GHz-with-gentoo-1.12.11.1 Timestamp of tree: Tue, 30 Jun 2009 19:45:02 +0000 distcc 3.1 x86_64-pc-linux-gnu [enabled] ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-java/java-config: 2.1.7 dev-lang/python: 2.5.4-r2, 2.6.2-r1 dev-python/pycrypto: 2.0.1-r8 dev-util/ccache: 2.4-r7 dev-util/cmake: 2.6.4 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.4 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-march=core2 -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y" FEATURES="ccache distcc distlocks fixpackages parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://gentoo-euetib.upc.es/mirror/gentoo/ http://ftp.udc.es/gentoo/ http://mirror.qubenet.net/mirror/gentoo/ http://mirror.ovh.net/gentoo-distfiles/ http://mirror.cambrium.nl/pub/os/linux/gentoo/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1" LINGUAS="en es" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--timeout=180" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="3dnowex 7zip X a52 aac aalib acl acpi addbookmarks additions akode alias alsa amarok amd64 amd64codecs apache2 arts artswrappersuid asf aspell atm audiofile autoreplace avahi avi bash-completion berkdb bitmap-fonts bittorrent-external blender-game bluetooth bonjour branding bzip2 bzlib cairo calendar cdda cddb cdparanoia cdr cgi chm cjk clamav cli connectionstatus consolekit contactnotes cracklib crypt css ctype cups curl cvs daap dba dbus dillo divx4linux dlloader doc dri dts dv dvb dvd dvdr dvdread eds embedded emboss encode evo examples exif faac faad fam fasttrack fbcon fbcondecor ffmpeg firefox flac flash font-server foomaticdb fortran ftp gd gdbm gif glitz gnutella gnutls gpm gps gs gsm gstreamer gtk gtk2 h323 hal hash highlight history httpd iconv id3tag ieee1394 imagemagick imap inifile inkjar innodb insecure-savers ipod ipv6 irc isdnlog ithreads jabber jack jadetex java javascript jingle joystick jpeg jpeg2k kde kpathsea kqemu ladspa laptop libcaca libnotify libsamplerate libv4l2 lirc lisamplerate live lm_sensors logrotate mad maildir matroska mbox mbrola mdnsresponder-compat messenger midi mikmod mjpeg mmx mmxext mng mod modplug moviepack moviesounds mozbranding mozcalendar mozdevelop mozilla mozsvg mozxmlterm mp3 mp4 mpeg mpeg4 mplayer msn mudflap multilib musicbrainz mysql mysqli mythtv ncurses network nls nodrm nowlistening nptl nptlonly nsplugin nvidia offensive ogg oggvorbis openct openexr opengl openmp pam pcre pcsc-lite pdf perl plasma plotutils png posix ppds pppd protutils python qt qt3 qt3support qt4 quicktime rar readline reflection rt2500pci rtc rtsp samba scanner sdk sdl session shout skins slang sndfile soap sockets sox speex spell spl sql sqlite srt sse sse2 sse3 ssl ssse3 startup-notification statistics stream subtitles subversion svg swat sysfs taglib tcltk tcpd test-programs texteffect tga theora threads tiff tk transmitter truetype truetype-fonts twolame type1-fonts udev unicode usb utempter utf8 v4l v4l2 vdpau vim-syntax visualization vlm vorbis webkit webpresence wifi wininst wmf wxwindows x264 xcb xcomposite xforms xine xinerama xml xmlreader xmlrpc xmlwriter xorg xosd xprint xscreensaver xsl xulrunner xv xvid yahoo yv12 zeroconf zip zlib zvbi" ALSA_CARDS="usb-audio virmidi loopback hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CAMERAS="canon" ELIBC="glibc" INITNG_PLUGINS="also bash_launcher chdir chroot conflict cpout critical ctrlaltdel daemon_clean debug_commands envparser find fmon fstat history idleprobe initctl interactive iparser last limit lockfile logfile netdev netprobe ngc4 ngcs nge pause provide reload renice rlparser simple_launcher stcmd stdout suid syncron syslog sysreq unneeded usplash" INPUT_DEVICES="evdev keyboard mouse wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en es" LIRC_DEVICES="serial devinput" USERLAND="GNU" VIDEO_CARDS="fbdev nv nvidia vesa" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
Confirmed that it builds fine if I bump ply to 2.5.
Ply is not even a dep for xulrunner, you are suffering from a broken python package. Python herd needs to look at this.
xulrunner very clearly uses ply (or has the ability to use it, which I'm somehow triggering). Notice the file that causes the error: [...]/work/mozilla-1.9.1/xpcom/idl-parser/xpidl.py, very clearly in the xulrunner tree. It's a parser, which is precisely what ply is about, and it imports ply at the top of the file. Python is fine. If there is an issue that I'm hitting and others aren't it's some kind of xulrunner build screwup where that xpidl.py is being called unnecessarily.
(In reply to comment #3) > xulrunner very clearly uses ply (or has the ability to use it, which I'm > somehow triggering). Notice the file that causes the error: > [...]/work/mozilla-1.9.1/xpcom/idl-parser/xpidl.py, very clearly in the > xulrunner tree. It's a parser, which is precisely what ply is about, and it > imports ply at the top of the file. > > Python is fine. If there is an issue that I'm hitting and others aren't it's > some kind of xulrunner build screwup where that xpidl.py is being called > unnecessarily. > /usr/lib64/python2.6/site-packages/ply/yacc.py:70: DeprecationWarning: the md5 module is deprecated; use hashlib instead import re, types, sys, cStringIO, md5, os.path ./qsgen.py:135: DeprecationWarning: the sets module is deprecated import sets Traceback (most recent call last): File "./qsgen.py", line 1487, in <module> main() File "./qsgen.py", line 1469, in main traceable=options.enable_traceables) File "./qsgen.py", line 281, in readConfigFile parser = xpidl.IDLParser(cachedir) File "/mnt/tmp/var/tmp/portage/net-libs/xulrunner-1.9.1/work/mozilla-1.9.1/xpcom/idl-parser/xpidl.py", line 1217, in __init__ optimize=1) TypeError: lex() got an unexpected keyword argument 'outputdir' gmake[4]: *** [dom_quickstubs.h] Error 1 The traceback if from python failing. There is no dep on ply, if there was I would have ensured it was added to the deps long before we ever got to final.
(In reply to comment #4) > The traceback if from python failing. There is no dep on ply, if there was I > would have ensured it was added to the deps long before we ever got to final. The traceback is from a python script that comes from xulrunner failing. It fails because it indeed uses ply, and because it indeed requires a feature that is not present in the sole version of ply available in Portage. Here's some information about why the xpidl parser was reimplemented in python using (surprise) Ply: http://benjamin.smedbergs.us/blog/2008-10-07/parsing-idl-in-python/ From the mozilla-1.9.1 tree: mozilla-1.9.1/xpcom/idl-parser/xpidl.py Here's the problem: mozilla bundles its own ply but _appends_ it to the python search path. This means that any system-installed ply takes precedence. If it is older than 2.5, build fails. From the top of xpidl.py: ========================= import sys, os.path, re sys.path.append(os.path.join(os.path.dirname(__file__), os.pardir, os.pardir, 'other-licenses', 'ply')) from ply import lex, yacc ========================= If they want to ensure they use their ply and not whatever ply the user has installed on their system, they need to change that to sys.path.insert(0,os.path...).
(In reply to comment #5) > (In reply to comment #4) > > The traceback if from python failing. There is no dep on ply, if there was I > > would have ensured it was added to the deps long before we ever got to final. > > The traceback is from a python script that comes from xulrunner failing. It > fails because it indeed uses ply, and because it indeed requires a feature that > is not present in the sole version of ply available in Portage. > > Here's some information about why the xpidl parser was reimplemented in python > using (surprise) Ply: > http://benjamin.smedbergs.us/blog/2008-10-07/parsing-idl-in-python/ > > From the mozilla-1.9.1 tree: > mozilla-1.9.1/xpcom/idl-parser/xpidl.py > > Here's the problem: mozilla bundles its own ply but _appends_ it to the python > search path. This means that any system-installed ply takes precedence. If it > is older than 2.5, build fails. From the top of xpidl.py: > > ========================= > import sys, os.path, re > > sys.path.append(os.path.join(os.path.dirname(__file__), os.pardir, os.pardir, > 'other-licenses', 'ply')) > from ply import lex, yacc > ========================= > > If they want to ensure they use their ply and not whatever ply the user has > installed on their system, they need to change that to > sys.path.insert(0,os.path...). > If you would write up the patch and attach it to the bug and I will test it, and get it commited to the main tree.
Created attachment 196221 [details, diff] Make xpidl compiler always prefer bundled Ply over system Ply Tested working with ply-2.3 from portage. The (unrelated) DeprecationWarning confirms in the file path that now the bundled Ply is being used.
https://bugzilla.mozilla.org/show_bug.cgi?id=501657 Patch is fine Nirbheek, go ahead and commit it. I spoke to Ben and his response is included. <bsmedberg> Anarchy: huh, that's interesting. Yes, please file it <Anarchy> bsmedberg, will do :) <Anarchy> bsmedberg, was found when ply-2.3 is installed on system cause build to blow up nicely :) <bsmedberg> yeah <bsmedberg> I never figured there would be a system ply
(In reply to comment #7) > Created an attachment (id=196221) [edit] > Make xpidl compiler always prefer bundled Ply over system Ply > > Tested working with ply-2.3 from portage. The (unrelated) DeprecationWarning > confirms in the file path that now the bundled Ply is being used. > Hector thanks for attaching the patch. I have include all your info in the patch that is sitting upstream so you can get proper credit for the bug find and fix.
Added to tree w/o a bump, thanks for reporting :) Using commit message: ------------------------------------------------------------------------------ Fix building when dev-python/ply is installed, bug 276018. Patch thanks to Hector Martin (Portage version: 2.2_rc33/cvs/Linux i686) ------------------------------------------------------------------------------
