>>> mail-client/thunderbird-8.0-r1 merged. Traceback (most recent call last): File "/usr/lib64/portage/pym/portage/dbapi/_MergeProcess.py", line 246, in _spawn prev_mtimes=self.prev_mtimes, counter=counter) File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 4321, in merge self._elog_process() File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 3145, in _elog_process funcname = funcnames[key] KeyError: u"':" >>> Failed to install mail-client/thunderbird-8.0-r1, Log file: >>> '/var/log/portage/build/mail-client/thunderbird-8.0-r1:20111117-130825.log' * Messages for package mail-client/thunderbird-8.0-r1: * QA Notice: command not found: * * ./db/makefiles.sh: line 38: $'\r': command not found ': command not foundfiles.sh: line 38: $' * QA Notice: command not found: * * ./db/makefiles.sh: line 38: $'\r': command not found ': command not foundfiles.sh: line 38: $' * * The following package has failed to build or install: * * (mail-client/thunderbird-8.0-r1::gentoo, ebuild scheduled for merge), Log file: * '/var/log/portage/build/mail-client/thunderbird-8.0-r1:20111117-130825.log' * I have no clue where the dbapi problem comes from. @mozilla And also I don't see the problem with the command not found. $ einfo =mail-client/thunderbird-8.0-r1 Portage 2.2.0_alpha75 (default/linux/amd64/10.0, gcc-4.6.2-asneeded, glibc-2.13-r4, 3.1.1-lh x86_64) ================================================================= System Settings ================================================================= System uname: Linux-3.1.1-lh-x86_64-Intel-R-_Core-TM-_i7_CPU_860_@_2.80GHz-with-gentoo-2.1 Timestamp of tree: Thu, 17 Nov 2011 11:00:01 +0000 ccache version 3.1.6 [enabled] app-shells/bash: 4.2_p10 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.5.4-r4, 2.6.7-r2, 2.7.2-r3, 3.1.4-r3, 3.2.2 dev-util/ccache: 3.1.6 dev-util/cmake: 2.8.6-r3 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1 sys-apps/openrc: 0.9.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1-r1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.2.4-r1, 4.3.6-r1, 4.4.6-r1, 4.5.3-r1, 4.6.2 sys-devel/gcc-config: 1.5-r2 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 2.6.39 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo sunrise bicatali betagarden dummy neurogeek science last-hope g-ctan Installed sets: ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -ftracer -march=corei7 -mcx16 -msahf -mpopcnt -msse4.2 -mtune=corei7 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=corei7 -frecord-gcc-switches -g -Wimplicit-function-declaration" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/eselect/ /usr/share/gnupg/qualified.txt /usr/share/nano/ /usr/share/openvpn/easy-rsa /var/lib/hsqldb /var/spool/torque" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -ftracer -march=corei7 -mcx16 -msahf -mpopcnt -msse4.2 -mtune=corei7 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=corei7 -frecord-gcc-switches -g -Wenum-compare" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="-t --jobs=12 --load-average=12 --keep-going --autounmask-write --autounmask --quiet-build=n" FEATURES="assume-digests binpkg-logs buildsyspkg ccache collision-protect distlocks ebuild-locks fixlafiles metadata-transfer multilib-strict news noinfo parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms sign split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe -ftracer -march=corei7 -mcx16 -msahf -mpopcnt -msse4.2 -mtune=corei7 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=corei7 -frecord-gcc-switches -g" GENTOO_MIRRORS=" /mnt/tmpfs/ http://gentoo.j-schmitz.net/mirror/ ftp://ftp.gentoo.mesh-solutions.com/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common" LINGUAS="en" MAKEOPTS="-j16 -l12" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="xz" PORTAGE_COMPRESS_FLAGS="-z -e -9 -v" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/data/local/sunrise/reviewed /data/local/bicatali /data/local/betagarden /data/local/dummy /data/local/neurogeek /data/local/sci /data/local/lh/ebuilds /data/local/g-ctan" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="64bit 7zip X acpi additions alsa amd64 arpwarp atlas bash-completion berkdb blas branding bzip2 c++ cairo caps cblas ccache cleartype cli command-args consolekit corefonts cracklib crypt cuda cups cupsddk custom-optimization cxx dbus dri dts exif fbcondecor fortran gdbm gif glib glibc-omitfp gmp gnome gnome-keyring gpm graphics gstreamer gtk hddtemp iconv icu ios ipod iproute2 iptables ipv6 ipython javascript jpeg jpeg2k kqemu lapack largefile lcms ldap libnotify libsexy lm_sensors lzma mailtrain md5sum mmx mmxext mng modules mp3 mudflap multilib multiuser nagios-dns nagios-ntp nagios-ping nagios-ssh nano-syntax ncurses network-cron nis nls nptl nptlonly nsplugin objc objc++ opencl opengl openmp openntpd pam pcre pdf png policykit pppd pymol qt-static qt3support readline rrdcgi rrdtool science sensord session smp sqlite sqlite3 sse sse2 sse3 sse4 sse4a sse5 ssl startup-notification svg sysfs system-sqlite tcpd threads tiff truetype type1 udev unicode vaapi vdpau x264 xcb xcomposite xinerama xorg xulrunner zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= mail-client/thunderbird-8.0-r1 was built with the following: USE="alsa crashreporter crypt custom-optimization dbus gconf ipc libnotify lightning (multilib) startup-notification system-sqlite webm -bindist -custom-cflags -debug -mozdom -wifi" LINGUAS="-ar -be -bg -bn -bn_BD -br -ca -cs -da -de -el -en -en_GB -en_US -es -es_AR -es_ES -et -eu -fi -fr -fy -fy_NL -ga -ga_IE -gd -gl -he -hu -id -is -it -ja -ko -lt -nb -nb_NO -nl -nn -nn_NO -pa -pa_IN -pl -pt -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -sq -sv -sv_SE -ta -ta_LK -tr -uk -vi -zh -zh_CN -zh_TW" CFLAGS="-pipe -march=corei7 -mtune=corei7 -mtune=corei7 -g -Wimplicit-function-declaration -Wno-return-type -w -mno-avx" CXXFLAGS="-pipe -march=corei7 -mtune=corei7 -mtune=corei7 -g -Wenum-compare -Wno-return-type -w -mno-avx"
Created attachment 292891 [details] /var/log/portage/build/mail-client/thunderbird-8.0-r1:20111117-130825.log.xz build.log
(In reply to comment #0) > I have no clue where the dbapi problem comes from. It looks like a bug in portage's elog code. > @mozilla > And also I don't see the problem with the command not found. Looks like a bug in ./db/makefiles.sh
(In reply to comment #2) > (In reply to comment #0) > > I have no clue where the dbapi problem comes from. > > It looks like a bug in portage's elog code. > > > @mozilla > > And also I don't see the problem with the command not found. > > Looks like a bug in ./db/makefiles.sh The file does not exist, I will look into it deeper.
(In reply to comment #0) > >>> mail-client/thunderbird-8.0-r1 merged. > Traceback (most recent call last): > File "/usr/lib64/portage/pym/portage/dbapi/_MergeProcess.py", line 246, in > _spawn > prev_mtimes=self.prev_mtimes, counter=counter) > File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 4321, in merge > self._elog_process() > File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 3145, in > _elog_process > funcname = funcnames[key] > KeyError: u"':" This should prevent the above error: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=87cd5dba474f758e291b87a39366bd7edd748bf7 It seems to be triggered by a corrupt elog line, so it will also output the content of the line in order to help diagnose the source of corruption.
Now I get: >>> mail-client/thunderbird-8.0-r1 merged. !!! malformed entry in log file: '/var/tmp/portage/mail-client/thunderbird-8.0-r1/temp/logging/install': ': command not found
(In reply to comment #5) > Now I get: > > >>> mail-client/thunderbird-8.0-r1 merged. > !!! malformed entry in log file: > '/var/tmp/portage/mail-client/thunderbird-8.0-r1/temp/logging/install': ': > command not found I'm not sure what can cause this. The log file is written by the following code from the elog_base function in /usr/lib/portage/bin/isolated-functions.sh: echo -e "$@" | while read -r ; do echo "$messagetype $REPLY" >> \ "${T}/logging/${EBUILD_PHASE:-other}" done The 'while read -r' loop is supposed to format the log file such that each line starts with a valid log level (INFO|WARN|ERROR|LOG|QA). This is the first report that we've had of a failure in this area for years. Maybe the $'\r' shown in comment #0 is causing the code to misbehave somehow, but I don't understand exactly how. I know that bash chokes on scripts that are have \r in line endings, so maybe it triggers other kinds of problems too. For example, maybe bash's 'read' builtin misbehaves when it reads a \r character.
This test case seems so show that read -r handles \r without any trouble: echo -e "abc\rdef\nabc\rdef" | while read -r ; echo "$REPLY" >> test.txt ; done hexdump shows the output matches the input (after expansion by echo -e, and accounting for little-endian byte order): hexdump test.txt 0000000 6261 0d63 6564 0a66 6261 0d63 6564 0a66
It turns out python's io.open() stream splits lines on \r, so I've fixed it to explicitly split on \n now: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=935694debc955a18a6a93af923f3f98caacf3f74
This is fixed in 2.1.10.36 and 2.2.0_alpha76.
