Bugzilla Bug 211949

All of a sudden, Portage stopped working. Either in the beginning of a merge,
or in the end (when it is about to unmerge old versions of a package), it would
suddenly die with these kinds of messages (there are many, I'm posting only a
few):

/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 31:
Colorset-Author=fRiQ: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 32:
Colorset-Name=DarkDesktop: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 47:
Desk-Pages-X=8: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 48:
Desk-Pages-Y=1: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 66:
EWMH-BS-Desktop-Bottom=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 67:
EWMH-BS-Desktop-Left=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 68:
EWMH-BS-Desktop-Right=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 69:
EWMH-BS-Desktop-Top=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 70:
EWMH-BS-Panel-Bottom=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 71:
EWMH-BS-Panel-Left=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 72:
EWMH-BS-Panel-Right=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 73:
EWMH-BS-Panel-Top=30: command not found

As the link points out, this is a problem because Bash does not allow
environment variables having hyphens in them.

In this particular instance, I think it's having trouble not because I
currently have environment variables with hyphens (I don't at present), but
because the package was built with those variables. 

In any case, I have no idea why this behavior began happening all of a sudden.
I, like the user in the link, use FVWM-Crystal, which is the main program
setting these env variables. However, we've been using it for months (years for
me) - yet Portage only failed recently.

A solution is posted in the link, which I'm using to get by. I have no idea if
it's a "good" solution.  

Reproducible: Always

Steps to Reproduce:
1.emerge something.
2.Either fails right before configure.
3.Or fails when unmerging old versions.

Actual Results:  
Dies a miserable death.

Expected Results:  
Should merge/unmerge safely.

emerge --info output:

=================================================================
System uname: 2.6.14-gentoo-r2 i686 Intel(R) Pentium(R) 4 CPU 2.53GHz
Timestamp of tree: Mon, 18 Feb 2008 03:16:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p17-r1
dev-java/java-config: 1.3.7, 2.1.4
dev-lang/python:     2.3.5-r3, 2.4.4-r6, 2.5.1-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 1.12.10-r5
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24, 1.5.26
virtual/os-headers:  2.6.23-r3
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -O3 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf
/etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/
/etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo
/etc/texmf/web2c"
CXXFLAGS="-march=pentium4 -O3 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks fixpackages metadata-transfer nostrip parallel-fetch
sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://gentoo.mirrors.tds.net/gentoo
http://gentoo.cites.uiuc.edu/pub/gentoo/ ftp://ftp.ussg.iu.edu/pub/linux/gentoo
http://gentoo.osuosl.org/ ftp://gentoo.mirrors.tds.net/gentoo"
LANG="en_US.utf8"
LINGUAS="en ar"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --stats --timeout=180 --exclude=/distfiles
--exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/xeffects
/usr/portage/local/layman/sunrise /usr/portage/local/layman/science
/usr/portage/local/layman/roslin /usr/portage/local/layman/other"
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"
USE="7zip X a52 aac aalib acpi aim alsa apache2 apm asf avi bash-completion
berkdb bitmap-fonts bittorrent browserplugin cairo caps cdparanoia cdr cli
cracklib crypt cups cvs dbus dga directfb divx4linux doc dri dvb dvd dvdnav
dvdr dvdread eds emacs emboss encode exif fam fbcon fbdev ffmpeg fftw firefox
flac fortran ftp gd gdbm gif gimp glitz gnutella gphoto2 gpm gstreamer gtk
guile hal hbci hdri i8x0 iconv icq imagemagick imap imlib ipv6 isdnlog jabber
java javascript jpeg jpeg2k justify kde lame lapack lcms ldap leim libcaca
libsamplerate live lyx mad maildir matroska mbox md5sum midi mikmod mjpeg mmx
mozilla mp2 mp3 mpeg mplayer msn mudflap musepack mysql ncurses network nls
nptl nptlonly nsplugin oav ofx ogg oggvorbis openexr opengl openmp oscar oss
pam pcre pda pdf pdflib perl pgp php plotutils png pnm pnp ppds pppd python qt
qt3 qt4 quicktime radio rar readline real reflection rss rtsp samba sasl sdl
session shout sox speex spell spl srt sse ssl stream subtitles subversion svg
svga tcpd tetex tga theora tiff tivo transparency truetype truetype-fonts
type1-fonts unicode usb v4l vcd vesa videos vidix vorbis wavpack win32codecs
wma wmf wxwindows x264 x86 xcomposite xine xml xorg xosd xscreensaver xv xvid
yahoo zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix
dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter
mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions
alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file
authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user
autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires
ext_filter file_cache filter headers include info log_config logio mem_cache
mime mime_magic negotiation rewrite setenvif speling status unique_id userdir
usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse"
KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001
mtxorb ncurses text" LINGUAS="en ar" USERLAND="GNU" VIDEO_CARDS="radeon vesa
fbdev"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LDFLAGS,
MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

solution is to rebuild the package in question

environment variables cannot have dashes in them ... it is simply invalid ...
fix whatever is broken and pulling that crap, dont try and "fix" it in portage

post the environment file in question

"solution is to rebuild the package in question"

*Which* package? This occurs for ALL packages. Should I have to rebuild world?
If this is something FVWM-Crystal messes up, perhaps it should be masked?

That particular file is gone, as I used the hack on the link to get it to be
merged. Here's one environment file sitting at:

/var/tmp/binpkgs/sys-apps/portage-2.1.4.4/temp

(big file - sent as attachment)

Created an attachment (id=144988) [details]
Portage environment file

This is the same thing as bug 210580 but I'm thinking maybe we should add a
filter to portage for this.

Yes. It's exactly that. I'll mark this as a duplicate and keep watching that.
Thank you.

*** This bug has been marked as a duplicate of bug 210580 ***

*** Bug 211882 has been marked as a duplicate of this bug. ***

Created an attachment (id=144990) [details]
remove any variable with name containing a hyphen

If this patch is saved as /tmp/filter_hyphens.patch then it can be applied as
follows:

patch /usr/lib/portage/bin/ebuild.sh /tmp/filter_hyphens.patch

Reopening since portage is going to handle it.

filtering hyphens is a special case.  either filter the variables completely
and properly, or dont filter at all.

http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html
Environment variable names [...] consist solely of uppercase letters, digits,
and the '_' (underscore) from the characters defined in Portable Character Set
and do not begin with a digit.

Well, in addition to the specification, we need to be pragmatic and consider
the variable names that current ebuilds and eclasses rely on. For example, is
it really safe for us to filter variable names containing lower case letters?

sorry, i did not mean to indicate we should do that.  what i was after was the
fact we'll hit this issue again as soon as some other application uses an @
symbol or a ^ or a % or something else.  you can easily write C code that calls
putenv("M@FOO=asdf") and it'll go into the environment.

try this definition instead:
In the shell command language, a word consisting solely of underscores, digits,
and alphabetics from the portable character set. The first character of a name
is not a digit.

Created an attachment (id=145147) [details]
exclude variable names that begin with a digit or contain non-alphanumeric
characters

Created an attachment (id=145185) [details]
exclude variable names that begin with a digit or contain non-alphanumeric
characters

This fixes an issue with the previous patch that prevented it from filtering
variable names containing multiple non-alphanumeric characters.

If this patch is saved as /tmp/filter_unsupported.patch then it can be applied
as follows:

cd /usr/lib/portage
patch -p0 -i /tmp/filter_unsupported.patch

Created an attachment (id=145244) [details]
exclude variable names that begin with a digit or contain non-alphanumeric
characters

This fixes an issue with the previous patch that caused the = character to
trigger false positives.

(From update of attachment 145244 [details])
There are still some remaining issues with the last "filter_unsupported.patch"
so for now I recommend the "filter_hyphens.patch" that was first posted.

Unfortunately, tightening the filter to include all non-alphanumeric characters
exposed some false positives that need to be handled. As of current svn, the
only remaining issue is to handle multi-line variable definitions that are
generated by the 'export' builtin.

This is supposed to be fixed in portage-2.2_pre5 or earlier.

This is supposed to be fixed in portage-2.2_pre5 or earlier.