Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 211949

Summary: Hyphens (or dashes) in environment variables causes Portage to fail
Product: Portage Development Reporter: Beetle B. <usefuljunk>
Component: UnclassifiedAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: major CC: progserega
Priority: High Keywords: InVCS
Version: unspecified   
Hardware: x86   
OS: Linux   
URL: http://forums.gentoo.org/viewtopic-t-625637.html
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 216231    
Attachments: Portage environment file
remove any variable with name containing a hyphen
exclude variable names that begin with a digit or contain non-alphanumeric characters
exclude variable names that begin with a digit or contain non-alphanumeric characters
exclude variable names that begin with a digit or contain non-alphanumeric characters

Description Beetle B. 2008-03-01 05:37:08 UTC
All of a sudden, Portage stopped working. Either in the beginning of a merge, or in the end (when it is about to unmerge old versions of a package), it would suddenly die with these kinds of messages (there are many, I'm posting only a few):

/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 31: Colorset-Author=fRiQ: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 32: Colorset-Name=DarkDesktop: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 47: Desk-Pages-X=8: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 48: Desk-Pages-Y=1: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 66: EWMH-BS-Desktop-Bottom=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 67: EWMH-BS-Desktop-Left=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 68: EWMH-BS-Desktop-Right=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 69: EWMH-BS-Desktop-Top=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 70: EWMH-BS-Panel-Bottom=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 71: EWMH-BS-Panel-Left=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 72: EWMH-BS-Panel-Right=3: command not found
/var/tmp/binpkgs/sys-devel/libtool-1.5.24/temp/environment: line 73: EWMH-BS-Panel-Top=30: command not found

As the link points out, this is a problem because Bash does not allow environment variables having hyphens in them.

In this particular instance, I think it's having trouble not because I currently have environment variables with hyphens (I don't at present), but because the package was built with those variables. 

In any case, I have no idea why this behavior began happening all of a sudden. I, like the user in the link, use FVWM-Crystal, which is the main program setting these env variables. However, we've been using it for months (years for me) - yet Portage only failed recently.

A solution is posted in the link, which I'm using to get by. I have no idea if it's a "good" solution.  

Reproducible: Always

Steps to Reproduce:
1.emerge something.
2.Either fails right before configure.
3.Or fails when unmerging old versions.

Actual Results:  
Dies a miserable death.

Expected Results:  
Should merge/unmerge safely.

emerge --info output:

=================================================================
System uname: 2.6.14-gentoo-r2 i686 Intel(R) Pentium(R) 4 CPU 2.53GHz
Timestamp of tree: Mon, 18 Feb 2008 03:16:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p17-r1
dev-java/java-config: 1.3.7, 2.1.4
dev-lang/python:     2.3.5-r3, 2.4.4-r6, 2.5.1-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 1.12.10-r5
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24, 1.5.26
virtual/os-headers:  2.6.23-r3
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -O3 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c"
CXXFLAGS="-march=pentium4 -O3 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks fixpackages metadata-transfer nostrip parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://gentoo.mirrors.tds.net/gentoo http://gentoo.cites.uiuc.edu/pub/gentoo/ ftp://ftp.ussg.iu.edu/pub/linux/gentoo http://gentoo.osuosl.org/ ftp://gentoo.mirrors.tds.net/gentoo"
LANG="en_US.utf8"
LINGUAS="en ar"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/portage/local/layman/xeffects /usr/portage/local/layman/sunrise /usr/portage/local/layman/science /usr/portage/local/layman/roslin /usr/portage/local/layman/other"
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"
USE="7zip X a52 aac aalib acpi aim alsa apache2 apm asf avi bash-completion berkdb bitmap-fonts bittorrent browserplugin cairo caps cdparanoia cdr cli cracklib crypt cups cvs dbus dga directfb divx4linux doc dri dvb dvd dvdnav dvdr dvdread eds emacs emboss encode exif fam fbcon fbdev ffmpeg fftw firefox flac fortran ftp gd gdbm gif gimp glitz gnutella gphoto2 gpm gstreamer gtk guile hal hbci hdri i8x0 iconv icq imagemagick imap imlib ipv6 isdnlog jabber java javascript jpeg jpeg2k justify kde lame lapack lcms ldap leim libcaca libsamplerate live lyx mad maildir matroska mbox md5sum midi mikmod mjpeg mmx mozilla mp2 mp3 mpeg mplayer msn mudflap musepack mysql ncurses network nls nptl nptlonly nsplugin oav ofx ogg oggvorbis openexr opengl openmp oscar oss pam pcre pda pdf pdflib perl pgp php plotutils png pnm pnp ppds pppd python qt qt3 qt4 quicktime radio rar readline real reflection rss rtsp samba sasl sdl session shout sox speex spell spl srt sse ssl stream subtitles subversion svg svga tcpd tetex tga theora tiff tivo transparency truetype truetype-fonts type1-fonts unicode usb v4l vcd vesa videos vidix vorbis wavpack win32codecs wma wmf wxwindows x264 x86 xcomposite xine xml xorg xosd xscreensaver xv xvid yahoo zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ar" USERLAND="GNU" VIDEO_CARDS="radeon vesa fbdev"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LDFLAGS, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 SpanKY gentoo-dev 2008-03-01 06:36:53 UTC
solution is to rebuild the package in question

environment variables cannot have dashes in them ... it is simply invalid ... fix whatever is broken and pulling that crap, dont try and "fix" it in portage

post the environment file in question
Comment 2 Beetle B. 2008-03-01 06:47:24 UTC
"solution is to rebuild the package in question"

*Which* package? This occurs for ALL packages. Should I have to rebuild world? If this is something FVWM-Crystal messes up, perhaps it should be masked?

That particular file is gone, as I used the hack on the link to get it to be merged. Here's one environment file sitting at:

/var/tmp/binpkgs/sys-apps/portage-2.1.4.4/temp

(big file - sent as attachment)
Comment 3 Beetle B. 2008-03-01 06:48:02 UTC
Created attachment 144988 [details]
Portage environment file
Comment 4 Zac Medico gentoo-dev 2008-03-01 07:29:02 UTC
This is the same thing as bug 210580 but I'm thinking maybe we should add a filter to portage for this.
Comment 5 Beetle B. 2008-03-01 07:34:46 UTC
Yes. It's exactly that. I'll mark this as a duplicate and keep watching that. Thank you.

*** This bug has been marked as a duplicate of bug 210580 ***
Comment 6 Zac Medico gentoo-dev 2008-03-01 07:38:17 UTC
*** Bug 211882 has been marked as a duplicate of this bug. ***
Comment 7 Zac Medico gentoo-dev 2008-03-01 08:35:03 UTC
Created attachment 144990 [details, diff]
remove any variable with name containing a hyphen

If this patch is saved as /tmp/filter_hyphens.patch then it can be applied as follows:

patch /usr/lib/portage/bin/ebuild.sh /tmp/filter_hyphens.patch
Comment 8 Zac Medico gentoo-dev 2008-03-01 08:36:18 UTC
Reopening since portage is going to handle it.
Comment 9 SpanKY gentoo-dev 2008-03-01 20:58:18 UTC
filtering hyphens is a special case.  either filter the variables completely and properly, or dont filter at all.

http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html
Environment variable names [...] consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit.
Comment 10 Zac Medico gentoo-dev 2008-03-01 21:23:04 UTC
Well, in addition to the specification, we need to be pragmatic and consider the variable names that current ebuilds and eclasses rely on. For example, is it really safe for us to filter variable names containing lower case letters?
Comment 11 SpanKY gentoo-dev 2008-03-01 22:05:17 UTC
sorry, i did not mean to indicate we should do that.  what i was after was the fact we'll hit this issue again as soon as some other application uses an @ symbol or a ^ or a % or something else.  you can easily write C code that calls putenv("M@FOO=asdf") and it'll go into the environment.

try this definition instead:
In the shell command language, a word consisting solely of underscores, digits, and alphabetics from the portable character set. The first character of a name is not a digit.
Comment 12 Zac Medico gentoo-dev 2008-03-02 23:34:28 UTC
Created attachment 145147 [details, diff]
exclude variable names that begin with a digit or contain non-alphanumeric characters
Comment 13 Zac Medico gentoo-dev 2008-03-03 10:30:17 UTC
Created attachment 145185 [details, diff]
exclude variable names that begin with a digit or contain non-alphanumeric characters

This fixes an issue with the previous patch that prevented it from filtering variable names containing multiple non-alphanumeric characters.

If this patch is saved as /tmp/filter_unsupported.patch then it can be applied as follows:

cd /usr/lib/portage
patch -p0 -i /tmp/filter_unsupported.patch
Comment 14 Zac Medico gentoo-dev 2008-03-04 04:57:32 UTC
Created attachment 145244 [details, diff]
exclude variable names that begin with a digit or contain non-alphanumeric characters

This fixes an issue with the previous patch that caused the = character to trigger false positives.
Comment 15 Zac Medico gentoo-dev 2008-03-04 08:03:44 UTC
Comment on attachment 145244 [details, diff]
exclude variable names that begin with a digit or contain non-alphanumeric characters

There are still some remaining issues with the last "filter_unsupported.patch" so for now I recommend the "filter_hyphens.patch" that was first posted.

Unfortunately, tightening the filter to include all non-alphanumeric characters exposed some false positives that need to be handled. As of current svn, the only remaining issue is to handle multi-line variable definitions that are generated by the 'export' builtin.
Comment 16 Marius Mauch (RETIRED) gentoo-dev 2008-03-20 18:15:06 UTC
This is supposed to be fixed in portage-2.2_pre5 or earlier.
Comment 17 Marius Mauch (RETIRED) gentoo-dev 2008-03-20 18:15:52 UTC
This is supposed to be fixed in portage-2.2_pre5 or earlier.