Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 135470 - /usr/lib/portage/bin/sed wrapper causes problems for userland_GNU systems
Summary: /usr/lib/portage/bin/sed wrapper causes problems for userland_GNU systems
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Portage team
URL: http://forums.gentoo.org/viewtopic-t-...
Whiteboard:
Keywords: InVCS
: 136094 136155 (view as bug list)
Depends on:
Blocks: 115839
  Show dependency tree
 
Reported: 2006-06-03 23:16 UTC by Andrey Falko
Modified: 2006-06-09 10:35 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
for post 17 the portage-2.1_rc4-r4/environment.bz2 (environment.bz2,27.62 KB, text/plain)
2006-06-08 11:42 UTC, Paul de Vries
Details
Andrey's environment.bz2 (environment.bz2,28.06 KB, application/octet-stream)
2006-06-08 13:14 UTC, Andrey Falko
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Falko 2006-06-03 23:16:08 UTC
Visit url to see my emerge output and my emerge --info.
Comment 1 Andrew Koller 2006-06-03 23:35:36 UTC
My emerge info is as follows:

Portage 2.1_rc4 (default-linux/x86/2006.0, gcc-4.1.1, glibc-2.4-r3, 2.6.16-gentoo-r8 i686)
=================================================================
System uname: 2.6.16-gentoo-r8 i686 Intel(R) Pentium(R) M processor 2.13GHz
Gentoo Base System version 1.12.0
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r2
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O3 -march=pentium-m -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d"
CXXFLAGS="-O3 -march=pentium-m -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer prelink sandbox sfperms strict"
GENTOO_MIRRORS="ftp://cs.ubishops.ca/pub/gentoo http://adelie.polymtl.ca/ "
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 7zip X a52 aac aalib acpi ada aim alsa amarok apache apache1 apache2 apm arts audiofile avi berkdb bitmap-fonts bluetooth bonobo bootsplash c++ canna cdparanoia cdr cdrom cgi chroot clamav clamd cli crypt cups dbus dri dvd dvdr dvdread eds emboss encode epiphany esd ethereal exif ffmpeg firefox flash foomaticdb fortran ftp gaim gdbm gif gimp gimpprint gkrellm gnome gpm gstreamer gtk gtk2 gtkhtml gzip hardenedphp icq ieee1394 imagemagick imlib ipod ipv6 isdnlog jabber jack java javascript jpeg jpeg2k kde kdeenablefinal libg++ libwww mad mikmod ming mmx mng motif mp3 mpeg mplayer msn mysql mysqli ncurses nfs nls nptl nptlonly ogg oggvorbis pam pcmcia pcre pdf pdflib perl php png pnp pppd python qemu-fast qt quicktime rar readline reflection reiserfs ruby sdl session slang spell spl sse sse2 ssl syslog tcltk tcpd timidity truetype truetype-fonts type1-fonts udev unicode userlocales verbose vorbis wifi win32codecs xchat xine xinerama xinetd xml xmms xorg xscreensaver xv xvid yahoo zip zlib elibc_glibc kernel_linux userland_GNU video_cards_radeon video_cards_fglrx video_cards_fbdev video_cards_vesa"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY


Seems to work.  I merged it successfully and functionality is good.  I don't believe this is a bug, perhaps just a lack of "sane" build settings?
Comment 2 Andrey Falko 2006-06-03 23:40:40 UTC
Running emerge --debug mesa, shows where it is stuck:

 * Removing DO_DEBUG defs in dri drivers...
+ LAST_E_CMD=einfon
+ return 0
+ LAST_E_CMD=einfo
+ return 0
+ find src/mesa/drivers/dri -name '*.[hc]' -exec egrep -l '\#define\W+DO_DEBUG\W+1' '{}' ';'
+ xargs sed -i -re 's/\#define\W+DO_DEBUG\W+1/\#define DO_DEBUG 0/'
   
Comment 3 Joshua Baergen (RETIRED) gentoo-dev 2006-06-04 11:00:18 UTC
It's possible that your 'sed' command is broken.  Try re-merging sed, or even re-merging sed with lighter CFLAGS and then trying this ebuild again.  My pentium-m runs:

-O2 -march=pentium-m -pipe -fomit-frame-pointer
Comment 4 Andrey Falko 2006-06-04 11:36:03 UTC
I did as you suggested. Still the same stoppage.
Comment 5 Joshua Baergen (RETIRED) gentoo-dev 2006-06-04 11:53:09 UTC
Can you try going into the source directly and running the 'sed' command manually, please?  Also, what version of sed do you have installed?
Comment 6 Andrey Falko 2006-06-04 12:38:00 UTC
sed version 4.1.5

That is what I get when I run sed in the source:
Ma3oxuct Mesa-6.5 #  sed -i -re 's/\#define\W+DO_DEBUG\W+1/\#define DO_DEBUG 0/'
sed: no input files
Comment 7 Andrey Falko 2006-06-04 13:41:29 UTC
Running these two commands, gets stuck.

find src/mesa/drivers/dri -name '*.[hc]' -exec egrep -l '\#define\W+DO_DEBUG\W+1' '{}' ';'
src/mesa/drivers/dri/i810/i810context.h
src/mesa/drivers/dri/i915/intel_context.h
src/mesa/drivers/dri/i915-tex/intel_context.h
src/mesa/drivers/dri/mach64/mach64_context.h
src/mesa/drivers/dri/mga/mgacontext.h
src/mesa/drivers/dri/r128/r128_context.h
src/mesa/drivers/dri/r200/r200_context.h
src/mesa/drivers/dri/r300/radeon_context.h
src/mesa/drivers/dri/radeon/radeon_context.h
src/mesa/drivers/dri/sis/sis_context.h
src/mesa/drivers/dri/tdfx/tdfx_context.h
Ma3oxuct Mesa-6.5 # xargs sed -i -re 's/\#define\W+DO_DEBUG\W+1/\#define DO_DEBUG 0/'
Comment 8 Andrey Falko 2006-06-04 14:26:56 UTC
I have been attempting to do a brute force modification on th ebuild...here is where I got to:

for t in $(find src/mesa/drivers/dri -name *.[hc])
       do
            str=$(egrep -l "\#define\W+DO_DEBUG\W+1" $t)
            if [[ -n $str ]]
                then
                sed -i -re s/\#define\W+DO_DEBUG\W+1/\#define DO_DEBUG 0/ $t
            fi
       done

This in place of: find src/mesa/drivers/dri -name *.[hc] -exec egrep -l "\#define\W+DO_DEBUG\W+1" {} \; | xargs sed -i -re "s/\#define\W+DO_DEBUG\W+1/\#define DO_DEBUG 0/" ;

The problem is that str does not get parsed (by portage I presume) with the quotes properly. 
Comment 9 Andrey Falko 2006-06-04 14:56:58 UTC
doing:
if [[ '' -ne "$str" ]]

works better, but now I am getting:
++ egrep -l '\#define\W+DO_DEBUG\W+1' src/mesa/drivers/dri/i915/intel_context.h
+ str=src/mesa/drivers/dri/i915/intel_context.h
+ [[ '' -ne src/mesa/drivers/dri/i915/intel_context.h ]]
/usr/portage/media-libs/mesa/mesa-6.5-r3.ebuild: line 107: [[: src/mesa/drivers/dri/i915/intel_context.h: division by 0 (error token is "/drivers/dri/i915/intel_context.h")
Comment 10 Tuan Van (RETIRED) gentoo-dev 2006-06-04 20:17:06 UTC
I think it's a portage bug from previous version. I have a similiar problem with mysql and mesa during sed.
`ps aux | grep sed` shows
/bin/bash /usr/lib/portage/bin/sed -i -re s/\#define\W+DO_DEBUG\W+1/\#define DO_DEBUG 0/ src/mesa/drivers/dri/mga/mgacontext.h src/mesa/drivers/dri/sis/sis_context.h src/mesa/drivers/dri/i810/i810context.h src/mesa/drivers/dri/i915/intel_context.h src/mesa/drivers/dri/r128/r128_context.h src/mesa/drivers/dri/r200/r200_context.h src/mesa/drivers/dri/r300/radeon_context.h src/mesa/drivers/dri/tdfx/tdfx_context.h src/mesa/drivers/dri/i915-tex/intel_context.h src/mesa/drivers/dri/mach64/mach64_context.h src/mesa/drivers/dri/radeon/radeon_context.h

notice `/usr/lib/portage/bin/sed` . It got installed and left over from previous portage (?) remove it and everything back to normal. May be a forced removal of /usr/lib/portage/bin/sed is needed?  
Comment 11 Zac Medico gentoo-dev 2006-06-04 20:53:28 UTC
The ebuild removes the file depending on userland_GNU.  Apparently, userland_GNU is not properly expanded sometimes, leading to this problem.  I'd be curious to see what the last version of portage was that you upgraded from.  You should be able to find that information from the output of `grep sys-apps/portage /var/log/emerge.log`.
Comment 12 Tuan Van (RETIRED) gentoo-dev 2006-06-04 21:01:45 UTC
on the box with this problem:
sys-apps/portage-2.0.50-r9
sys-apps/portage-2.0.51_pre20
sys-apps/portage-2.0.51_pre23
sys-apps/portage-2.0.51_rc1
sys-apps/portage-2.0.51_rc6
sys-apps/portage-2.0.51_rc7
sys-apps/portage-2.0.51_rc9
sys-apps/portage-2.0.51-r2
sys-apps/portage-2.0.51-r3
sys-apps/portage-2.0.51-r7
sys-apps/portage-2.0.51-r8
sys-apps/portage-2.0.51-r10
sys-apps/portage-2.0.51-r12
sys-apps/portage-2.0.51-r13
sys-apps/portage-2.0.51-r14
sys-apps/portage-2.0.51-r15
sys-apps/portage-2.0.51-r3
sys-apps/portage-2.0.51-r15
sys-apps/portage-2.0.51.16
sys-apps/portage-2.0.51-r15
sys-apps/portage-2.0.51.18
sys-apps/portage-2.0.51.19
sys-apps/portage-2.0.51.20-r5
sys-apps/portage-2.0.51.21
sys-apps/portage-2.0.51.21-r1
sys-apps/portage-2.0.51.22
sys-apps/portage-2.0.51.22-r1
sys-apps/portage-2.0.51.22-r2
sys-apps/portage-2.0.52-r1
sys-apps/portage-2.0.53_rc3
sys-apps/portage-2.0.53_rc4
sys-apps/portage-2.0.53_rc5
sys-apps/portage-2.0.53_rc6
sys-apps/portage-2.0.53_rc7
sys-apps/portage-2.0.53
sys-apps/portage-2.1_pre1
sys-apps/portage-2.1_pre2
sys-apps/portage-2.1_pre3-r1
sys-apps/portage-2.1_pre4
sys-apps/portage-2.1_pre4-r1
sys-apps/portage-2.1_pre5-r1
sys-apps/portage-2.1_pre5-r2
sys-apps/portage-2.1_pre5-r4
sys-apps/portage-2.1_pre7-r2
sys-apps/portage-2.1_pre7-r3
sys-apps/portage-2.1_pre7-r4
sys-apps/portage-2.1_pre7-r5
sys-apps/portage-2.1_pre9-r4
sys-apps/portage-2.1_pre10
sys-apps/portage-2.1_pre10-r2
sys-apps/portage-2.1_pre10-r4
sys-apps/portage-2.1_pre10-r5
sys-apps/portage-2.1_rc1-r2
sys-apps/portage-2.1_rc2-r3
sys-apps/portage-2.1_rc4-r1

on the box without this problem:
sys-apps/portage-2.0.53
sys-apps/portage-2.1_pre1
sys-apps/portage-2.1_pre2
sys-apps/portage-2.1_pre3
sys-apps/portage-2.1_pre3-r1
sys-apps/portage-2.1_pre4
sys-apps/portage-2.1_pre4-r1
sys-apps/portage-2.1_pre5-r1
sys-apps/portage-2.1_pre5-r2
sys-apps/portage-2.1_pre5-r3
sys-apps/portage-2.1_pre5-r4
sys-apps/portage-2.0.54
sys-apps/portage-2.1_pre5-r4
sys-apps/portage-2.1_pre6-r1
sys-apps/portage-2.1_pre6-r2
sys-apps/portage-2.1_pre6-r3
sys-apps/portage-2.1_pre6-r5
sys-apps/portage-2.1_pre6-r6
sys-apps/portage-2.1_pre7
sys-apps/portage-2.1_pre7-r1
sys-apps/portage-2.1_pre7-r2
sys-apps/portage-2.1_pre7-r3
sys-apps/portage-2.1_pre7-r4
sys-apps/portage-2.1_pre7-r5
sys-apps/portage-2.1_pre9-r4
sys-apps/portage-2.1_pre9-r5
sys-apps/portage-2.1_pre10
sys-apps/portage-2.1_pre10-r2
sys-apps/portage-2.1_pre10-r5
sys-apps/portage-2.1_rc1-r1
sys-apps/portage-2.1_rc1-r2
sys-apps/portage-2.1_rc1-r3
sys-apps/portage-2.1_rc2
sys-apps/portage-2.1_rc2-r1
sys-apps/portage-2.1_rc2-r3
sys-apps/portage-2.1_rc3
sys-apps/portage-2.1_rc3-r2
sys-apps/portage-2.1_rc4
sys-apps/portage-2.1_rc4-r1
Comment 13 Zac Medico gentoo-dev 2006-06-04 21:14:50 UTC
That's interesting that it seems to be happening with recent versions of portage.  Do the use flags for the first version of portage that the problem was noticed with contain userland_GNU or not?  You can look in  /var/db/pkg/sys-apps/portage-${PVR}/USE to find out the USE flags that it had at build time.
Comment 14 Tuan Van (RETIRED) gentoo-dev 2006-06-04 21:41:28 UTC
I did downgrade portage then re-emerge portage again so my USE may not be preserved as when I experience the problem (yes it has userland_GNU). May be the OP Andrey Falko can help here.
Comment 15 Andrey Falko 2006-06-04 22:27:56 UTC
Ma3oxuct Mesa-6.5 # cat /var/db/pkg/sys-apps/portage-2.1_rc4-r1/USE
x86 3dnow X a52 aac aalib alsa amuled apache2 apm arts avi bash-completion bcmath berkdb bitmap-fonts bzip2 calender cbbd cdda cli crypt cups dba dlloader dri dvd dvdr dvdread eds emboss encode esd fame ffmpeg firefox flac foomaticdb fortran freetype ftp gd gdbm gif gphoto2 gpm gstreamer gtk gtk2 hal imap imlib ipv6 isdnlog java jpeg kde libg++ libwww lm_sensors mad matroska mikmod mjpeg motif mp3 mpeg ncurses network nls nogecko-sdk nptl nptlonly nsplugin nvidia ogg openal opengl oss pam pcre pdflib perl png postgres pppd python qt quicktime readline real reflection remote ruby sdl session spell spl sqlite sse2 ssl tcltk tcpd tetex theora truetype truetype-fonts type1-fonts udev unicode vcd vorbis win32codecs wxwindows xml xml2 xmms xorg xv xvid zlib elibc_glibc input_devices_keyboard input_devices_mouse kernel_linux userland_GNU video_cards_none

The problem first appeared for me with portage-2.1_rc4 (because it appeared to me does not mean that it was not there before however). I have since upgraded to portage-2.1-rc4-r1, and thus can only provide the flags for it (i doubt that they changed).
Comment 16 Zac Medico gentoo-dev 2006-06-08 11:31:32 UTC
*** Bug 136094 has been marked as a duplicate of this bug. ***
Comment 17 Paul de Vries 2006-06-08 11:39:33 UTC
I have same issue when emerging libtool. It also hangs with sed on the folowing line:
    date=`./mkstamp < ./ChangeLog` && \
    eval `egrep '^[[:space:]]*PACKAGE.*=' configure` && \
    eval `egrep '^[[:space:]]*VERSION.*=' configure` && \
    sed -e "s/@PACKAGE@/${PACKAGE}/" -e "s/@VERSION@/${VERSION}/" \
        -e "s%@TIMESTAMP@%$date%" ./ltmain.in > ltmain.shT || return 1

running above statement in bash shell does work.

portage-2.1_rc4-r4 # cat USE
x86 X a52 aac alsa apache2 apm avi bash-completion berkdb bitmap-fonts bzip2 cli crypt cups directfb dri dvd emboss encode exif faad2 fam fbcon ffmpeg flac foomaticdb fortran gdbm gif gimp gnome gpm gstreamer gtk gtk2 hal imlib isdnlog jpeg jpg kdeenablefinal lcms libg++ libwww live mad mikmod mmxext motif mozilla mp3 mpeg mysql mythtv ncurses network nls nptl ogg opengl oss pam pcre pdflib perl png postgres ppds pppd python quicktime readline real reflection sdl session spell spl sqlite ssl svga tcltk tcpd tiff truetype truetype-fonts type1-fonts udev unicode usb v4l vorbis win32codecs xml xml2 xmms xorg xv zlib elibc_glibc initng_plugins_also initng_plugins_bash_launcher initng_plugins_chdir initng_plugins_chroot initng_plugins_conflict initng_plugins_cpout initng_plugins_critical initng_plugins_cron initng_plugins_daemon initng_plugins_dev initng_plugins_dllaunch initng_plugins_envparser initng_plugins_find initng_plugins_fstat initng_plugins_history initng_plugins_idleprobe initng_plugins_initctl initng_plugins_interactive initng_plugins_iparser initng_plugins_last initng_plugins_limit initng_plugins_logfile initng_plugins_netprobe initng_plugins_ngc4 initng_plugins_pause initng_plugins_pidfile initng_plugins_reload initng_plugins_renice initng_plugins_rlparser initng_plugins_simple_launcher initng_plugins_stcmd initng_plugins_stdout initng_plugins_suid initng_plugins_syncron initng_plugins_syslog initng_plugins_unneeded initng_plugins_provide input_devices_keyboard input_devices_mouse kernel_linux linguas_en userland_GNU video_cards_matrox
Comment 18 Paul de Vries 2006-06-08 11:42:55 UTC
Created attachment 88703 [details]
for post 17 the portage-2.1_rc4-r4/environment.bz2
Comment 19 Zac Medico gentoo-dev 2006-06-08 12:18:09 UTC
Hmm, userland_GNU is in the environment.  I still have no idea why the userland_GNU test would fail.  Are any of you doing anything with USE in /etc/portage/bashrc?
Comment 20 Paul de Vries 2006-06-08 12:32:03 UTC
(In reply to comment #19)
> Are any of you doing anything with USE in /etc/portage/bashrc?
No, I do not have a file called: /etc/portage/bashrc
Also in my own .bashrc I do not do anything with USE 
Comment 21 Andrey Falko 2006-06-08 13:08:23 UTC
(In reply to comment #20)
> (In reply to comment #19)
> > Are any of you doing anything with USE in /etc/portage/bashrc?
> No, I do not have a file called: /etc/portage/bashrc
> Also in my own .bashrc I do not do anything with USE 
> 

Same with me.
Comment 22 Andrey Falko 2006-06-08 13:14:49 UTC
Created attachment 88716 [details]
Andrey's environment.bz2
Comment 23 Andrey Falko 2006-06-08 13:51:30 UTC
when emerging libutempter, gets stuck here:

++ '[' -n default ']'
++ abi=default
++ local var=LIBDIR_default
++ echo lib
+ make DESTDIR=/var/tmp/portage/libutempter-1.1.4.1/image/ libdir=/usr/lib libexecdir=/usr/lib/misc includedir=/usr/include install
Comment 24 Zac Medico gentoo-dev 2006-06-08 15:13:42 UTC
In svn r3475 I've committed a fix for the sed wrapper that prevents the endless exec loop.  I'd still like to know what's causing the userland_GNU test to fail though...
Comment 25 Jakub Moc (RETIRED) gentoo-dev 2006-06-09 00:41:55 UTC
*** Bug 136155 has been marked as a duplicate of this bug. ***
Comment 26 James Youngquist 2006-06-09 00:52:24 UTC
When I manually run the sed command from the ebuild on the src it works without error.

Killing the sed process during the emerge allows the build to continue, though I don't know if it successfully removed the DO_DEBUG defines.
Comment 27 Zac Medico gentoo-dev 2006-06-09 01:00:47 UTC
(In reply to comment #26)
> Killing the sed process during the emerge allows the build to continue, though
> I don't know if it successfully removed the DO_DEBUG defines.

The recommended solution is to simply `rm /usr/lib/portage/bin/sed`.  It never should have been installed in the first place (the userland_GNU test failed for some unknown reason).
Comment 28 Zac Medico gentoo-dev 2006-06-09 07:44:57 UTC
The exec loop fix has been released in 2.1 final.  If anyone findes out what causes the userland_GNU test to fail, or is able to reproduce it consistently, please file a new bug.
Comment 29 Paul de Vries 2006-06-09 10:20:02 UTC
(In reply to comment #28)
> The exec loop fix has been released in 2.1 final.
I am sorry but for me 2.1 does not fix it. Still same issue with sed.
Comment 30 Paul de Vries 2006-06-09 10:22:32 UTC
(In reply to comment #29)
> (In reply to comment #28)
> > The exec loop fix has been released in 2.1 final.
> I am sorry but for me 2.1 does not fix it. Still same issue with sed.
> 
After 'rm /usr/lib/portage/bin/sed' the emerge does work.
Comment 31 Zac Medico gentoo-dev 2006-06-09 10:35:58 UTC
(In reply to comment #30)
> After 'rm /usr/lib/portage/bin/sed' the emerge does work.

Yeah, that was the old sed wrapper.  If the new one had been installed (which it shouldn't, normally), then the exec loop would not happen anymore.