Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 81031 - cut -f1 does not work when built with -Os
Summary: cut -f1 does not work when built with -Os
Status: RESOLVED NEEDINFO
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: x86 Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
: 83338 88420 (view as bug list)
Depends on: 74803
Blocks:
  Show dependency tree
 
Reported: 2005-02-06 15:03 UTC by Stefan Rystedt
Modified: 2005-08-04 08:57 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Rystedt 2005-02-06 15:03:40 UTC
I have truble with emerging pam. It wont compile any of its modules. I have found out that the problem is that for some reason 'cut -f1' does not work in the shell created under portage. The line
MODDIRS=$(shell /bin/ls -d pam_*/Makefile | cut -f1 -d/)
gives an empty MODDIRS rather then a list of all directories containing modules.

I have modified the make script to start an xterm instead of compiling the files in MODDIRS.
I then run

ebuild /usr/portage/sys-libs/pam/pam-0.77-r6.ebuild compile

making sure that my modified Makefile did not get owerwritten.

Here is the result I get for some test with cut with that exterm.

sh-3.00# echo "1/2/3/4" | cut -f 1 -d/

sh-3.00# echo "1/2/3/4" | cut -f 2 -d/
2
sh-3.00# echo "1/2/3/4" | cut -f 1- -d/

sh-3.00# echo "1/2/3/4" | cut -f 2- -d/
2/3/4
sh-3.00# 
sh-3.00# which cut
/bin/cut
sh-3.00# 
As you can see cut -f1 just return a blank line. Running in a normal xterm everything works just fine so it seems it has something to do with the enviroment variables that gets set when runing emerge or ebuld. But which variable can make cut missbehave like this?

Can this be releated to bug #74803
I have sys-apps/coreutils  5.2.1-r4 installed.


Reproducible: Always
Steps to Reproduce:




Portage 2.0.51-r15 (default-linux/x86/2004.2, gcc-3.3.5,
glibc-2.3.4.20040808-r1, 2.6.10-gentoo-r6 i686)
=================================================================
System uname: 2.6.10-gentoo-r6 i686 AMD Athlon(tm) XP 3000+
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Dec 18 2004, 19:01:51)]
ccache version 2.3 [enabled]
dev-lang/python:     2.3.4
sys-devel/autoconf:  2.59-r6, 2.13
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.6.8.1-r1, 2.6.8.1-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-Os -march=athlon-xp -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env
/usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-Os -march=athlon-xp -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks fixpackages sandbox sfperms strict"
GENTOO_MIRRORS=" http://ds.thn.htu.se/linux/gentoo
http://mirror.pudas.net/gentoo ftp://mirror.pudas.net/gentoo
ftp://ftp.du.se/pub/os/gentoo http://ftp.du.se/pub/os/gentoo"
LANG="en_US.utf8"
LC_ALL="en_US.utf8"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowex X Xaw3d acl acpi aim alsa apm ared aredmem audiofile avi
ba-completion bash-completion berkdb bitmap-fonts bonobo bzlib cdb cdparanoia
cdr codecs crypt cups curl dga divx4linux dvd dvdr dvdread emacs encode evo f77
faac fam fbcon ffmpeg fftw flac font-server foomaticdb fortran ftp gb gdbm gif
glut gnome gphoto2 gpm gstreamer gtk gtk2 gtkhtml iconv icq imagemagick imlib
ithreads jabber java jikes joystick jpeg ladcca ldap leim libg++ libwww live lzo
mad matroska mbox mikmod mime mmx mmx2 mng motif mozilla moznocompose moznoirc
moznomail mozsvg mp3 mpeg ncurses network nls nptl nvidia oggvorbis openal
opengl oscar oss pam pcre pdflib perl pic pie png ppds python qt quicktime
readline rtc ruby sdl slang speex spell sse ssl svg tcpd theora tiff truetype
truetype-fonts type1-fonts unicode usb v4l2 wmf xml xml2 xmms xpm xv xvid xvmc zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LDFLAGS, PORTDIR_OVERLAY
Comment 1 Martin Schlemmer (RETIRED) gentoo-dev 2005-02-07 01:48:07 UTC
Different issue then bug #74803 - yours can find cut, but cut -f1 seems to be
hosed.
Comment 2 Martin Schlemmer (RETIRED) gentoo-dev 2005-02-07 11:20:51 UTC
I checked the cut sources, and also traced it, etc - it does not seem to use
getenv at least, so I do not know about env variables influencing it.
Comment 3 Stefan Rystedt 2005-02-07 13:18:54 UTC
I have done some more testing and found out that 'cut -f 1 file' works.
Here is the result from my new tests.

sh-3.00# echo '1/2/3/4' | cut -f1 -d/

sh-3.00# echo '1/2/3/4' > file
sh-3.00# cut -f1 -d/ file
1
sh-3.00# cut -f1 -d/ < file

sh-3.00# 

As you can see if given cut a file as an argument everething works just fine but not if i redirect stdin. I am using bash 3.0-r7 and readline 5.0-r1 since i read some where that they had better suport for utf8 wich I use as default language.
I would have made more sense if cut missbehaved all the time but it is only under emerge or ebuild. Why does these things allways happen to me :( Maby becasue I just can't keep my fingers of all the nice extra features I just have to have :-)
Comment 4 SpanKY gentoo-dev 2005-02-07 13:44:43 UTC
does `echo '1/2/3/4' | cut -f1 -d/ -` work ?
Comment 5 Stefan Rystedt 2005-02-07 13:57:37 UTC
Nope that does not work. Should the last - really be there? I did not work ithere way. I am trying to run cut with ddd but I have a hard time to get ddd to find the sources of cut. 
I recompiling corutils with 
USE=debug FEATURES="noclean nostrip" emerge -v coreutils 
that should do it shouldn't it?
Comment 6 SpanKY gentoo-dev 2005-02-07 14:01:44 UTC
yes, '-' tells cut that it should read stdin
Comment 7 Stefan Rystedt 2005-02-07 14:18:11 UTC
Oh I see.
Whell I foud out why I could not get ddd to find the sources I had to give the -g flag to the compiler. I thought that the USE flag debug would do that.
Any way after recompiling with the same comand as before except adding CFLAGS='-g -O0' cut started to work. So it does not only have to do with the enviroment it allso has to do with how it is compiled. The horriable flags I have set in the make.conf is CFLAGS="-Os -march=athlon-xp -pipe" not to much to brag about. Im going to do some testing to see what flags seems to break things.
Comment 8 Stefan Rystedt 2005-02-07 14:37:44 UTC
Well the cflag was -Os I though that this should be a safe flag.
To sum everything up. 

If compiling coreutils with the optimazation flag -Os with gcc 3.3.5 cut with the flag -f1 will not work when runing under portage if the input comes from a redirected stdin.

I just have to reapet myself why are these things happening to me.

Let me now if you want to look into this bug more so that you want more info. I my self is now going to change the -Os flag to good old -O2 in my make.conf and hopfully live happy ever after:) You are free to close this bug if you want.
Comment 9 SpanKY gentoo-dev 2005-02-07 16:04:44 UTC
-Os by itself is normally 'safe' ... it's when you add in -march / -mcpu that things get funky
Comment 10 Stefan Rystedt 2005-02-07 22:16:54 UTC
Whell in this case it is only the -Os flag since it breakes when I set the CFLAG='-Os -pipe' and nothing else.
Comment 11 Stefan Rystedt 2005-02-08 14:17:42 UTC
I have some more info if you are intrested. I read in the gentoo forums that some one else had a simulare problem and he had solved it with FEATURES="-sandbox" ebuild pam-0.77-r6.ebuild install 
So it looks like it is the -Os flag in combination with sandbox that breaks cut. I have not tested this myself. I could if you really want me to.
Comment 12 Martin Schlemmer (RETIRED) gentoo-dev 2005-02-10 10:03:35 UTC
Would be nice if you can manage it.
Comment 13 Stefan Rystedt 2005-02-10 11:21:19 UTC
I have tested with the FEATURES="-sandbox" and then it works. So it is only when coreutils is compiled with -Os and using sandbox this bugg appear. I am runnig portage as the root user.
Comment 14 Were 2005-02-10 15:07:56 UTC
This bug seems to have many ramifications.
Doing an emerge -e world with gcc 3.4.3 the I had problems with the following:
pam: ebuild uses cut
sun-jdk: ebuild uses cut
tightvnc: uses gcc.class (that uses cut) to apply a patch
libmpeg3: uses gcc.class ...
piave: uses gcc.class ...
kdenlive: uses gcc.class ...

I assume that every build checking the gcc version to apply a patch conditionally is affected.
I am happy using -Os in make.conf, so I think is better forcing the cflags just for the coreutils ebuild in case -Os is configured, adding to the ebuild 

replace-flags -Os -O2 && ewarn "Replacing -Os with -O2"

In a quick test, coreutils-5.2.1-r4 now compiles with -O2 and piave get patched using gcc.class
Comment 15 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2005-02-26 19:22:51 UTC
*** Bug 83338 has been marked as a duplicate of this bug. ***
Comment 16 Manuel McLure 2005-03-24 21:52:41 UTC
In my experience (bug 74803) I've found that it's a combination of -Os, LANG=XXX.UTF-8 and the sandbox. Any of those can be different and the problem will not occur.
Comment 17 Martin Schlemmer (RETIRED) gentoo-dev 2005-07-04 07:28:28 UTC
*** Bug 88420 has been marked as a duplicate of this bug. ***
Comment 18 Martin Schlemmer (RETIRED) gentoo-dev 2005-07-04 07:29:08 UTC
Can anybody try sandbox-1.2.10 with unstable portage and see if recompiling
coreutils with -Os still causes this?
Comment 19 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-04 08:57:16 UTC
Reopen if new info.