After upgrading grep to 2.5.3, "grep -i" does not work. $ echo "AaBbCc" > testfile $ grep --version GNU grep 2.5.3 $ grep -i aabbcc testfile $ Reproducible: Always Steps to Reproduce: After upgrading grep to 2.5.3, "grep -i" does not work. $ echo "AaBbCc" > testfile $ grep --version grep (GNU grep) 2.5.1 $ grep -i aabbcc testfile AaBbCc $ sudo emerge --usepkg =grep-2.5.3 $ grep --version GNU grep 2.5.3 $ grep -i aabbcc testfile $ So I got frustrated, and tested with the original tarball of grep 2.5.3. $ down/grep-2.5.3/src/grep --version GNU grep 2.5.3 $ down/grep-2.5.3/src/grep -i aabbcc testfile AaBbCc $ Now you can see, the original tarball of grep 2.5.3 does not have any problem. What happened with the Gentoo portage of grep 2.5.3?
Hi, I cannot reproduce your problem on my machine: lars@barcelona:~/temp> echo "AaBbCc" > testfile lars@barcelona:~/temp> grep --version | grep grep GNU grep 2.5.3 lars@barcelona:~/temp> grep -i aabbcc testfile AaBbCc lars@barcelona:~/temp> Could you maybe provide your emerge --info so we can compare our systems for any difference? Mine is this: barcelona:~ # emerge --info Portage 2.1.5.6 (default/linux/amd64/2008.0/desktop, gcc-4.2.4, glibc-2.7-r2, 2.6.25.17 x86_64) ================================================================= System uname: 2.6.25.17 x86_64 Quad-Core AMD Opteron(tm) Processor 2354 Timestamp of tree: Sun, 21 Sep 2008 06:00:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-java/java-config: 1.3.7, 2.1.6-r1 dev-lang/python: 2.5.2-r8 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 2.0.0 sys-apps/openrc: 0.2.5 sys-apps/sandbox: 1.2.18.1-r3 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.25-r4 ACCEPT_KEYWORDS="amd64 ~amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=opteron -mtune=opteron -O2 -finline-functions -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/X11/Sessions /etc/X11/app-defaults /etc/X11/xinit /etc/bash_completion.d /etc/bonobo-activation /etc/ca-certificates.conf /etc/cups /etc/dbus-1 /etc/env.d /etc/env.d/java/ /etc/eselect/compiler /etc/fish /etc/fonts /etc/fonts/fonts.conf /etc/foomatic /etc/gconf /etc/gentoo-release /etc/gimp /etc/gnome-vfs-2.0 /etc/gtk /etc/gtk-2.0 /etc/hotplug /etc/hotplug.d /etc/imlib /etc/init.d /etc/iproute2 /etc/nas /etc/ntop /etc/pam.d /etc/pango /etc/profile.d /etc/revdep-rebuild /etc/sasl2 /etc/sgml /etc/ssl /etc/ssmtp /etc/t1lib /etc/terminfo /etc/udev/rules.d /etc/xinetd.d /etc/xml /etc/zsh" CXXFLAGS="-march=opteron -mtune=opteron -O2 -finline-functions -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--alphabetical --with-bdeps=y" FEATURES="ccache collision-protect distlocks parallel-fetch sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror ftp://ftp.tu-clausthal.de/pub/linux/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LDFLAGS="-Wl,-O1,--hash-style=gnu,--sort-common,--as-needed" LINGUAS="de" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://192.168.0.254/gentoo-portage" USE="3dnow 3dnowext X a52 aac acpi alsa amd64 amd64codecs asf avi berkdb branding bzip2 cairo cdparanoia cdr cli cracklib crypt cups dvd dvdr dvdread emboss encode fam ffmpeg flac gdbm gif gnutls gpg gtk gtk2 iconv idn imagemagick imlib isdnlog jpeg kde kdehiddenvisibility lame libwww mad midi mikmod mjpeg mmx mmxext mp3 mpeg mudflap multilib ncurses nls nptl nptlonly nsplugin ogg oggvorbis opengl pam pcre pdf png pppd qt3 qt3support quicktime readline reflection sdl session silc slang smp spell spl sse sse2 ssl startup-notification svg sysfs tga theora threads tiff truetype usb v4l2 vcd vorbis x264 xcomposite xml xml2 xorg xprint xulrunner xv xvid zlib" ALSA_CARDS="emu10k1 hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LINGUAS="de" USERLAND="GNU" VIDEO_CARDS="nv nvidia" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
My system: $ emerge --info Portage 2.2_rc8 (default-linux/x86/2007.0, gcc-4.2.4, glibc-2.8_p20080602-r0, 2.6.25-gentoo-r6 i686) ================================================================= System uname: Linux-2.6.25-gentoo-r6-i686-AMD_Athlon-tm-_XP_2600+-with-glibc2.0 Timestamp of tree: Sat, 20 Sep 2008 18:15:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-java/java-config: 1.3.7, 2.1.6 dev-lang/python: 2.5.2-r5 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 1.12.12 sys-apps/sandbox: 1.2.18.1-r3 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1 sys-devel/binutils: 2.18.50.0.1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.3.5, 2.2.4 virtual/os-headers: 2.6.25-r4 ACCEPT_KEYWORDS="x86 ~x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=athlon-xp -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/4.0/env /usr/kde/4.0/share/config /usr/kde/4.0/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -march=athlon-xp -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks parallel-fetch preserve-libs sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS=" http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo/ http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ http://gentoo.intergenia.de/ http://gentoo.mneisen.org/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo/ ftp://ftp.gentoo.mesh-solutions.com/gentoo/ http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LANG="en_US.UTF-8" LC_ALL="" LDFLAGS="" LINGUAS="en de ko" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /usr/local/portage/layman/sunrise /usr/local/portage/layman/gentoo-kr /usr/local/portage/layman/gentoo-de /usr/local/portage/layman/gnash-cvs" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="GLITZ SVG X a52 aac acl acpi alsa arts avi berkdb bitmap-fonts cdr cjk cli cracklib crypt cups cvs dbus divx4linux dri dts dv dvd dvdr dvdread encode fame ffmpeg firefox fortran gdbm gnutls gpm hal iconv ieee1394 imap immqt-bc ipod ipv6 irc isdnlog java jpeg2k kde latex lm_sensors mad matroska midi mp3 mpeg mudflap mysql ncurses nls nptl nptlonly ogg opengl openmp oss pam pcre pdf perl png ppds pppd python qt quicktime readline real reflection scim session sms spl ssl subtitles subversion tcpd theora truetype truetype-fonts unicode usb vorbis win32codecs x86 xine xorg xscreensaver xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en de ko" USERLAND="GNU" VIDEO_CARDS="radeon" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Oh, I've just found an interesting point. When I set LC_CTYPE as "C", "grep -i" works without any problem. $ export LANG=C; export LC_CTYPE=C $ locale LANG=C LC_CTYPE=C LC_NUMERIC=C LC_TIME=C LC_COLLATE=C LC_MONETARY=C LC_MESSAGES=C LC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=C LC_IDENTIFICATION=C LC_ALL= $ grep -i aabbcc testfile AaBbCc $ $ export LC_CTYPE=en_US.UTF-8 $ locale LANG=C LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C LC_COLLATE=C LC_MONETARY=C LC_MESSAGES=C LC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=C LC_IDENTIFICATION=C LC_ALL= $ grep -i aabbcc testfile $ export LC_CTYPE=de_DE.UTF-8 $ grep -i aabbcc testfile $ export LC_CTYPE=ko_KR.UTF-8 $ grep -i aabbcc testfile $ It means I cannot use "grep -i" when I use UTF-8 locale for LC_CTYPE. I don't know anything about the debian patch which was added from grep 2.5.3, but that patch probably has a locale bug.
Well there's this 66-match_icase.patch from debian. When looking at the patch, there's the following "header": This fixes echo Y | LC_ALL=en_US.UTF-8 grep -i '[y]' The expected output is: Y Without this patch, it works on non UTF-8 environment, but fails on UTF-8 environment. Maybe the patch doesn't fix the problem properly...
If you exclude 65-dfa-optional.patch from your Gentoo portage of grep 2.5.3, you can make it work. So I tried to find out the real reason looking into 65-dfa-optional.patch, but I couldn't find anything wrong. I also tried to build grep 2.5.3 on Debian Linux, applying all the same patches as provided. In this case, however, "grep -i" works pretty well.
Created attachment 165996 [details] a temporary working version of portage ebuild file for grep 2.5.3
*** Bug 238163 has been marked as a duplicate of this bug. ***
i have the same problem here on an amd64 computer. I've masked grep 2.5.3 locally meanwhile.
vapier: any objections to moving the broken stuff to a p.mask'd -r2, and rolling out an -r1 that doesn't have the bug just so that everybody gets working systems again while we work it out?
Until the problem is solved a workaround could be: alias grep="LC_ALL=C grep"
The comment in 65-dfa-optional.patch says: >The DFA algorithm is slow with mutlibytes characters. >This patch disables the DFA algorithm, but it can be re-enabled by setting >the GREP_USE_DFA environment variable. Well, without this environment variable, neither echo Y | LC_ALL=en_US.UTF-8 grep -i '[y]' mentioned to be fixed by some other patch before, nor echo Bla | LC_ALL=de_DE.UTF-8 grep -i bla echo Bla | LC_ALL=de_DE.UTF-8 grep -i Bla works for me, and export GREP_USE_DFA=1 fixes all of this. Maybe this variable should be set by default (when you still want to use the patch)? I prefer grep to work correctly rather then faster but wrong...
considering this quietly breaks scripts it should be masked.
Please mark this as "blocks bug 239003" The broken -i switch in grep-2.5.3 prevents gnome-2.24 from building.
Mandriva dropped 65-dfa-optional.patch until it's fixed some weeks ago: https://qa.mandriva.com/show_bug.cgi?id=41922 for solving this
Seems that dropping the patch is not the best option. Debian package was not applying it as read in grep changelog: grep (2.5.3~dfsg-2) unstable; urgency=low * Removed 65-dfa-optional.patch. (Closes: #439827, #440195, #440342) -- Santiago Ruano Rincón <santiago@debian.org> Thu, 06 Sep 2007 00:16:20 -0500 as seems to be also causing: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=439827 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=440195 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=440342 But they reverted the changes as dropping the patch seemed to make grep a lot slower in utf8 systems: grep (2.5.3~dfsg-2.1) unstable; urgency=high * Non-maintainer upload. * Reinstate patches by Nicolas François < nicolas.francois@centraliens.net> Closes: #181378, #442882 But, in ubuntu changelog, I found the following: grep (2.5.3~dfsg-5ubuntu2) intrepid; urgency=low * debian/rules: Configure with --without-included-regex, so that the broken configure test does not erroneously think that our glibc doesn't support regexes. (It fails to use _GNU_SOURCE in configure). The internal one is broken and e. g. does not work with "grep -i". Thanks to Jean-Baptiste Lallement for tracking this down! (LP: #241990) And the bug was https://bugs.launchpad.net/ubuntu/+source/grep/+bug/241990 Then, maybe this is a better solution (other option could be check opensuse's grep package, that seems to apply a completely different patchset)
when I have tried to reproduce, I updated to affected grep version an I am unable to reproduce this. Reading Changelog, seems that vapier fixed it recently (maybe a version bump would be needed): 19 Sep 2008; Mike Frysinger <vapier@gentoo.org> grep-2.5.3.ebuild: Apply Debian patchset to fix "-o -i" failures. Affected people maybe should try to reemerge latest grep for getting the fix Good luck!
I'm not sure to follow you. The bug is about '-i' alone, i dont think i've used the '-o' option.
(In reply to comment #17) > I'm not sure to follow you. The bug is about '-i' alone, i dont think i've used > the '-o' option. > Well, I have tried to reproduce with only with "-i" and I can't reproduce (none of the test cases posted in this bug)
Same problem on my system, but it's not just an 'ignore case' bug, even this doesn't work: echo AA | grep -i "A" ; echo $? 1 Changing any LC* or LANG settings doesn't help, grep -i only works when setting GREP_USE_DFA=1 installed grep is: grep-2.5.3 with patches * Applying grep_2.5.3~dfsg-6.diff ... * Applying Debian patchset (2.5.3~dfsg-6) ... * 2-man_rgrep.patch ... * 55-bigfile.patch ... * 60-dfa.c-case_fold.patch ... * 61-dfa.c-case_fold-charclass.patch ... * 63-dfa.c-case_fold-range.patch ... * 64-egf-speedup.patch ... * 65-dfa-optional.patch ... * 66-match_icase.patch ... * 67-w.patch ... * 68-no-grep.texi.patch ... * 70-man_apostrophe.patch ... * 75-dfa_calloc.patch ... * 80-catalan_translation.patch ...
the testsuite also fails any tests using -i (i think there's 4 of them).
After some testing, configuring and compiling I found out that this bug is a configure problem. When emerging sys-apps/grep-2.5.3 with USE="pcre -nls -static" the configure options '--disable-nls --enable-perl-regexp' are used but this isn't enough. The grep-internal regex routines are still used, grep is not linked against libpcre. After adding --without-included-regex to configure the -i problem is gone. The test suite still reports one test script with 3 failing test cases not related to -i. IMHO two of these are broken test cases written probably for a buggy grep. It looks like there are still some bugs when using combinations of the grep flags -m -C -v.
and on my system grep is also working without libpcre (with sys-libs/glibc 2.8_p20080602) configure: --disable-perl-regexp --without-included-regex
(In reply to comment #21) > After some testing, configuring and compiling I found out > that this bug is a configure problem. > > When emerging sys-apps/grep-2.5.3 with USE="pcre -nls -static" > the configure options '--disable-nls --enable-perl-regexp' are used > but this isn't enough. The grep-internal regex routines are still > used, grep is not linked against libpcre. > After adding --without-included-regex to configure the -i problem > is gone. > The test suite still reports one test script with 3 failing test cases > not related to -i. IMHO two of these are broken test cases written > probably for a buggy grep. It looks like there are still some bugs > when using combinations of the grep flags -m -C -v. > Then, solution commented in comment #15 seems to work, thanks for testing :-)
*** Bug 239309 has been marked as a duplicate of this bug. ***
Created attachment 166970 [details] grep-2.5.3 ebuild that uses --without-included-regex if USE="pcre" Quick ebuild that also seems to solve the problem. When USE="pcre", it adds the --without-included-regex to ensure that the libpcre is linked against...
See also: https://bugs.edge.launchpad.net/ubuntu/+source/grep/+bug/241990/comments/8 Without --without-included-regex, the configure test erroneously believes glibc does not have regex support.
2.5.3-r1 in the tree now as fixed. --without-included-regex is always included, that's the only way I could make it pass all my tests (it was broken regardless of USE=+/-pcre).
