Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 608644 - =sys-apps/grep-2.28 broken with multiple -e patterns
Summary: =sys-apps/grep-2.28 broken with multiple -e patterns
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords: PMASKED, REGRESSION
: 608652 608658 608696 608824 608892 (view as bug list)
Depends on:
Blocks:
 
Reported: 2017-02-08 17:43 UTC by Ben Kohler
Modified: 2017-02-10 21:51 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
gentoo-sources-4.9.8 build.log (build.log,1.92 KB, text/plain)
2017-02-08 17:43 UTC, Ben Kohler
Details
(compressed) genpatches-4.9-10.base.tar.err (genpatches-4.9-10.base.tar.err.gz,190.32 KB, application/x-gzip)
2017-02-08 17:46 UTC, Ben Kohler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ben Kohler gentoo-dev 2017-02-08 17:43:32 UTC
Created attachment 462870 [details]
gentoo-sources-4.9.8 build.log

Since upgrading to grep-2.28, all kernel source ebuilds using kernel-2 eclass are failing in the src_unpack phase.
Comment 1 Ben Kohler gentoo-dev 2017-02-08 17:46:28 UTC
Created attachment 462872 [details]
(compressed) genpatches-4.9-10.base.tar.err
Comment 2 Thomas Deutschmann gentoo-dev Security 2017-02-08 18:33:49 UTC
With grep-2.27-r1:

> $ grep -V
> grep (GNU grep) 2.27
> Copyright (C) 2016 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
> $ echo "/var/tmp/portage/sys-kernel/gentoo-sources-4.9.8/distdir/genpatches-4.9-10.base.tar.xz" | grep -qs -e '\.tar' -e '\.tbz' -e '\.tgz'
> $ echo $?
> 0

With grep-2.28:

> $ grep -V
> grep (GNU grep) 2.28
> Copyright (C) 2017 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
> $ echo "/var/tmp/portage/sys-kernel/gentoo-sources-4.9.8/distdir/genpatches-4.9-10.base.tar.xz" | grep -qs -e '\.tar' -e '\.tbz' -e '\.tgz'
> $ echo $?
> 1

So with grep-2.28 the if clause in https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/kernel-2.eclass#n1136 is failing and it is taking the "else" route.
Comment 3 Matthias Maier gentoo-dev 2017-02-08 18:37:42 UTC
Offending snippet of code:

kernel-2.eclass:

"""
  1134     #unpack any passed tarballs                      
  1135     for i in ${UNIPATCH_LIST}; do              
  1136         if echo ${i} | grep -qs -e "\.tar" -e "\.tbz" -e "\.tgz" ; then
  ...
  1156         else
  ...
  1200         fi
"""


With grep-2.28 the if condition in line 1136 is wrongly evaluated to false.
Comment 4 Matthias Maier gentoo-dev 2017-02-08 18:44:16 UTC
Oh wow, that's horrible:

grep-2.28:

   % echo ".tar" | grep -e "\.tar"; echo $?           
   .tar
   0

   % echo ".tar" | grep -e "\.tar" -e "\.tbz"; echo $?
   1

grep-2.27:

  % echo ".tar" | grep -e "\.tar"; echo $?           
  .tar
  0

  % echo ".tar" | grep -e "\.tar" -e "\.tbz"; echo $?
  .tar
  0


I recommend immediate masking of grep-2.28 until we have thoroughly investigated this logical change.
Comment 5 Tetja Rediske 2017-02-08 18:50:21 UTC
What does work:

echo ".tar" | grep -e '\.tar' -e '.tbz'
0

Seems like from the 2nd -e onwards '.' looses its special meaning.
Comment 6 Matthias Maier gentoo-dev 2017-02-08 18:56:33 UTC
No, it doesn't. It is much weirder.

This matches:

echo "atbz" | grep -e "\.tar" -e ".tbz"; echo $?
atbz
^^^^
0

And this as well:

echo ".tbz" | grep -e "\.tar" -e ".tbz"; echo $? 
.tbz
^^^^
0

And this reports a match on "ta":

echo ".tar" | grep -e "\.tar" -e "tar"; echo $?
.tar
 ^^
0

(where it should match ".tar" entirely.)
Comment 7 Lars Wendler (Polynomial-C) gentoo-dev 2017-02-08 19:18:37 UTC
commit 0106f0a2645379f39df58746702ac979fd2c3c1f
Author: Lars Wendler <polynomial-c@gentoo.org>
Date:   Wed Feb 8 20:16:58 2017

    package.mask: Masked =sys-apps/grep-2.28 (bug #608644).
Comment 8 Tetja Rediske 2017-02-08 19:24:36 UTC
Tracked it down to this commit:

290ca116c9172d97b2b026951fac722d3bd3ced9 is the first bad commit
commit 290ca116c9172d97b2b026951fac722d3bd3ced9
Author: Paul Eggert <eggert@cs.ucla.edu>
Date:   Tue Dec 20 18:04:39 2016 -0800

    grep: fix performance with multiple patterns

    Problem reported by Jaroslav Skarvada (Bug#22357).
    * NEWS: Document this and other recent performance fixes.
    * src/grep.c (E_MATCHER_INDEX): New constant.
    (all_single_byte_after_folding):
    New function, split out from fgrep_icase_available.
    (fgrep_icase_available): Use it.
    (try_fgrep_pattern): New function, which also uses it.
    (main): With two or more patterns, use try_fgrep_pattern to fix
    performance regression.  The number "two" here is just a heuristic.

:100644 100644 aa0483b70b8eadfa64e27682650a279cde04b556 2e7d1aea21dc240f32f2168e77cf2b98a2e7d86f M      NEWS
:040000 040000 6d750b93d03a481ce29ca9d35afaf7c1445cec98 c1afd951f43433a64481f93192f4c9bb6ecb3f42 M      src
Comment 9 Lars Wendler (Polynomial-C) gentoo-dev 2017-02-08 19:42:24 UTC
*** Bug 608652 has been marked as a duplicate of this bug. ***
Comment 10 Magnus Granberg gentoo-dev 2017-02-08 20:12:51 UTC
*** Bug 608658 has been marked as a duplicate of this bug. ***
Comment 11 Lars Wendler (Polynomial-C) gentoo-dev 2017-02-08 21:40:17 UTC
commit 7ed559030a245b1a7fa0a2260fc8af54768e1c76
Author: Lars Wendler <polynomial-c@gentoo.org>
Date:   Wed Feb 8 22:36:46 2017

    package.mask: Removed grep-2.28 mask.

commit aa2f866c5f633501863c0fb48ea7d368761596d2
Author: Lars Wendler <polynomial-c@gentoo.org>
Date:   Wed Feb 8 22:36:09 2017

    sys-apps/grep: Revbump to fix multiple pattern issue (bug #608644).

    Removed old.

    Package-Manager: Portage-2.3.3, Repoman-2.3.1
Comment 12 Lars Wendler (Polynomial-C) gentoo-dev 2017-02-09 09:08:58 UTC
*** Bug 608696 has been marked as a duplicate of this bug. ***
Comment 13 SpanKY gentoo-dev 2017-02-09 15:23:44 UTC
(In reply to Lars Wendler (Polynomial-C) from comment #11)

nice work!
Comment 14 Magnus Granberg gentoo-dev 2017-02-10 21:50:41 UTC
*** Bug 608824 has been marked as a duplicate of this bug. ***
Comment 15 Magnus Granberg gentoo-dev 2017-02-10 21:51:38 UTC
*** Bug 608892 has been marked as a duplicate of this bug. ***