Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 501256 - sys-apps/portage with bash-4.3_rc2 does not handle escaped plain chars in regexes
Summary: sys-apps/portage with bash-4.3_rc2 does not handle escaped plain chars in reg...
Status: RESOLVED INVALID
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Portage team
URL: https://lists.gnu.org/archive/html/bu...
Whiteboard:
Keywords:
: 501222 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-02-13 22:00 UTC by Raphaël Droz
Modified: 2014-02-28 22:32 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
stacktrace of a src_compile failure (stacktrace.log,858 bytes, text/x-log)
2014-02-13 22:02 UTC, Raphaël Droz
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Raphaël Droz 2014-02-13 22:00:32 UTC
=== Introduction
$ python3.3<<<"import re; print('|'.join(sorted(re.escape(x) for x in set(['python_targets_python2_6']))))"
python_targets_python2_6
$ python3.2<<<"import re; print('|'.join(sorted(re.escape(x) for x in set(['python_targets_python2_6']))))"
python\_targets\_python2\_6


=== Google-friendly keywords
USE Flag 'python_targets_python2_6' not in IUSE for <ebuild>
USE Flag 'python_targets_python2_7' not in IUSE for <ebuild>
USE Flag 'python_targets_python3_2' not in IUSE for <ebuild>


=== How to reproduce
$ eselect python set python3.2
$ emerge =app-shells/bash-4.2_p45
$ /bin/bash
$ emerge dev-python/pyxdg
# build successful
$ emerge =app-shells/bash-4.3_rc2
$ /bin/bash
$ emerge dev-python/pyxdg
# build failed with the attached stacktrace


=== Explanation
The issue itself is in /usr/lib/portage/bin/phase-helpers.sh where, line 195, [[ $u =~ $PORTAGE_IUSE ]] return FALSE.
It means that the USE flag $u (eg: python_targets_python2_6) is not matched by the ebuild generated regular expression built from iuse_implicit.

This regular expression comes from /usr/lib/portage/pym/portage/package/ebuild/config.py, line 106, inside _lazy_iuse_regex()

There are 2 way to get different behaviours:
1) python 3.3 and 3.2 quote differently the underscores [1]
2) [2], [3] (and [4]) induce that bash 4.3 changed its handling of escaping in ERE regexp.

While for python 3.2, the regexp built by _lazy_iuse_regex() is the same (given it does not depend upon bash), the strangeness appears in the use() function of phase-helpers.sh.
Adding set -xv shows:

bash 4.2:
[[ ! python_targets_python2_6 =~ ^(alpha|amd64|amd64\-fbsd|amd64\-linux|arm|arm64|arm\-linux|elibc\_AIX|elibc\_Cygwin|elibc\_Darwin|elibc\_FreeBSD|elibc\_HPUX|elibc\_Interix|elibc\_NetBSD|elibc\_OpenBSD|elibc\_SunOS|elibc\_Winnt|elibc\_glibc|elibc\_mintlib|elibc\_musl|elibc\_uclibc|hppa|hppa\-hpux|ia64|ia64\-hpux|ia64\-linux|kernel\_AIX|kernel\_Cygwin|kernel\_Darwin|kernel\_FreeBSD|kernel\_HPUX|kernel\_Interix|kernel\_NetBSD|kernel\_OpenBSD|kernel\_SunOS|kernel\_Winnt|kernel\_freemint|kernel\_linux|m68k|m68k\-mint|mips|ppc|ppc64|ppc64\-linux|ppc\-aix|ppc\-macos|ppc\-openbsd|prefix|python\_targets\_python2\_6|python\_targets\_python2\_7|s390|sh|sparc|sparc64\-freebsd|sparc64\-solaris|sparc\-fbsd|sparc\-solaris|userland\_BSD|userland\_GNU|x64\-freebsd|x64\-macos|x64\-openbsd|x64\-solaris|x86|x86\-cygwin|x86\-fbsd|x86\-freebsd|x86\-interix|x86\-linux|x86\-macos|x86\-netbsd|x86\-openbsd|x86\-solaris|x86\-winnt)$ ]]


bash 4.3:
[[ ! python_targets_python2_6 =~ ^(alpha|amd64|amd64\\-fbsd|amd64\\-linux|arm|arm64|arm\\-linux|elibc\\_AIX|elibc\\_Cygwin|elibc\\_Darwin|elibc\\_FreeBSD|elibc\\_HPUX|elibc\\_Interix|elibc\\_NetBSD|elibc\\_OpenBSD|elibc\\_SunOS|elibc\\_Winnt|elibc\\_glibc|elibc\\_mintlib|elibc\\_musl|elibc\\_uclibc|hppa|hppa\\-hpux|ia64|ia64\\-hpux|ia64\\-linux|kernel\\_AIX|kernel\\_Cygwin|kernel\\_Darwin|kernel\\_FreeBSD|kernel\\_HPUX|kernel\\_Interix|kernel\\_NetBSD|kernel\\_OpenBSD|kernel\\_SunOS|kernel\\_Winnt|kernel\\_freemint|kernel\\_linux|m68k|m68k\\-mint|mips|ppc|ppc64|ppc64\\-linux|ppc\\-aix|ppc\\-macos|ppc\\-openbsd|prefix|python\\_targets\\_python2\\_6|python\\_targets\\_python2\\_7|s390|sh|sparc|sparc64\\-freebsd|sparc64\\-solaris|sparc\\-fbsd|sparc\\-solaris|userland\\_BSD|userland\\_GNU|x64\\-freebsd|x64\\-macos|x64\\-openbsd|x64\\-solaris|x86|x86\\-cygwin|x86\\-fbsd|x86\\-freebsd|x86\\-interix|x86\\-linux|x86\\-macos|x86\\-netbsd|x86\\-openbsd|x86\\-solaris|x86\\-winnt)$ ]

In the later, both dash and underscores get a \\.
The problem does not arise with python 3.3 since it does not add any backslashes at all.


I've not exactly understood *how* this variable is initialized differently according to the bash version since I missed the link between python dictionary initialization and the shell envvars initialization.


IMHO : don't escape that regexp in config.py (or even initialize a bash array).
Oh and, please fix the comment of _lazy_iuse_regex() which states that
[this regexp] "is used only to generate QA notices"
... which is plain wrong since
1) it generates ERROR since EAPI=5
2) it's also very useful in trashing the afternoon of some people  :|




[1] http://bugs.python.org/issue2650
[2] http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00154.html
[3] http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00091.html
[4] http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00147.html (I wonder about change "zz")

Reproducible: Always
Comment 1 Raphaël Droz 2014-02-13 22:02:33 UTC
Created attachment 370362 [details]
stacktrace of a src_compile failure
Comment 2 Raphaël Droz 2014-02-13 22:56:01 UTC
*** Bug 501222 has been marked as a duplicate of this bug. ***
Comment 3 SpanKY gentoo-dev 2014-02-14 17:19:56 UTC
yes, you should not use bash-4.3 atm.  it has changed the way it handles \ in regex compares.  there's debate whether it will stay this way before 4.3 final.
Comment 4 Raphaël Droz 2014-02-28 16:42:56 UTC
bash 4.3 is out.
http://lists.gnu.org/archive/html/bug-bash/2014-02/msg00081.html
Comment 5 SpanKY gentoo-dev 2014-02-28 22:32:03 UTC
... and bash-4.3 doesn't have this problem