| Summary: | sys-apps/portage with bash-4.3_rc2 does not handle escaped plain chars in regexes | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Raphaël Droz <raphael.droz+floss> |
| Component: | Unclassified | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED INVALID | ||
| Severity: | major | CC: | raphael.droz+floss |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | https://lists.gnu.org/archive/html/bug-bash/2014-01/msg00149.html | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: | stacktrace of a src_compile failure | ||
Created attachment 370362 [details]
stacktrace of a src_compile failure
*** Bug 501222 has been marked as a duplicate of this bug. *** yes, you should not use bash-4.3 atm. it has changed the way it handles \ in regex compares. there's debate whether it will stay this way before 4.3 final. bash 4.3 is out. http://lists.gnu.org/archive/html/bug-bash/2014-02/msg00081.html ... and bash-4.3 doesn't have this problem |
=== Introduction $ python3.3<<<"import re; print('|'.join(sorted(re.escape(x) for x in set(['python_targets_python2_6']))))" python_targets_python2_6 $ python3.2<<<"import re; print('|'.join(sorted(re.escape(x) for x in set(['python_targets_python2_6']))))" python\_targets\_python2\_6 === Google-friendly keywords USE Flag 'python_targets_python2_6' not in IUSE for <ebuild> USE Flag 'python_targets_python2_7' not in IUSE for <ebuild> USE Flag 'python_targets_python3_2' not in IUSE for <ebuild> === How to reproduce $ eselect python set python3.2 $ emerge =app-shells/bash-4.2_p45 $ /bin/bash $ emerge dev-python/pyxdg # build successful $ emerge =app-shells/bash-4.3_rc2 $ /bin/bash $ emerge dev-python/pyxdg # build failed with the attached stacktrace === Explanation The issue itself is in /usr/lib/portage/bin/phase-helpers.sh where, line 195, [[ $u =~ $PORTAGE_IUSE ]] return FALSE. It means that the USE flag $u (eg: python_targets_python2_6) is not matched by the ebuild generated regular expression built from iuse_implicit. This regular expression comes from /usr/lib/portage/pym/portage/package/ebuild/config.py, line 106, inside _lazy_iuse_regex() There are 2 way to get different behaviours: 1) python 3.3 and 3.2 quote differently the underscores [1] 2) [2], [3] (and [4]) induce that bash 4.3 changed its handling of escaping in ERE regexp. While for python 3.2, the regexp built by _lazy_iuse_regex() is the same (given it does not depend upon bash), the strangeness appears in the use() function of phase-helpers.sh. Adding set -xv shows: bash 4.2: [[ ! python_targets_python2_6 =~ ^(alpha|amd64|amd64\-fbsd|amd64\-linux|arm|arm64|arm\-linux|elibc\_AIX|elibc\_Cygwin|elibc\_Darwin|elibc\_FreeBSD|elibc\_HPUX|elibc\_Interix|elibc\_NetBSD|elibc\_OpenBSD|elibc\_SunOS|elibc\_Winnt|elibc\_glibc|elibc\_mintlib|elibc\_musl|elibc\_uclibc|hppa|hppa\-hpux|ia64|ia64\-hpux|ia64\-linux|kernel\_AIX|kernel\_Cygwin|kernel\_Darwin|kernel\_FreeBSD|kernel\_HPUX|kernel\_Interix|kernel\_NetBSD|kernel\_OpenBSD|kernel\_SunOS|kernel\_Winnt|kernel\_freemint|kernel\_linux|m68k|m68k\-mint|mips|ppc|ppc64|ppc64\-linux|ppc\-aix|ppc\-macos|ppc\-openbsd|prefix|python\_targets\_python2\_6|python\_targets\_python2\_7|s390|sh|sparc|sparc64\-freebsd|sparc64\-solaris|sparc\-fbsd|sparc\-solaris|userland\_BSD|userland\_GNU|x64\-freebsd|x64\-macos|x64\-openbsd|x64\-solaris|x86|x86\-cygwin|x86\-fbsd|x86\-freebsd|x86\-interix|x86\-linux|x86\-macos|x86\-netbsd|x86\-openbsd|x86\-solaris|x86\-winnt)$ ]] bash 4.3: [[ ! python_targets_python2_6 =~ ^(alpha|amd64|amd64\\-fbsd|amd64\\-linux|arm|arm64|arm\\-linux|elibc\\_AIX|elibc\\_Cygwin|elibc\\_Darwin|elibc\\_FreeBSD|elibc\\_HPUX|elibc\\_Interix|elibc\\_NetBSD|elibc\\_OpenBSD|elibc\\_SunOS|elibc\\_Winnt|elibc\\_glibc|elibc\\_mintlib|elibc\\_musl|elibc\\_uclibc|hppa|hppa\\-hpux|ia64|ia64\\-hpux|ia64\\-linux|kernel\\_AIX|kernel\\_Cygwin|kernel\\_Darwin|kernel\\_FreeBSD|kernel\\_HPUX|kernel\\_Interix|kernel\\_NetBSD|kernel\\_OpenBSD|kernel\\_SunOS|kernel\\_Winnt|kernel\\_freemint|kernel\\_linux|m68k|m68k\\-mint|mips|ppc|ppc64|ppc64\\-linux|ppc\\-aix|ppc\\-macos|ppc\\-openbsd|prefix|python\\_targets\\_python2\\_6|python\\_targets\\_python2\\_7|s390|sh|sparc|sparc64\\-freebsd|sparc64\\-solaris|sparc\\-fbsd|sparc\\-solaris|userland\\_BSD|userland\\_GNU|x64\\-freebsd|x64\\-macos|x64\\-openbsd|x64\\-solaris|x86|x86\\-cygwin|x86\\-fbsd|x86\\-freebsd|x86\\-interix|x86\\-linux|x86\\-macos|x86\\-netbsd|x86\\-openbsd|x86\\-solaris|x86\\-winnt)$ ] In the later, both dash and underscores get a \\. The problem does not arise with python 3.3 since it does not add any backslashes at all. I've not exactly understood *how* this variable is initialized differently according to the bash version since I missed the link between python dictionary initialization and the shell envvars initialization. IMHO : don't escape that regexp in config.py (or even initialize a bash array). Oh and, please fix the comment of _lazy_iuse_regex() which states that [this regexp] "is used only to generate QA notices" ... which is plain wrong since 1) it generates ERROR since EAPI=5 2) it's also very useful in trashing the afternoon of some people :| [1] http://bugs.python.org/issue2650 [2] http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00154.html [3] http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00091.html [4] http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00147.html (I wonder about change "zz") Reproducible: Always