| Summary: | app-shells/dash: Please let echo honor escape sequences for XSI compliance | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Martin Väth <martin> |
| Component: | Current packages | Assignee: | Gentoo's Team for Core System packages <base-system> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | floppym, ulm, whissi |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | http://www.mail-archive.com/dash@vger.kernel.org/msg01094.html | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=337329 https://bugs.gentoo.org/show_bug.cgi?id=527848 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
|
Description
Martin Väth
2016-08-07 14:07:19 UTC
Perhaps a reasonable compromise would be a USE flag for those people who want the behavior provided by the upstream software instead of the slightly crippled version provided by the current ebuild. i already listed a number of scenarios beyond embedded systems where the extensions do not exist. further, XSI compliance does *not* have an -e option, but always respects the escape sequences which is not how shells that do recognize the -n option generally behave. the use of <=autoconf-2.13 is an extremely low number (and you'll find other portable problems than just echo as Gentoo has patched them over the years). i don't find those compelling enough of a case to consider. if this is important to you, you can revert locally, or use a diff shell. I tend to agree with floppym on this. We should probably allow echo -n to work the same way upstream does. I do not see a reason to patch our version to behave differently. A good compromise would be to keep the patch, but apply it conditionally, and not by default. (In reply to William Hubbs from comment #3) not going to happen -- this patch has done far more good than bad and uncovered real bugs that no one noticed before (In reply to SpanKY from comment #2) > i already listed a number of scenarios beyond embedded systems Certainly not with an official POSIX certification - they wouldn't get it without being XSI compliant. Of course, without certification, systems can break whatever they want to. What does this tell us? Is it an excuse to copy the breakage of XSI compliance of these systems intentionally and being the only distribution which does so (i.e., intentionally)? Moreover, is it a valid excuse to break dash upstream's intended code? > which is not how shells that do recognize the -n option generally behave. For instance, zsh behaves like this. And moreover: vanilla dash behaves like this. So you do consider dash upstream's decision to follow XSI and support the optional -n a bug which needs to be patched? Did you file a bug for this upstream? If yes, why didn't dash follow your bugfix after so many versions? Maybe they didn't show the intention to break XSI compliance and user experience like you want to? How shells in general _do_ behave is to support either the string expansion always _or_ with the -e flag. In fact, historically bash was for a long time the only exception of not always doing string expansion but requiring the -e flag; and those shells which appeared later and followed bash and thus have -e, also all have -n in addition. Which is why many historical scripts (in particular, all configure files produced by <=autoconf-2.13) rely on at least one of these two. Note that all these historical script couldn't rely on printf, because printf was not available on many systems. Do you really believe it was the intention of the POSIX standard that new shells should break all these historical scripts? In fact, to my knowledge there is no shell (except for gentoo's crippled dash) which has no way to output some escapes with echo and which does not even have a way to suppress the trailing line-feed without using the historically non-available printf. If you really want to break upstream's intention and break user experience to force the POSIX minimum requirement, then patch out -n recognition, but not the XSI compliant behaviour. > not going to happen Yeah, where would we end if users would have a choice with gentoo: Between being your test animals (not explicitly warning them, but hidden to them unless they are unnaturally careful and check every patch) or having the possibility to install a working and XSI-compliant shell as was apparently dash upstream's intention. > this patch has done far more good than bad Let's compare: On the downside there is broken user experience, breakage of some ebuilds (at the very least breakage of their output), and breakage of old scripts which work with any other shell except for the non-standard gentoo-dash. On the plus side: Maybe one or two POSIX incompatibilities could be detected which otherwise wouldn't ever have hurt anybody. Sounds like a good reason to justify forcing the users to a non-POSIX system. I didn't see you hastily fixing all ebuilds which use some configure with autoconf-2.13 or report these "issues" upstream. I suppose upstream will just laugh at you if you explain them that the need to "fix" it is that their configure fails to run with a shell which is intentionally incompatible to all other shells and not even XSI compliant. I already made this experience. so it is now up to you to convince all upstream. BTW: Why didn't you do the same patch to bash? Then much more of these "bugs" would be reported! Are you afraid of offending a larger user base than those which you currently misuse? (Which are those who use dash as a system shell; presumably they do so _because_ they expect it to be standard-compliant which in gentoo it actually is not) Other ebuilds have at least USE=vanilla for such controversial patches. I'd like to say, as a simple user, that by installing dash, I was expecting it to do exactly what it does currently (on Gentoo): support POSIX.1-2008, without extensions. I'd argue that most people installing dash or posh expect the same result. Also, echo -n or -e is currently the Shell equivalent of C's undefined behaviour; supporting it just because some scripts use it seems wrong to me. (In reply to johncarmack from comment #6) > I'd like to say, as a simple user, that by installing dash, I was expecting > it to do exactly what it does currently (on Gentoo): support POSIX.1-2008, > without extensions Which is not the case: dash has many extensions over POSIX like e.g.: local test a -gt b test with >4 arguments, -a, -o, '(', etc. Moreover, dash does not follow POSIX everywhere (e.g. it completely lacks multi-character support). > I'd argue that most people installing dash or posh expect the same result. I'd say that most people installing dash expect dash and not what some gentoo developer thinks what dash should "ideally" be. In particular, scripts with #!/bin/dash from the net should just work as upstream intends and not randomly fail on gentoo, because dash on gentoo behaves differently than on all other distributions. > Also, echo -n or -e is currently the Shell equivalent of C's undefined > behaviour ... like GNU extensions. BTW. this bug is not even about echo -n or -e but about e.g. "echo 'a\c'" not following POSIX. > supporting it just because some scripts use it seems wrong to me. We are not speaking about "supporting it", but following upstream's code. So you do vote that Gentoo should patch out all GNU extensions from all GNU and other tools unconditionally, to guarantee that everybody has at most the POSIX-minimal system instead of the one which corresponds to upstream packages. If not, then your complaint should not go to gentoo but to upstream: Complain at dash upstream that they decide to support some things which go beyond minimal POSIX-embedded specification and that you consider it wrong that scripts working since 20 years continue to work instead of breaking and require patching - only because some standard specification found no minimal consensus over existing shells and allowed exceptions for some minimal situations like embedded systems. (In reply to Martin Väth from comment #7) > test a -gt b Sorry, I meant -nt Now that you say it, I guess it's dash's fault and emerging dash should give dash (with its defaults). I'm pretty angry with POSIX, anyway, since they allow '\n' in filenames but don't provide the -0 or -z option as a standard for all tools. Note that the person who is the author of the POSIX compilance test suite (Geoff Clare) claims that you can only get a POSIX certification for an embedded system if you do not implement certain "extensions". If you like to certify a desktop system, you need to include support for "extensions" as well. Support for multi-byte characters (like UTF-8) is part of that requirements and dash does not support them (try to set up a multi-byte locale and then include a multi-byte character in IFS with dash. You will see that dash does not handle that case correctly. For the same reason, you cannot decide to ignore the escape sequences with the echo command. Now it would be interesting what Gentoo likes to be.... * an embedded system with minimalistic POSIX support or * something that is close to a POSIX desktop (In reply to SpanKY from comment #4) > (In reply to William Hubbs from comment #3) > > not going to happen -- this patch has done far more good than bad and > uncovered real bugs that no one noticed before It would be nice if you did verify your claim. Given that this patch cannot help POSIX scripts, it seems to be no more than an excuse. For the record: I had to patch games-puzzle/pingus because of our crippled dash: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2d7c75bb39c63bcead7154614999fbdd2da3a538 I am not sure this change of dash's behavior does a good service. This even turned up in upstream's mailing list (see URL). After discussing this with Lars, I have moved the "dumb-echo" patch behind a USE flag. https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=68b674d3521a1e08d57b51270de888bfff0beeac (In reply to Martin Väth from comment #5) XSI extensions are not required to get an official POSIX certification. even then, that certification is generally meaningless: very few people spend the time or money to get it, and when they do, it's for a very specific configuration of the system that is never shipped by default and no users ever see. citing support for certs that no one gets or cares about doesn't sway positions. citing some shells that work this way by default is also irrelevant. i already provided multiple examples of other shells that don't, and none of them violate POSIX. nor this patch doesn't violate POSIX. dash's default behavior is known to break packages (where an escape sequence showed up in the echo, but -e wasn't used), but the only packages this patch breaks are ones that are already broken and need to be fixed. as for upstream dash and their intentions, this behavior is strongly inviolation of their own mission statement: DASH is a POSIX-compliant implementation of /bin/sh that aims to be as small as possible. It does this without sacrificing speed where possible. In fact, it is significantly faster than bash (the GNU Bourne-Again SHell) for most tasks. this patch maintains POSIX compliance while speeding it up and shrinking it. i've reverted the change to default this to off and put it behind USE=vanilla. that flag already strongly implies "the build is not supported". https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1a2e3cf9241bb6fef01d8d5e065e790fc713aa86 (In reply to joerg.schilling from comment #11) if you can't be bothered to read the already cited references, then troll elsewhere. you're a well known troll already in the open source world. (In reply to Lars Wendler (Polynomial-C) from comment #12) you mean you made it more portable for more shells (In reply to SpanKY from comment #15) I would strongly prefer that you install the vanilla version by default. If you want your version to be the default, please submit the changes upstream. (In reply to Mike Gilbert from comment #16) we've been running this way for over 5 years. the results have been finding broken packages and getting them fixed. i see no reason to change this now. upstream behavior also breaks scripts and is a bad default and doesn't match other shells. there's no requirement to support this. if you want the bad behavior, you can get it via USE=vanilla. here's another example i came across today of dash's misbehavior breaking things: https://sourceware.org/bugzilla/show_bug.cgi?id=16704 |
