Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 171630 - app-shells/dash does not assign variables mid-statement
Summary: app-shells/dash does not assign variables mid-statement
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-03-20 22:38 UTC by Roy Marples (RETIRED)
Modified: 2012-11-05 16:23 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Fix the issue (linux-header-install-sh.patch,709 bytes, patch)
2007-03-20 22:40 UTC, Roy Marples (RETIRED)
Details | Diff
emerge --info (emerge.info,7.89 KB, text/plain)
2007-03-21 08:55 UTC, Roy Marples (RETIRED)
Details
Use one var (Makefile-onevar.patch,670 bytes, patch)
2007-05-15 15:11 UTC, Roy Marples (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Roy Marples (RETIRED) gentoo-dev 2007-03-20 22:38:38 UTC
With dash as /bin/sh, make headers_install doesn't create the __ASM_STUB_ header properly

Result:
#ifndef __ASM_STUB_
#define __ASM_STUB_

Expect:
#ifndef __ASM_STUB_TYPES_H
#define __ASM_STUB_TYPES_H
Comment 1 Roy Marples (RETIRED) gentoo-dev 2007-03-20 22:40:48 UTC
Created attachment 113929 [details, diff]
Fix the issue

Basically dash requires a semi colon here.
Comment 2 SpanKY gentoo-dev 2007-03-21 05:14:00 UTC
that doesnt sound right to me at all

why would you require a semicolon in order to set a variable ?  you only need whitespace which is what the makefile provides ...
Comment 3 Kevin F. Quinn (RETIRED) gentoo-dev 2007-03-21 08:35:56 UTC
Maybe it's because it's on the rhs of an assignment already - I'm guessing the value cmd_gen is interpreted by the shell rather than make.  However whitespace-separated assignments are also valid for the shell:

$ F=one G=two
$ echo $F $G
one two

This works for me in bash, busybox - and dash-0.5.2.7, dash-0.5.3.7, so the Roy's analysis is incomplete.


I had a quick look at the posix spec:

http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_09_01

implies a command consisting of just a sequence of variables should work and will affect the environment.


Roy; could you explain why you think the original makefile is broken?
Maybe it's not dash causing the problem, but the version of make being used.
Comment 4 Roy Marples (RETIRED) gentoo-dev 2007-03-21 08:55:41 UTC
Created attachment 113954 [details]
emerge --info

I agree, and taking that code snippet by itself it does not need the semi colon. But the thing as a whole requires it. So something in the parsing is causing dash to barf here. What that something is I am not sure. Any insight into finding out the root cause would be nice :)
Comment 5 SpanKY gentoo-dev 2007-03-24 07:51:34 UTC
it's because dash does not export evaluated variables until the next statement

the headers basically do something like:
A=moo B="$A more"

dash does not make $A available until the next statement, that's why inserting the semi-colon there "fixes" things

so run this command chain to see the difference between bash and dash:
unset A B; A="moo" B="$A more"; echo $A , $B
Comment 6 SpanKY gentoo-dev 2007-03-24 09:11:55 UTC
my reading of the spec says that dash is broken here ... ive also checked on the bash mailing list and they agree ...

so time to bug the dash maintainers :P
Comment 7 Kevin F. Quinn (RETIRED) gentoo-dev 2007-03-24 10:37:03 UTC
busybox:
$ bb
~ $ unset A B; A="moo" B="$A more"; echo $A , $B
moo , more

so busybox is doing the same as dash.

It would be interesting to know what the Unix shells do (solaris, hpux etc) - given that the posix spec in this sort of area should be documenting the common denominator.  I.e. if bash is the _only_ shell interpreting things this way, it's not likely that posix is intended to specify it (it is perhaps intended to leave it unspecified, to allow for variations in common sh implementations.

If posix is intended to allow for the variation, then it would make sense to avoid depending on this behaviour, especially where it costs nothing.
Comment 8 Roy Marples (RETIRED) gentoo-dev 2007-03-24 11:44:09 UTC
FreeBSD sh does the same as dash and bb - not too suprising as they're all descended from ash.
Comment 9 Kevin F. Quinn (RETIRED) gentoo-dev 2007-03-24 12:08:39 UTC
Just tried OpenSolaris 5.11 (belenix) and it's sh (unsurprisingly) does the same as dash, bb.
Comment 10 SpanKY gentoo-dev 2007-03-24 21:37:52 UTC
tbh, i dont really thing it matters what shells do what in general ... if a shell's behavior differs from POSIX, then it's broken ... i'd say the same thing if we thought bash was doing the wrong thing
Comment 11 Kevin F. Quinn (RETIRED) gentoo-dev 2007-03-24 22:16:04 UTC
That's just it though - I'm coming to the conclusion that bash is doing the wrong thing in this case.  We'll see how the thread develops on bug-bash.
Comment 12 Roy Marples (RETIRED) gentoo-dev 2007-03-25 02:45:23 UTC
(In reply to comment #10)
> tbh, i dont really thing it matters what shells do what in general ... if a
> shell's behavior differs from POSIX, then it's broken ... i'd say the same
> thing if we thought bash was doing the wrong thing

Where in the POSIX shell spec are you seeing a defined behaviour as I cannot find any reference to this.

FWIW, C does not define a paramter evaluation order either - something that Cardoe found out. It's not right to left - it could be left to right or something else.

So taking the above into account, which is true?

unset A B; A="moo" B="$A, more" A="foo"; echo $A $B

If this is the case then the only way to guarantee correct parameter evaluation is to separate them by statements.

unset A B; A="moo"; B="$A, more"; A="foo"; echo $A $B

Of course, I could be talking out of my ass and vapier will pull the POSIX page out explaining why I am wrong :)
Comment 13 SpanKY gentoo-dev 2007-03-25 04:11:50 UTC
Kevin already posted the link to the relevant section in the POSIX spec ... in there it states that evaluation happens "from the beginning of the command text to the end"

i dont see how C's handling of , and ; is relevant here ... we're talking about POSIX shell
Comment 14 Kevin F. Quinn (RETIRED) gentoo-dev 2007-03-25 10:30:14 UTC
Got some more clarification from the bash maintainer.  It does appear that bash is posix-compliant in this respect, and that it is dash that is non-compliant (along with pretty much all other shells).  The other shells I mentioned aren't Posix shells and don't claim compliance.  ksh93 (which makes a big thing about posix-compliance - didn't think to try it before) does the same as bash.

Thread on bug-bash at:
http://www.mail-archive.com/bug-bash%40gnu.org/msg02622.html
http://lists.gnu.org/archive/html/bug-bash/2007-03/msg00075.html

Summary - take it up with the dash maintainers, since NetBSD does care about Posix-compliance for its shell (according to http://netbsd.gw.com/cgi-bin/man-cgi?sh++NetBSD-current).  Refer them to the above thread (better than paraphrasing what we've said here).
Comment 15 Doug Goldstein (RETIRED) gentoo-dev 2007-05-09 17:24:17 UTC
FFS can we just make a simple fix by adding a ; It's not like making this change will break bash or rape kittens.

I have contacted dash upstream and CC'd everyone on this bug.

When the time comes, you can pull out the ; and be happy to be ; free, but until then add the ;
Comment 16 Doug Goldstein (RETIRED) gentoo-dev 2007-05-11 13:56:06 UTC
Spanky: Are you not upstream for busybox? Can you file this bug with them or fix it?
Comment 17 SpanKY gentoo-dev 2007-05-12 11:29:05 UTC
i really dont think busybox is an issue at this time

busybox shell is used at runtime on embedded systems, not on a development system

i'm sure there's plenty of other things dash has fixed that have not made their way into busybox ... the shell situation in busybox is just all fucked anyways
Comment 18 Roy Marples (RETIRED) gentoo-dev 2007-05-12 23:34:05 UTC
If you won't add that little semi colon, could you at least put a test into linux-headers ebuild so that it won't install when /bin/sh isn't bash OR force the Makefile into using bash somehow?

Just because you don't think busybox should be used as a shell on our boxes doesn't stop the fact that our users for all their sins might want to.
Comment 19 SpanKY gentoo-dev 2007-05-13 22:17:37 UTC
busybox's shell will fail a lot more POSIX tests than this one ... simply put, it is not supported as a development shell and only gets limited support as a runtime shell
Comment 20 Doug Goldstein (RETIRED) gentoo-dev 2007-05-13 22:28:47 UTC
Fine, can you just add a little semi-colon? That's all we want. You would have spent less time adding the semi-colon then trying to argue with us.
Comment 21 Kevin F. Quinn (RETIRED) gentoo-dev 2007-05-15 05:32:19 UTC
Note that adding the ';' means FNAME gets set in the environment rather than just for the one command.  That means the maintainer would need to verify (and re-verify on every release) that FNAME isn't used elsewhere.

I notice we're now talking about busybox, when the bug was originally about dash.  Has dash been fixed?
Comment 22 Doug Goldstein (RETIRED) gentoo-dev 2007-05-15 14:55:58 UTC
Kevin: You were CC'd in my original e-mail to the DASH maintainer. I have not received an answer from him.
Comment 23 SpanKY gentoo-dev 2007-05-15 15:00:49 UTC
we're not talking about busybox, forget it was mentioned
Comment 24 Roy Marples (RETIRED) gentoo-dev 2007-05-15 15:11:24 UTC
Created attachment 119375 [details, diff]
Use one var

(In reply to comment #21)
> Note that adding the ';' means FNAME gets set in the environment rather than
> just for the one command.  That means the maintainer would need to verify (and
> re-verify on every release) that FNAME isn't used elsewhere.

Valid point. This new patch addresses this concern.

> I notice we're now talking about busybox, when the bug was originally about
> dash.  Has dash been fixed?

No, but it applies to current busybox in portage as well, regardless of Mikes views about its suitability as /bin/sh for a Gentoo development system. Yes, dash is probably more POSIX compliant than busybox, but there's also no need to go out of our way to needlessly ignore it when patches are trivial.
Comment 25 SpanKY gentoo-dev 2007-05-15 15:21:59 UTC
the triviality of a patch is irrelevant when referring to its correctness
Comment 26 Roy Marples (RETIRED) gentoo-dev 2007-05-15 15:41:54 UTC
(In reply to comment #25)
> the triviality of a patch is irrelevant when referring to its correctness

What part of the patch is incorrect?
Comment 27 SpanKY gentoo-dev 2007-05-19 04:20:26 UTC
the part where it's needed
Comment 28 Dror Levin (RETIRED) gentoo-dev 2009-12-12 23:55:01 UTC
Is this bug still relevant? Seems like dash still behaves this way, but what packages are affected, if any?
Comment 29 SpanKY gentoo-dev 2009-12-13 14:40:10 UTC
packages that either have changed their syntax or changed their shebang
Comment 30 SpanKY gentoo-dev 2012-11-05 16:23:55 UTC
seems to work with dash-0.5.7