Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 245356

Summary: sys-apps/portage should remove POSIXLY_CORRECT from env
Product: Portage Development Reporter: Arthur Hagen <art-gt>
Component: Core - ConfigurationAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal Keywords: InVCS
Priority: High    
Version: 2.0   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 210077, 216231    
Attachments: filter POSIXLY_CORRECT, applies to portage-2.1.4.5
filter POSIXLY_CORRECT, applies to portage-2.2_rc13

Description Arthur Hagen 2008-11-03 03:41:48 UTC
emerge consistently fails on emerging packages if POSIXLY_CORRECT is set.

Example:
The following works fine:
# env -u POSIXLY_CORRECT emerge --oneshot eselect-vi

However:
# env POSIXLY_CORRECT=1 emerge --oneshot eselect-vi
Calculating dependencies... done!
>>> Verifying ebuild Manifests...

>>> Emerging (1 of 1) app-admin/eselect-vi-1.1.5 to /
/usr/lib/portage/bin/ebuild.sh: line 1118: `debug-print': not a valid identifier


The cause of the problem is found in the bash reference, section 6.11 "Bash POSIX mode":

...
15. Function names must be valid shell names. That is, they may not contain characters other than letters, digits, and underscores, and may not start with a digit. Declaring a function with an invalid name causes a fatal syntax error in non-interactive shells.
...

Since scripts running with POSIXLY_CORRECT set generally also runs with it unset, there's no penalty to making sure that Gentoo system scripts will run also when it is set.  The problem is most likely going to bite those who try to bootstrap from a system where POSIXLY_CORRECT is the default, but might also bite regular users who set it system-wide (or for root) because they themselves develop for portability.
Comment 1 SpanKY gentoo-dev 2008-11-03 06:40:14 UTC
the only fix here is to have portage remove POSIXLY_CORRECT from the environment.  portage and ebuilds use bash, not POSIX.
Comment 2 Zac Medico gentoo-dev 2008-11-03 07:01:08 UTC
Created attachment 170597 [details, diff]
filter POSIXLY_CORRECT, applies to portage-2.1.4.5

If this patch is saved as /tmp/posixly_correct.patch, then it can be applied as follows:

  cd /usr/lib/portage
  patch -p0 < /tmp/posixly_correct.patch
Comment 3 Zac Medico gentoo-dev 2008-11-03 07:03:43 UTC
Created attachment 170598 [details, diff]
filter POSIXLY_CORRECT, applies to portage-2.2_rc13

This is fixed in svn r11804.
Comment 4 Arthur Hagen 2008-11-03 13:15:51 UTC
(In reply to comment #1)
> the only fix here is to have portage remove POSIXLY_CORRECT from the
> environment.

Removing POSIXLY_CORRECT would cause any programs that portage calls to be without POSIXLY_CORRECT, which is not posixly correct.

> portage and ebuilds use bash, not POSIX.

Non sequitur.  Bash *does* support posix, so why not make the scripts compatible with both posix and non-posix behaviour?  It's not like anything gets lost.

Also worth noting is that it looks like the ebuild.sh script started out posix compliant, with most of the multi-word functions using underscores, not hyphens.

In fact, the only thing that breaks it is three function names in ebuild.sh and isolated-functions.sh:

debug-print
debug-print-function
debug-print-section

To me, it looks like whoever added the debug- functions simply didn't know about the reason for using underscores, which is not a crime and easily correctable.

A simple search/replace on these three, changing hyphens to underscores not only makes the scripts work with POSIXLY_COMPLIANT, but also makes the function names follow the same standard as the rest of the scripts.

Removing POSIXLY_COMPLIANT is, again, NOT the solution, unless care is taken to reinsert it before calling any external executable.  There's a REASON why there is a POSIXLY_COMPLIANT variable, and simply unsetting it is as wrong as, say, changing TZ.

This is, IMO, a much better fix:

perl -pi.bak -e 's/debug-print-function/debug_print_function/g' ebuild.sh isolated-functions.sh
perl -pi.bak -e 's/debug-print-section/debug_print_section/g' ebuild.sh isolated-functions.sh
perl -pi.bak -e 's/debug-print/debug_print/g' ebuild.sh isolated-functions.sh
Comment 5 Arthur Hagen 2008-11-03 13:17:22 UTC
(In reply to comment #4)

Err.  POSIXLY_CORRECT, not POSIXLY_COMPLIANT.  :-)
Comment 6 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2008-11-03 17:05:19 UTC
(In reply to comment #4)

debug-print etc. cannot be removed because they are also used in many eclasses.
Comment 7 SpanKY gentoo-dev 2008-11-03 17:37:42 UTC
this discussion is not going to happen here.  the tree supports bash, end of story.  instead of attempting to rehash things, do your homework and read the mailing lists.

filtering POSIXLY_CORRECT is the correct solution.  if your worried about correct behavior, stop setting it in the first place.
Comment 8 Arthur Hagen 2008-11-03 18:14:16 UTC
(In reply to comment #6)
> (In reply to comment #4)
> 
> debug-print etc. cannot be removed because they are also used in many eclasses.

No-one has advocated removing them.  Re-read, please.
Comment 9 Arthur Hagen 2008-11-03 18:26:23 UTC
(In reply to comment #7)
> this discussion is not going to happen here.  the tree supports bash, end of
> story.

You're talking like you believe that POSIXLY_CORRECT and bash are two different things.  They aren't.  POSIXLY_CORRECT is an environment setting that changes some of the behaviors of many programs *including* bash.  It's still bash, and just as valid bash code.

> instead of attempting to rehash things, do your homework and read the
> mailing lists.

Um, from what I can recall, the only time I have seen POSIXLY_CORRECT discussed, vapier was in favor of supporting it when doing so didn't break anything.
If you allude to something else, please provide pointers, not vagueness.

> filtering POSIXLY_CORRECT is the correct solution.

Um, WHY?  When making the scripts work with POSIXLY_CORRECT both set and unset is so easy?

> if your worried about
> correct behavior, stop setting it in the first place.

If it's incorrect, why does Gentoo have the --posix option in bash in the first place?
Don't turn this into a war for/against POSIX.  POSIX is there, whether you like it or not.  And I see no convincing argument for NOT supporting it when doing so is simple, and what runs with POSIXLY_CORRECT set will also run without it.
Comment 10 SpanKY gentoo-dev 2008-11-03 18:48:48 UTC
if it didnt break anything, then you wouldnt have file a bug now would you ?

your point about bash having "--posix" is just stupid.  what we choose to support in the ebuild tree has nothing to do with what bash itself supports.  bash can be used in a lot more places than the tree.

as for bash vs POSIX, there is no war because it's already been decided.  stop confusing things.  all code that is in the package manager uses the bash standard, not POSIX.  what goes on outside of the package manager is your business, but what goes on inside of the package manger is not.
Comment 11 SpanKY gentoo-dev 2008-11-03 19:46:42 UTC
sorry, they may have been a bit harsh.  but hopefully you got the point that the behavior external is completely up to you, but we cant have the user arbitrarily changing the behavior of programs while attempting to build packages.  having POSIXLY_CORRECT can easily break programs that scripts run inside the tree, while otherwise having no benefit whatsoever.
Comment 12 Zac Medico gentoo-dev 2008-11-11 22:21:25 UTC
This is fixed in 2.2_rc14.