Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 148702 - when LINGUAS is unset, portage does not imply all languages by expanding all linguas into USE
Summary: when LINGUAS is unset, portage does not imply all languages by expanding all ...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Configuration (show other bugs)
Hardware: All Other
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 181949
  Show dependency tree
 
Reported: 2006-09-22 14:37 UTC by Kaiting Chen
Modified: 2007-06-15 03:24 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
for unset USE_EXPAND variables, enable all corresponding flags from IUSE (iuse.patch,1.45 KB, patch)
2007-06-09 01:05 UTC, Zac Medico
Details | Diff
iuse.patch (iuse.patch,1.41 KB, patch)
2007-06-09 10:46 UTC, Harald van Dijk (RETIRED)
Details | Diff
make * a wildcard token which triggers expansion of flags from IUSE (wildcard_token.patch,3.29 KB, patch)
2007-06-09 23:25 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kaiting Chen 2006-09-22 14:37:37 UTC
Sorry, this is kind of long. My point is at the very bottom.

When LINGUAS is unset in make.conf, packages that use Gettext assume that that means that the user wants to support all languages. However, in this case, ebuilds like man-pages treat an unset LINGUAS as meaning that the user wants no languages to be supported. This is bad because it is possibly confusing.

For example, I want to build a server that supports all the languagues that I can. I do not set a LINGUAS in my make.conf. Coreutils installs all the .mo files that it supports into /usr/share/locale. Then, I figure that I want my users to be able to use man pages, so I emerge man-pages. If you look at the man-pages ebuild, it specifies man-pages in other languages as dependencies depending on what LINGUAS are set (through the USE_EXPAND flags). However, because I did not specify a LINGUAS, no man-pages in other languages are pulled in as dependencies.

What should happen is that an unset LINGUAS should be USE_EXPAND'ed to all linguas_?? style USE flags.
Comment 1 Zac Medico gentoo-dev 2006-09-22 14:43:14 UTC
You should leave LINGUAS unset if you want all languages.  It's broken in 2.1.1 though (fixed in 2.1.2_pre1 and will also be fixed in 2.1.1-r1).

*** This bug has been marked as a duplicate of 147428 ***
Comment 2 Kaiting Chen 2006-09-22 15:53:54 UTC
It's not a dupe of 147428. The problem is that if I leave LINGUAS unset, it doesn't set any of the USE_EXPAND LINGUAS use flags.

If I leave LINGUAS unset, then man-pages should depend on man-pages-pl, man-pages-it, and friends, but it doesn't.

Leaving LINGUAS unset should set linguas_pl, linguas_it, and friends. (Because you're saying that you want to support every language).
Comment 3 SpanKY gentoo-dev 2006-09-22 16:01:45 UTC
to be honest, i prefer this behavior
Comment 4 Zac Medico gentoo-dev 2006-09-22 16:13:55 UTC
If we do something like this, it shouldn't be a special just for LINGUAS.  If not all USE_EXPAND variables, we could a special class of USE_EXPAND variables that behave this way.  We'd have to pull the linguas_* flags from a package's IUSE.
Comment 5 Kaiting Chen 2006-09-22 16:14:43 UTC
I guess your right. This behavior is probably the best.
But there should be a way to set all of the linguas_ USE flags.

For example,
LINGUAS="*"

or if it's possible, even something like
LINGUAS="* -ro"
to have everything except for one language.

Maybe another variable is needed.
Comment 6 SpanKY gentoo-dev 2006-09-22 19:00:56 UTC
i think your * idea is acceptable ... but i'm not the one implementing things :)
Comment 7 Simon Stelling (RETIRED) gentoo-dev 2006-09-23 00:30:05 UTC
i'm not aware of how far the differences between USE and USE_EXPAND go, but why not use the usual incremental stacking?

I.e. profile sets a useful default (fex LINGUAS="en_GB en_US de_CH" ;)) and those who don't want that default set LINGUAS="de_DE -de_CH" which would result in LINGUAS="en_GB en_US de_DE"

Just an early morning thought...
Comment 8 Harald van Dijk (RETIRED) gentoo-dev 2006-09-25 14:52:26 UTC
> i'm not aware of how far the differences between USE and USE_EXPAND go, but why
> not use the usual incremental stacking?
> 
> I.e. profile sets a useful default (fex LINGUAS="en_GB en_US de_CH" ;)) and
> those who don't want that default set LINGUAS="de_DE -de_CH" which would result
> in LINGUAS="en_GB en_US de_DE"
> 
> Just an early morning thought...

That would make it impossible to get LINGUAS unset. Avoiding a need for this by setting the default LINGUAS to /all/ possible settings is not really feasible either; there exist LINGUAS settings that are not specified in linguas.desc (IIRC, one of the shells (ksh?) supports pig-latin); every single package in the tree would need to be checked.
Comment 9 Marius Mauch (RETIRED) gentoo-dev 2007-06-06 23:16:51 UTC
So, what can we do here? I agree with Mike that LINGUAS="" should not enable all the linguas_* flags, people would go berserk on such a change. Tempted to close this as CANTFIX ...
Comment 10 Harald van Dijk (RETIRED) gentoo-dev 2007-06-07 16:45:54 UTC
> So, what can we do here? I agree with Mike that LINGUAS="" should not enable
> all the linguas_* flags, people would go berserk on such a change.

Agreed, but that's not what this is about. This is about unset LINGUAS implying all the linguas_* flags. Setting LINGUAS="" would still mean no linguas_* flags. And for that, I very much doubt people would go berserk, because that's already the behaviour gettext has, which a whole lot of packages in the tree use.
Comment 11 SpanKY gentoo-dev 2007-06-07 17:58:45 UTC
people dont notice .po and .mo files installed, they do notice additional packages being pulled in

i dont have a problem with the proposed behavior though:
unset LINGUAS -> all linguas
LINGUAS="" -> no linguas

i'm not sure gettext makes this distinction though ... can someone test to see how gettext packages treat `unset LINGUAS` vs `LINGUAS=""`
Comment 12 Harald van Dijk (RETIRED) gentoo-dev 2007-06-07 18:12:45 UTC
From bash's configure script (from gettext's po.m4):

[...]
    LINGUAS="${LINGUAS-%UNSET%}"
[...]
              if test "%UNSET%" != "$LINGUAS"; then
                desiredlanguages="$LINGUAS"
              else
                desiredlanguages="$ALL_LINGUAS"
              fi
[...]

And with LINGUAS unset, bash installs
$ qlist bash | grep '\.mo'
/usr/share/locale/en@quot/LC_MESSAGES/bash.mo
/usr/share/locale/en@boldquot/LC_MESSAGES/bash.mo
/usr/share/locale/ru/LC_MESSAGES/bash.mo

while with LINGUAS set but empty, bash installs
$ qlist bash | grep '\.mo'
(nothing)
Comment 13 SpanKY gentoo-dev 2007-06-08 07:41:42 UTC
then what are we waiting for people !?  get on it !
Comment 14 Zac Medico gentoo-dev 2007-06-09 01:05:51 UTC
Created attachment 121537 [details, diff]
for unset USE_EXPAND variables, enable all corresponding flags from IUSE

This seems to do the job.  It has the potential to surprise some people, but it's easy enough for them to disable it with an empty variable in make.conf.
Comment 15 Harald van Dijk (RETIRED) gentoo-dev 2007-06-09 09:59:02 UTC
There seems to be a problem with the patch:

$ emerge -pv mozilla-firefox

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] www-client/mozilla-firefox-2.0.0.4  USE="bindist filepicker ipv6 mozdevelop -debug -gnome -java -moznopango -restrict-javascript -xforms -xinerama -xprint" LINGUAS="af* ar* be* bg* ca* cs* da* de* el* en_GB* es* es_AR* es_ES* eu* fi* fr* fy* fy_NL* ga* ga_IE* gu* gu_IN* he* hu* it* ja* ka* ko* ku* lt* mk* mn* nb* nb_NO* nl* nn* nn_NO* pa* pa_IN* pl* pt* pt_BR* pt_PT* ro* ru* sk* sl* sv* sv_SE* tr* zh* zh_CN* zh_TW*" 6,683 kB

Total: 1 package (1 reinstall), Size of downloads: 6,683 kB

$ emerge -pv bash mozilla-firefox
These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] app-shells/bash-3.2_p17  USE="nls -afs -bashlogger -vanilla" 0 kB
[ebuild   R   ] www-client/mozilla-firefox-2.0.0.4  USE="bindist filepicker ipv6 mozdevelop -debug -gnome -java -moznopango -restrict-javascript -xforms -xinerama -xprint" LINGUAS="-af -ar -be -bg -ca -cs -da -de -el -en_GB -es -es_AR -es_ES -eu -fi -fr -fy -fy_NL -ga -ga_IE -gu -gu_IN -he -hu -it -ja -ka -ko -ku -lt -mk -mn -nb -nb_NO -nl -nn -nn_NO -pa -pa_IN -pl -pt -pt_BR -pt_PT -ro -ru -sk -sl -sv -sv_SE -tr -zh -zh_CN -zh_TW" 0 kB

Total: 2 packages (2 reinstalls), Size of downloads: 0 kB
Comment 16 Harald van Dijk (RETIRED) gentoo-dev 2007-06-09 10:46:31 UTC
Created attachment 121561 [details, diff]
iuse.patch

Looks like all that's missing is setting has_changed. With this minor modification, it seems to be working right for me.
Comment 17 Marius Mauch (RETIRED) gentoo-dev 2007-06-09 11:44:07 UTC
Actually in my previous post I referred to both LINGUAS="" and unset LINGUAS, most users won't understand the difference there and become very irritated when they see this new behavior, so if you really want to stick with this you better make a huge announcement about it.
Comment 18 Zac Medico gentoo-dev 2007-06-09 11:44:48 UTC
Thanks, this is in svn r6781.
Comment 19 Zac Medico gentoo-dev 2007-06-09 11:48:20 UTC
(In reply to comment #17)
> Actually in my previous post I referred to both LINGUAS="" and unset LINGUAS,
> most users won't understand the difference there and become very irritated when
> they see this new behavior, so if you really want to stick with this you better
> make a huge announcement about it.

Yeah, I imagine some people could get quite disturbed.  I huge announcement will certainly be necessary.  In most cases these variables or their corresponding flags are already set in the profile though, so that will make the new behavior less noticeable.
Comment 20 Zac Medico gentoo-dev 2007-06-09 11:53:01 UTC
If we want to make it slightly more flexible, we could trigger the new behavior via a special token, so you'd set LINGUAS="*" or something like that.
Comment 21 Harald van Dijk (RETIRED) gentoo-dev 2007-06-09 12:09:21 UTC
That way, the gettext behaviour would be lost. If LINGUAS is set, gettext packages don't install all translations.
Comment 22 Zac Medico gentoo-dev 2007-06-09 12:30:28 UTC
Just because it's set in make.conf doesn't mean that it has to be set in the ebuild environment.  We'd just have to modify the patch slightly so that the * token causes the variable to be unset unless there are some flags from IUSE.
Comment 23 Harald van Dijk (RETIRED) gentoo-dev 2007-06-09 12:45:59 UTC
Fair enough. If you're going that route, it might also be nice to allow LINGUAS="en *", which would unset LINGUAS if no linguas_* flags are in IUSE, and would move any linguas_en option to the front of the list.
Comment 24 Zac Medico gentoo-dev 2007-06-09 21:53:45 UTC
(In reply to comment #23)
> Fair enough. If you're going that route, it might also be nice to allow
> LINGUAS="en *", which would unset LINGUAS if no linguas_* flags are in IUSE,
> and would move any linguas_en option to the front of the list.

Similarly, one should be able to set linguas_* in package.use in order to enable all linguas flags for a specific package. I'm adding Diego to CC since he's expressed interest for this type of thing in the past (wrt ALSA_CARDS).
Comment 25 Zac Medico gentoo-dev 2007-06-09 23:25:18 UTC
Created attachment 121633 [details, diff]
make * a wildcard token which triggers expansion of flags from IUSE

With this patch, you can do things like LINGUAS="en *" or put linguas_* in package.use and it should behave intuitively.
Comment 26 Harald van Dijk (RETIRED) gentoo-dev 2007-06-10 08:52:21 UTC
(In reply to comment #25)
> With this patch, you can do things like LINGUAS="en *" or put linguas_* in
> package.use and it should behave intuitively.

The resulting LINGUAS environment variable does not appear to be in any particular order:

$ cat /etc/portage/overlay/test/test/test-1.0.ebuild
KEYWORDS=~$ARCH
IUSE="linguas_af linguas_ar linguas_be linguas_bg linguas_ca linguas_cs
linguas_da linguas_de linguas_el linguas_en_GB linguas_es_AR linguas_es_ES
linguas_es linguas_eu linguas_fi linguas_fr linguas_fy_NL linguas_fy
linguas_ga_IE linguas_ga linguas_gu_IN linguas_gu linguas_he linguas_hu
linguas_it linguas_ja linguas_ka linguas_ko linguas_ku linguas_lt linguas_mk
linguas_mn linguas_nb_NO linguas_nb linguas_nl linguas_nn_NO linguas_nn
linguas_pa_IN linguas_pa linguas_pl linguas_pt_BR linguas_pt_PT linguas_pt
linguas_ro linguas_ru linguas_sk linguas_sl linguas_sv_SE linguas_sv linguas_tr
linguas_zh_CN linguas_zh linguas_zh_TW"
pkg_setup() {
        echo "LINGUAS = ${LINGUAS}"
        die
}
$ grep ^LINGUAS /etc/make.conf
LINGUAS="en *"
$ sudo emerge test/test
Calculating dependencies... done!
>>> Verifying ebuild Manifests...

>>> Emerging (1 of 1) test/test-1.0 to /
 * checking ebuild checksums ;-) ...                                                                   [ ok ]
 * checking auxfile checksums ;-) ...                                                                  [ ok ]
 * checking miscfile checksums ;-) ...                                                                 [ ok ]
LINGUAS = fy de pa_IN da be fr bg pt_PT pt_BR fi zh_TW ja he fy_NL mn ko sv mk sv_SE ka hu zh_CN nb_NO sk ku gu_IN ca it el gu sl zh pl af eu es ar pt ga cs nb tr ga_IE ru nl nn nn_NO es_ES ro en_GB lt pa es_AR

!!! ERROR: test/test-1.0 failed.
Call stack:
  ebuild.sh, line 1642:   Called dyn_setup
  ebuild.sh, line 704:   Called qa_call 'pkg_setup'
  ebuild.sh, line 44:   Called pkg_setup
  test-1.0.ebuild, line 13:   Called die

!!! (no error message)
!!! If you need support, post the topmost build error, and the call stack if relevant.
!!! A complete build log is located at '/var/tmp/portage/test/test-1.0/temp/build.log'.

!!! This ebuild is from an overlay: '/etc/portage/overlay'
Comment 27 Harald van Dijk (RETIRED) gentoo-dev 2007-06-10 08:59:54 UTC
(In reply to comment #26)
> (In reply to comment #25)
> > With this patch, you can do things like LINGUAS="en *" or put linguas_* in
> > package.use and it should behave intuitively.
> 
> The resulting LINGUAS environment variable does not appear to be in any
> particular order:

Of course, I missed the "en" at the start only because linguas_en was not present in IUSE. To keep the ebuild's order of the rest, though, it might be nice to change iuse = set([...]) to just iuse = [...].
Comment 28 Zac Medico gentoo-dev 2007-06-15 00:10:07 UTC
(In reply to comment #27)
> To keep the ebuild's order of the rest, though, it might be
> nice to change iuse = set([...]) to just iuse = [...].

Done in svn r6787.
Comment 29 Zac Medico gentoo-dev 2007-06-15 03:24:17 UTC
This has been released in 2.1.3_rc1.