Bugzilla Bug 70648

In esearch-0.7.ebuild you can see this code:

    if use linguas_it; then
        insinto /usr/share/man/it/man1
        doins it/{esearch,eupdatedb,esync}.1
    fi

linguas_it is not in IUSE and not in use.desc or use.local.desc.  On the surface,
several QA warnings would be issued.  use flag missing from IUSE, invalid use flag and so on.

I wonder if portage should instead not warn about use flags derived from USE_EXPAND.

There are other known issues with LINGUAS, too. The current GWN informs about
the glibc userlocales use-flag/feature.
http://www.gentoo.org/news/en/gwn/20041108-newsletter.xml#doc_chap7

I wonder, why we have two approaches implemented. This does not make sense. I
like the glibc approach much better and would love it, if someone would write a
GLEP regarding a unified i18n/l18n handling for all packages.

*** Bug 101998 has been marked as a duplicate of this bug. ***

Created an attachment (id=69088) [details]
iuse.patch

Just a quick way to not warn for this, for the portage 2.1 snapshot. It could
probably use some cleanups, but it's functional. Please comment on anything
that needs changing.

> I wonder, why we have two approaches implemented. This does not make sense.
> I like the glibc approach much better and would love it, if someone would
> write a GLEP regarding a unified i18n/l18n handling for all packages.

userlocales helps with internationalisation, LINGUAS helps with localisation.
And even if these are unified (as someone who builds locales I don't need
translations for, I would hate that, by the way), the LINGUAS approach is the
one used by the GNU folks (documented in the info page for gettext) without
help from ebuilds, so a lot of work would be needed to use anything else.

Actually, a correction... it's not that strict a difference between i18n and
l10n, and locale info does cover localisation as well, of course. What I said
wasn't really right, but I hope what I meant is clear enough.

*** Bug 62001 has been marked as a duplicate of this bug. ***

*** Bug 108239 has been marked as a duplicate of this bug. ***

Every effort to try and find an acceptable way to get this fixed has met with   
upstream finding it unacceptable. Please reopen when upstream has sorted out a 
solution that they are happy with. 

*** Bug 97746 has been marked as a duplicate of this bug. ***

*** Bug 115287 has been marked as a duplicate of this bug. ***

Uhm, sorry but I don't think that UPSTREAM is an acceptable resolution for an
existing bug/inconsistency in portage - which upstream is upstream for portage
issues? There's either a trivial fix that makes portage shut up and stop
emitting unhelpful warnings or a huge PITA implementing something else. 

What's so difficult about fixing this?
i18n/l18n/VIDEO_CARDS/ALSA_CARDS/userland_*/kernel_*/elibc_*/... implementation
is really a separate issue and debate, the QA check this bug is about should
check for and moan about USE flags and not USE_EXPANDed stuff, that's it.

The purpose of the IUSE check is to ensure that flags are not affecting the 
build process which the user has not been informed about. Doesn't USE_EXPAND 
stuff fall under that category? 

(In reply to comment #11)
> The purpose of the IUSE check is to ensure that flags are not affecting the 
> build process which the user has not been informed about. Doesn't USE_EXPAND 
> stuff fall under that category? 

Well, no? That's why those two are separate things, I guess? There's been a
technical reason for implementing USE_EXPAND (turning all these variables into
"proper" USE flags is a big PITA indeed), so if portage allows and provides for
USE_EXPAND, it should not then complain about it at the same time, it's a bit
schizophrenic behaviour... :-)

I really see this bug as an inconsist portage behaviour; whether the current
implementation is an ideal solution or not should be left for another debate
(-dev ml I suppose). While I'd like to handle e.g. VIDEO_CARDS in the same way
as userlocales in glibc, this solution is not exactly beneficial for LINGUAS,
where it would cause a big maintenance overhead with no real benefits, since
this is how things are already done upstream and there's no need to reinvent the
wheel.

The fix definitely is not to emit these incorrect warnings. The current
behaviour is really weird.

Yep, the fix is either to remove USE_EXPAND support or to get the information 
to the user (and to portage). The latter means IUSE. 

(In reply to comment #14)
> Yep, the fix is either to remove USE_EXPAND support or to get the information 
> to the user (and to portage). The latter means IUSE. 

Jason, the whole point of USE_EXPAND is not to have extra IUSE entries, but to
silently inject a whole set of use flags. What you're saying here is ridiculous.

So.. how should users be informed of what can be set via USE_EXPAND?  
Until that issue is solved, the QA notices are valid. 

(In reply to comment #16)
> So.. how should users be informed of what can be set via USE_EXPAND?  
> Until that issue is solved, the QA notices are valid. 

I don't really how these QA notices are valid - they produce warnings about
variables that users have specifically and intentionally set for exactly the
same ebuilds that now moan about them. Which information are you getting to the
users? A cryptic message that makes them file erroneous bugs?

boo hoo, why dont we just fix the USE_EXPAND issue

it's actually quite trivial ... i'll fix it tonight

(In reply to comment #17) 
> I don't really how these QA notices are valid - they produce warnings about 
> variables that users have specifically and intentionally set... 
 
Wrong. They produce warnings about the ebuild references to USE flags that 
(may) have been produced via USE_EXPAND but don't exist in IUSE. If the user 
has not set anything and doesn't know anything about it, the QA messages still 
occur. 
 
Look, it's simple: 
 
# emerge kde-i18n 
Calculating dependencies ...done! 
>>> emerge (1 of 1) kde-base/kde-i18n-3.5.0 to / 
... 
 
 * You must set the LINGUAS environment variable to a list of valid 
 * language codes, one for each language you would like to install. 
 * e.g.: LINGUAS="sv de pt" 
 * 
 * The available language codes are: 
af ar az bg bn br bs ca cs cy da de el en_GB eo es et 
eu fa fi fr fy ga gl he hi hr hu is it ja km ko lt lv 
mk mn ms nb nds nl nn pa pl pt pt_BR ro ru rw se sk sl 
sr sr@Latn ss sv ta tg tr uk uz zh_CN zh_TW 
 
This is not an acceptable way to teach users about USE_EXPAND variables. 

(In reply to comment #19)
> This is not an acceptable way to teach users about USE_EXPAND variables. 

I agree with this part, which is why I've been pushing IUSE_${USE_EXPAND_VAR}
for a while.

Except that IUSE_${USE_EXPAND_VAR} isn't viable due to the way portage caching 
works. Even if keys could be modified on the fly (2.0.53 supports this), the 
entire cache would be invalidated every time USE_EXPAND changed. 

(In reply to comment #19)
> (In reply to comment #17) 

> Wrong. They produce warnings about the ebuild references to USE flags that 
> (may) have been produced via USE_EXPAND but don't exist in IUSE. If the user 
> has not set anything and doesn't know anything about it, the QA messages still 
> occur. 

Round and round we go... That's a *separate* issue. USE_EXPAND is a feature in
portage, while it's there, it should not produce bogus warnings. (The other way
round: If you require putting all that bloat into IUSE, USE_EXPAND can be punted
from portage as well.)

> # emerge kde-i18n 
> Calculating dependencies ...done! 
> >>> emerge (1 of 1) kde-base/kde-i18n-3.5.0 to / 
> ... 
>  
>  * You must set the LINGUAS environment variable to a list of valid 
>  * language codes, one for each language you would like to install. 
>  * e.g.: LINGUAS="sv de pt" 
> ...
> This is not an acceptable way to teach users about USE_EXPAND variables. 

LINGUAS is documented in
http://www.gentoo.org/doc/en/guide-localization.xml#doc_chap3, ALSA_CARDS is
documented in http://www.gentoo.org/doc/en/alsa-guide.xml; VIDEO_CARDS is
documented e.g. in http://www.gentoo.org/doc/en/ati-faq.xml. You can document
it/refer to documentation sources in man make.conf or whereever else. 

Spitting out cryptic QA notices does not document anything, just makes people
file invalid bugs. If you want to implement this feature in a better way, that's
all fine; until then, make portage shut up because USE_EXPAND is a valid portage
 feature and so should not make portage itself complain when it's used.

Discussion on changing the way it works is really outside the scope of this bug
(and solution is in no way as trivial as a fix for this particular issue). 

USE_EXPAND and IUSE 
http://thread.gmane.org/gmane.linux.gentoo.devel/29043 
 
Dirt: To shove under the rug or not shove under the rug? (aka another round of 
USE_EXPAND) 
http://thread.gmane.org/gmane.linux.gentoo.devel/31843 
 
Those two threads pretty much cover all the issues. Mike, make sure you 
address them all with your quick and easy fix. ;) 

(In reply to comment #21)
> Except that IUSE_${USE_EXPAND_VAR} isn't viable due to the way portage caching 
> works. Even if keys could be modified on the fly (2.0.53 supports this), the 
> entire cache would be invalidated every time USE_EXPAND changed. 

OK, then what's an equivalent solution that doesn't clutter the user's -vp
output with video_cards_[A-Z]?

What if they were included in IUSE, but portage parsed it for things prefixed
with valid USE_EXPAND and presented them separately?

I've got a patch that does that already. 
 
[ebuild 

I've got a patch that does that already. 
 
[ebuild  N    ] kde-base/kde-i18n-3.4.1  +arts -debug +kdeenablefinal  
-xinerama  LINGUAS="-ar -bg -bn -br -bs -ca -cs -cy -da -de -el -en_GB -eo -es  
-et -eu -fi -fr -fy -ga -he -hi -hsb -hu -is -it +ja -lt -mk -nb -nds -nl -nn  
-pa -pl -pt -pt_BR -ro -ru -se -sk -sl -sr -sr@Latn -sv -ta -tg -tr -uk  
-zh_CN" 0 kB 
 

(In reply to comment #25)
> I've got a patch that does that already. 

Well that's really all I care about, and I would be happy to add things into
IUSE with that patch in portage.

this bug is about shutting the bogus QA warning in use() when utilize
USE_EXPANDed stuff

this is trivial to do

as to advertising their existence, that is no the point of this bug ... we have
other bugs covering that

so we might as well shut up the bogus warnings and then worry about how to make
some of the IUSE_EXPANDed more visible

Whatever. I don't understand these weird policies of "who cares how much it 
sucks as long as users don't know about it", but it's fixed in trunk now so go 
ahead and hide the evidence of breakage for 2.0.54 if you like. 

To me it's pretty braindead to have something like LINGUAS="fr de jp" and then
make it visible as if it were normal use flags. In this case you could indeed
drop it and use regular use flags. I think ebuilds with more than a dozen use
flags are confusing to users and I definitely don't want to see such an output
as you propose in comment #25, Jason.

Don't you see it already with php and others? Making the output more readable 
is a separate issue again. However, it's impossible to make the output more 
readable without have the information to begin with. 

(In reply to comment #30)
> Don't you see it already with php and others?

There's a big difference between displaying lots of *relevant* USE flags for one
particular ebuild, that are actually useful for selecting required
functionality, and between displaying annoying huge list of LINGUAS for every
single ebuild which happens to support them.

LINGUAS is not per-package, but per-system variable for vast majority of people,
and it won't make them jump ecstastically, screaming "cool, this package does
not support German and Polish, but I can instead use Japanese and Hindi, yay!".
It's a thing that people don't care about beyond initial setup when installing
Gentoo and remains untouched then.

LINGUAS should be considered separately from all other things mentioned here, as
should userland/kernel USE_EXPANDed variables be considered separately (and
completely hidden from users in this particular case, IMHO).

VIDEO_CARDS/ALSA_CARDS/INPUT_DEVICES/... is yet another thing, there I'd
actually agree that displaying these variables with emerge -pv would be
beneficial to user. 

Anyway, I most certainly agree with vapier here; there are *separate* bugs for
the above mentioned issues, and mailing lists are perhaps preferred way to
discuss such things.

(In reply to comment #30)
> Don't you see it already with php and others? Making the output more readable 
> is a separate issue again.

The php ebuilds are the best examples with way too much use flags and suck big
time. 

> However, it's impossible to make the output more 
> readable without have the information to begin with. 

Um, I don't want USE_EXPANDed flags to be that visible at all. It's simply not
necessary imho.

(In reply to comment #32)
> (In reply to comment #30)
> > Don't you see it already with php and others? Making the output more readable 
> > is a separate issue again.
> 
> The php ebuilds are the best examples with way too much use flags and suck big
> time. 
> 

PHP is a complex package, and that complexity is required to keep all options
open.  Feel free to bug PHP's upstream about reducing the number of configure
options, since that is basically the equivelant of what you are asking.

>> However, it's impossible to make the output more 
>> readable without have the information to begin with. 
>
>Um, I don't want USE_EXPANDed flags to be that visible at all. It's simply not
>necessary imho.

Without portage printing out what flags are available how is a user supposed to
garner what flags will affect that package?

Is there policy stating that USE_EXPAND flags should have documentation?

Really the only thing I dislike about this whole deal is LINGUAS, but it
unfortunately is being implemented via USE_EXPAND, and so I guess it will
probably end up suffering for a bit because NO ONE CAME UP WITH A BETTER SOLUTION :)

(In reply to comment #33)
> PHP is a complex package, and that complexity is required to keep all options
> open.

I don't think it's always necessary to keep _every_ possible option open,
especially if it doesn't force none or only a single, light dependency. The
worst is "gd gd-external". All packages should use shared libs without exception.

> Without portage printing out what flags are available how is a user supposed
> to garner what flags will affect that package?

Users should use the env variables and the number of USE_EXPANDed flags should
be kept to a minimum anyways.

 
> Is there policy stating that USE_EXPAND flags should have documentation?

The point is that we need another - hopefully less fritless - discussion about
the use flag system, including  use flag groups etc..

> Really the only thing I dislike about this whole deal is LINGUAS, but it
> unfortunately is being implemented via USE_EXPAND, and so I guess it will
> probably end up suffering for a bit because NO ONE CAME UP WITH A BETTER
SOLUTION :)

USE_EXPANDed flags are not needed a lot and especially VIDEO_CARDS and
INPUT_DEVICES are questionable, since xorg is split into single drivers. A simple

|| ( driver_all driver_1 ... driver_N )

dependency should suffice without the need of a use flag, if I did miss not
anything. driver_all would be a meta ebuild as found in kde-base/ as long as we
don't have proper ebuild sets implemented.

(In reply to comment #34)
> USE_EXPANDed flags are not needed a lot and especially VIDEO_CARDS and
> INPUT_DEVICES are questionable, since xorg is split into single drivers. A simple
> 
> || ( driver_all driver_1 ... driver_N )
> 
> dependency should suffice without the need of a use flag, if I did miss not
> anything. driver_all would be a meta ebuild as found in kde-base/ as long as we
> don't have proper ebuild sets implemented.

That blows for usability. People shouldn't even need to know what kind of video
card they have installed. That's half the point of using USE_EXPAND -- you can
either get _all_ drivers or, if you know yours, select it (or them if you have
multiple cards) at emerge time.

I want to be able to say "emerge xorg-x11" and get everything I need, but no
more. I would much rather configure variables for this than add 20 extra
packages to the list of things I need to emerge.

Incidentally, the patch has a USE_EXPAND_HIDDEN that simply indicates that a  
USE_EXPAND variable shouldn't be shown. Users could quite easily update that  
to silence output that they don't care about. 
 
By the way, when was the last time you used the build, bootstrap, readline, 
nls or the userlocales flags? There's a bunch of USE flags that apply to many 
packages that are never changed since initial setup. These would be just as 
nice to be able to hide. 

(In reply to comment #35)
> That blows for usability. People shouldn't even need to know what kind of
> video card they have installed. 

Eh? Users have to know which card(s) they use, otherwise they can't set
VIDEO_CARDS="foo bar", so this point doesn't hold. There's no difference to
plain dependencies.

> That's half the point of using USE_EXPAND -- you can
> either get _all_ drivers or, if you know yours, select it (or them if you have
> multiple cards) at emerge time.

The simple || dependency works exactly the same way.

> I want to be able to say "emerge xorg-x11" and get everything I need, but no
> more. I would much rather configure variables for this than add 20 extra
> packages to the list of things I need to emerge.

You do that once, the updates come for free, without the need of any extra flag
or variable. Moreover we'll have user defined package sets, so this point is
moot as well.

(In reply to comment #37)
> (In reply to comment #35)
> > That blows for usability. People shouldn't even need to know what kind of
> > video card they have installed. 
> 
> Eh? Users have to know which card(s) they use, otherwise they can't set
> VIDEO_CARDS="foo bar", so this point doesn't hold. There's no difference to
> plain dependencies.

And there's already a patch to do so.

> > That's half the point of using USE_EXPAND -- you can
> > either get _all_ drivers or, if you know yours, select it (or them if you have
> > multiple cards) at emerge time.
> 
> The simple || dependency works exactly the same way.

Really? Tell me how I can type emerge xorg-x11 and get all drivers installed if
I specify none, without a huge hack in the ebuild.

> > I want to be able to say "emerge xorg-x11" and get everything I need, but no
> > more. I would much rather configure variables for this than add 20 extra
> > packages to the list of things I need to emerge.
> 
> You do that once, the updates come for free, without the need of any extra flag
> or variable. Moreover we'll have user defined package sets, so this point is
> moot as well.

Set up the variables once, same thing. Except they can apply to multiple
packages (e.g. x11-drm). I don't know what you're talking about with
user-defined package sets.

In addition, the variables are obvious when you type emerge -vp xorg-x11 (with
Jason's patch), whereas finding random package names that happen to fit in an ||
dep is not.

(In reply to comment #38)
> > The simple || dependency works exactly the same way.
> 
> Really? Tell me how I can type emerge xorg-x11 and get all drivers installed if
> I specify none, without a huge hack in the ebuild.

As I outlined above already: You create one drivers_all meta ebuild depending on
all single driver ebuilds and stick it in as the leftmost || dependency.
 
> Set up the variables once, same thing. Except they can apply to multiple
> packages (e.g. x11-drm). 

Installing a package once and having Portage doing the rest for you is even simpler.

> I don't know what you're talking about with
> user-defined package sets.

GLEP 21
 
> In addition, the variables are obvious when you type emerge -vp xorg-x11 (with
> Jason's patch)

Installing something having a list of dependencies with each a lot of use flags
filling the whole screen is not user friendly, but a usability problem, due to
the improperly presented complexity, you can delve into. Moreover using the use
flag system as a general tool to denote every package, leads to more use flags
than packages in the worst case. And users moan already about the sheer number
of use flags here and there. 

> whereas finding random package names that happen to fit in an ||
> dep is not.

When you're able to choose the correct use flag denoting your graphics card,
you'll be able to choose the correct package as well.

*** Bug 127451 has been marked as a duplicate of this bug. ***

*** Bug 132769 has been marked as a duplicate of this bug. ***

*** Bug 134052 has been marked as a duplicate of this bug. ***

*** Bug 125690 has been marked as a duplicate of this bug. ***

*** Bug 144264 has been marked as a duplicate of this bug. ***

*** Bug 143877 has been marked as a duplicate of this bug. ***

*** Bug 142605 has been marked as a duplicate of this bug. ***

*** Bug 167252 has been marked as a duplicate of this bug. ***

*** Bug 194188 has been marked as a duplicate of this bug. ***

*** Bug 193967 has been marked as a duplicate of this bug. ***

*** Bug 194379 has been marked as a duplicate of this bug. ***

Any news on this issue, except a gazillion of duplicate bugs?!

I see 2 possible solutions:

1) Simply put the flags in IUSE to solve the legitimate QA warning.
2) Implement a more convenient way of doing #1 (like bug 133327).

For bug #133327, we can add do an eutils function if we want to be backward
compatible, otherwise we can to some EAPI-1 syntax extension.

*** Bug 195051 has been marked as a duplicate of this bug. ***