ebuilds should be able to simply look like: IUSE_VIDEO_CARDS="ati nvidia" IUSE="sdl" and then portage would expand the values in IUSE_VIDEO_CARDS automatically and stick it into IUSE instead, we have to expand IUSE_VIDEO_CARDS ourselves and stick it into IUSE: IUSE_VIDEO_CARDS="video_cards_ati video_cards_nvidia" IUSE="sdl ${IUSE_VIDEO_CARDS}"
bugs--
This was gone over on the mailing lists. If you want support for IUSE_VIDEO_CARDS, the _entire_ cache will need to be invalidated whenever USE_EXPAND is modified. As well as that, a list of the USE_EXPAND that the cache was built with will need to be tagged into each cache file causing a large increase in size. IOW, it's not possible while a cache is needed.
CANTFIX isnt really acceptable ... just because portage's current design is unable to cope with this doesnt make the current method correct at some point this will have to be addressed and portage will have to be less stupid about it
so, someday when current design aspects suck less to handle this, we'll re-open
(In reply to comment #2) > This was gone over on the mailing lists. If you want support for > IUSE_VIDEO_CARDS, the _entire_ cache will need to be invalidated whenever > USE_EXPAND is modified. As well as that, a list of the USE_EXPAND that the > cache was built with will need to be tagged into each cache file causing a > large increase in size. Well, the cache would really only become invalid when a change in USE_EXPAND triggers a change in IUSE, and in the given example that would only happen when VIDEO_CARDS is added or removed from USE_EXPAND. However, such "spontaneous" changes in IUSE are actually irrelevant because IUSE really depends on conditionals that exist in the ebuild/eclasses and those conditionals remain constant when USE_EXPAND changes. In our example, if VIDEO_CARDS were removed from USE_EXPAND, portage would no longer perform automatic expansion of that variable for the user, but conceivably, the user could manually expand the variable to achieve the same effect as if it had never been removed.
(In reply to comment #2) > IOW, it's not possible while a cache is needed. After chatting with ferringb I'm now aware that some ebuilds may use the list of user's USE flags to generate new conditionals on the fly, such that the number of conditionals is not static and can shrink or grow (effectively causing IUSE to change). So, apparently we need some type of syntax to express these "use groups" (or whatever they are) symbolically such that the IUSE cache remains valid. For example, an ebuild that uses all possible VIDEO_CARDS would express that with IUSE="@video_cards", or if it only uses 2 video cards then it could express it as IUSE="@video_cards(ati,nvidia)".
Changing the literal value of IUSE for USE_EXPAND variables doesn't make much sense. At least, it makes just as much sense as changing IUSE for USE variables. There are many cases where certain USE flags are ignored for a package unless a certain other USE flag is enabled as well. Is there any real difference between that and this "new" USE_EXPAND case?
Created attachment 87667 [details, diff] Automatically add IUSE_${envvar} to IUSE I think this is essentially what Mike is asking for in this bug. However, there is one important thing to note which is that additions to USE_EXPAND _must_ be committed before any ebuilds that make use of the new var are committed.
(In reply to comment #8) > additions to USE_EXPAND _must_ be committed before any ebuilds that make use of the new var are committed. > USE_EXPAND can be a profile dependent incremental, so the generated metadata would be profile specific, which is not allowed. Perhaps we could simply grab all IUSE_* vars from the env?
(In reply to comment #9) > (In reply to comment #8) > > additions to USE_EXPAND _must_ be committed before any ebuilds that make use of the new var are committed. > > USE_EXPAND can be a profile dependent incremental, so the generated metadata > would be profile specific, which is not allowed. Perhaps we could simply grab > all IUSE_* vars from the env? While it is theoretically possible for profiles to add to USE_EXPAND (and to reset it) none of the profiles should be doing this. USE_EXPAND only moved to base/make.defaults from make.globals so that vars could be added without requiring a newer version of portage. Having said that, grabbing all IUSE_* from the environment would definitely get around the guaranteed-to-be-ignored caching rule/issue. I've not sure on the best way of doing that in bash (MY_IUSE_VARS="$(set | grep '^IUSE_')"?) so if someone could fix that (and hopefully get rid of awk's tolower() usage) we might have a sane fix.
Created attachment 87706 [details, diff] Expands and adds all IUSE_* variables to IUSE IUSE_* is found with $(set | grep -E '^IUSE_' | sed 's/=.*//'). Is there a better way to do this? The previous `awk '{print tolower($0)}'` has been replaced with `tr '[A-Z]' '[a-z]'`.
If this is to be implemented, now is the best time. There's no support at all in 2.0.x and changing it after 2.1.x goes stable will be somewhat difficult. Ebuild authors would need to be doing something like: IUSE_VIDEO_CARDS="ati nvidia" OLD_IUSE_VIDEO_CARDS="video_cards_ati video_cards_nvidia" IUSE="${OLD_IUSE_VIDEO_CARDS} sdl" which is worse than what needs to be done now. That said, IUSE is currently unique_array()'ed so the above wouldn't cause any display issues. Mike, you're the resident bash expert. Thoughts on the patch? Everyone, doing it this way would mean that all of IUSE_* would be reserved for USE_EXPAND.. Anyone against that?
(In reply to comment #12) > Mike, you're the resident bash expert. Thoughts on the patch? Jason, Just to let you know he is in China for a few weeks. Probably without any form of internet till he returns.
Created attachment 87827 [details, diff] expand IUSE_* vars only when necessary This patch does effectively the same thing bug it allows backward compatibility with existing old-style IUSE_* variables. I think we should go ahead and do this then announce it on the gentoo-dev mailing list.
I was just thinking, we can avoid clutter in the vdb IUSE entry if you uniqify them like this: IUSE="$(for x in ${IUSE}; do echo ${x}; done | LANG=C sort -u)" And, we should grep the tree for strange uses of IUSE_* variables before we deploy this...
Thinking a little more, this feature kind of useless as long as 2.0 is in use and doesn't support it. We can distribute the metadata via rsync but that won't help cvs users. Maybe it's better just to implement "use groups" or something similar.
Your patch looks fine to me - glad the set | grep is gone. :) Not sure what you mean about rsync/cvs/uselessness/use groups though.. Can you expand on your thoughts a little please?
(In reply to comment #17) > Not sure what you mean about rsync/cvs/uselessness/use groups though.. Well, I suppose it doesn't matter too much if users running old versions of portage don't see the new IUSE_* flags in metadata that they generate locally (though they will be able to see them in metadata generated for rsync distribution, as soon as I update osprey). So, I suppose it's useful for ebuild maintainers right now if they want to drop the old-style manual expansions. However, this same functionality could be represented by a new "use groups" syntax for IUSE. A possible alternative interim solution to the proposed patch would be to provide an eclass function (in eutils for example) that allows ebuilds to perform semi-automatic expansion. An eclass implementation would could work with all existing versions of portage right away.
Created attachment 87933 [details, diff] add an iuse_expand() function to eutils.eclass An ebuild that wishes to automatically expand IUSE_* variables into IUSE need only inherit eutils, define the IUSE_* variables, then call the iuse_expand() function. With this approach, there is no need to detect already expanded USE flags or make the list unique because ebuilds written to use this function will define the variables such that it is unnecessary. The class function has many considerable advantages over direct integration into portage: 1) It works with all existing versions of portage immediately (including third party package managers such as pkgcore and paludis). 2) There is no global special case for IUSE_* variables, so there is no need to grep the tree for potential strange uses of IUSE_* varables. Only the maintainers of ebuilds that use the iuse_expand() function need to be concerned about the IUSE_* variables. 3) Similar functionality will likely be provided in the future via "use groups" or something similar.
Yep, I'm in favour of the eutils extension.
the problem with that is we get global scope calls to `tr` in the ebuilds now ... wont that cause a QA warning ?
You only get those QA warnings for stuff in QA_INTERCEPTORS and luckily "tr" isn't in there now.
I found a bash based version at http://www.codecomments.com/archive287-2005-2-392096.html. Apparently, it would suffice for short strings but some testing would be probably be warranted.
On my system, for 5000 runs echoing to tr and assigning the result takes about 14.6s. With the function listed in the previous URL, I get the following results. 1 char 4.7s 5 chars 6.6s 10 chars 9.5s 15 chars 12.2s Looking at the function, it could be optimized a fair bit for our specific case though.
*** Bug 156565 has been marked as a duplicate of this bug. ***
So what's the status on this?
*** Bug 167252 has been marked as a duplicate of this bug. ***
*** Bug 170748 has been marked as a duplicate of this bug. ***
*** Bug 172598 has been marked as a duplicate of this bug. ***
*** Bug 172601 has been marked as a duplicate of this bug. ***
Anyone going for this?
*** Bug 136953 has been marked as a duplicate of this bug. ***
*** Bug 70648 has been marked as a duplicate of this bug. ***
Does anyone want this anymore? I think most ebuilds are using 'for' loop-based solution nowadays. The idea sounds certainly doable but it requires fresh discussion on gentoo-dev@. If you want it, please start it. If you don't, we'll just WONTFIX it.
No progress since 7 years. Closing.