Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 104852 - USE flag filtering for UFED
Summary: USE flag filtering for UFED
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Sven Eden
URL:
Whiteboard:
Keywords:
: 367953 (view as bug list)
Depends on:
Blocks:
 
Reported: 2005-09-04 20:07 UTC by Justus Ranvier
Modified: 2013-04-15 08:09 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Justus Ranvier 2005-09-04 20:07:08 UTC
The number of USE flags is large and growing all the time. Tools like ufed are
less useful if you have to sort through hundreds of USE flags, most of which are
flags that only apply to packages that aren't even installed. The following are
suggested categories that USE flags could be seperated into, with ufed allowing
each category to be displayed or hidden as desired by the user:

1. hardware flags (x86, amd64, sparc, mmx)
2. internal portage flags (bootstrap, static)
3. global flags
4. local flags for installed packages only
5. all local flags

Optionally, ufed could be configured to store changes to local flags in
/etc/portage/package.use, while putting changes to global flags in /etc/make.conf

Reproducible: Always
Steps to Reproduce:
1.
2.
3.
Comment 1 Harald van Dijk (RETIRED) gentoo-dev 2005-09-05 05:06:11 UTC
First, some comments:     
     
> 1. hardware flags (x86, amd64, sparc, mmx)     
x86, amd64, sparc, etc. don't get shown by ufed under normal circumstances;     
they're not to be set by the user. For the others, ufed has no way of detecting     
which flags are hardware flags, and such flags may be added at any time,   
meaning ufed would have to be updated constantly.   
   
> 2. internal portage flags (bootstrap, static)     
static is not an internal portage flag; it's neither used by portage or     
bootstrap scripts, nor useless after your base system is installed. (For     
example, busybox can be good to always build with USE=static, and some other     
packages don't build unless a dependency is installed with USE=static.) The     
only flags I know of that would fall in this category are bootstrap and build,     
and they can be and are specifically hidden by ufed 0.40_pre* as a special   
case, since I don't expect future internal flags. (IIRC, other versions hide   
them too, but I haven't checked it right now.)   
   
> 3. global flags   
Agreed, there needs to be a way to filter all local flags.   
   
> 4. local flags for installed packages only   
> 5. all local flags   
I'd rather not separate these, and instead only add a check for "flags for    
installed packages", but provide a way for the user to combine filters so that    
those specific categories can be created.   
   
> Optionally, ufed could be configured to store changes to local   
> flags in /etc/portage/package.use, while putting changes to   
> global flags in /etc/make.conf   
While I do wish to add support for package.use, I have no idea how to do this   
without massively changing the design of ufed. Users may well want local flags   
added to make.conf, and global flags added to package.use, or add local flags   
to package.use only for one package rather than all that support the flag. For   
now, I see this specific idea more as one for the long term.   
   
Extra note: in another utility, I had split local flags by category name (so   
that you could select to see all local flags used by packages in sys-*),   
which proved useful, and I may add to ufed (strictly optionally) as well.   
   
That said, none of this will find its way into ufed 0.40, but filtering will   
probably be one of the first big things I'll try to add in some form after 
that.  
Comment 2 Justus Ranvier 2005-09-05 06:14:55 UTC
I was imagining a set of hotkeys that would toggle the various categories on and
off, or possibly command line parameters to do the same thing.

1. I was sure that I had seen these flags in ufed before; maybe that was an
earlier version...

2. (I meant build instead of static, really) My comment from item 1 applies here
too.

For package.use, ufed could check to see if a flag is already defined there
before making any changes. If it finds a local flag in package.use it can make
the change to that file, otherwise add it to /etc/make.conf.

Unltimately, you can have two user-selectable policies: all flags in make.conf
or globals in make.conf with locals in package.use. Each policy could be
overridden on a package-specific basis. 
Comment 3 Harald van Dijk (RETIRED) gentoo-dev 2005-09-05 06:45:08 UTC
> I was imagining a set of hotkeys that would toggle the various categories on 
and 
> off, or possibly command line parameters to do the same thing. 
Command line parameters are probably easiest to do, perhaps with a default 
specified in a configuration file. 
 
As for package.use: let's say it contains 
app-text/acroread nsplugin 
media-video/realplayer -nsplugin 
What would you want ufed to do? Turn on or turn off the checkbox by default? 
And what would package.use contain after saving? 
 
Or, how about if it contains 
sys-devel/gcc nocxx 
=sys-devel/gcc-3.4* 
where the second line serves to ignore the first line for that specific 
version? Again, what would you want ufed to do, both initially and when saving? 
 
The current interface, a simple checkbox for each flag, isn't powerful enough 
to handle package.use properly. 
Comment 4 Justus Ranvier 2005-09-05 11:20:50 UTC
> As for package.use: let's say it contains 
> app-text/acroread nsplugin 
> media-video/realplayer -nsplugin 
> What would you want ufed to do? Turn on or turn off the checkbox by default? 
> And what would package.use contain after saving? 
>  

It doesn't really make sense for a "local" flag to apply to more than one
package. How does portage define global vs. local flags? If all flags that apply
to more than one package are defined as global, then this problem is easier to
solve.

> Or, how about if it contains 
> sys-devel/gcc nocxx 
> =sys-devel/gcc-3.4* 
> where the second line serves to ignore the first line for that specific 
> version? Again, what would you want ufed to do, both initially and when saving? 
>  
> The current interface, a simple checkbox for each flag, isn't powerful enough 
> to handle package.use properly. 

I have a proposal that works, except for the version-specific example:

The interface will have two modes "global" and "local" (substitute more suitable
names as approprate). In "global" mode, make.conf is edited. in "local" mode
package.use is edited.

"global" mode is the same as the current interface, except the user may select
which flags are displayed: global only (default), global + local flags for
installed packages, or all flags.

"local" mode looks similar, but the list of flags is sorted by package. Users
can configure which flags are displayed and which packages are displayed.
Checkboxes can have three states (+, -, blank). A + or - will create an entry in
package.use, a blank checkbox will remove any reference to that flag on the
associated line in package.use.

The options for package display are: installed packages only (default) or all
packages

The options for flag display are: local flags only (default) or all flags

Empty sets should not displayed (a package should not be displayed unless it has
at least one flag to be edited, based on the current display options).

The "local" mode should have some way to indicate what flags will be active or
inactive, based on the profile and make.conf settings, so that unnecessary
package.use lines can be avoided.

This should make it relatively easy to have ufed put the USE entries where you
want them and deal with the first example you gave. As for version-specific
entries in package.use, the "local" mode could display the entry as if it was a
seperate package, displayed below the generic entry. It's probably sufficient to
 require these lines to be added manually as long as the program can deal with
them once they exist.
Comment 5 Harald van Dijk (RETIRED) gentoo-dev 2005-09-05 12:11:45 UTC
> It doesn't really make sense for a "local" flag to apply to more than one  
> package. How does portage define global vs. local flags? If all flags that  
apply  
> to more than one package are defined as global, then this problem is easier  
to  
> solve.  
  
Indeed it would be, but it's not that simple. Flags are made global if they are  
generic enough to be made global, or if they are used by enough packages (no  
exact number exists). An example of a flag that does not need to be global but  
does apply to multiple packages:  
% grep mzscheme $PORTDIR/profiles/use.local.desc  
app-editors/gvim:mzscheme - Include support for the mzscheme interpreter  
app-editors/vim:mzscheme - Include support for the mzscheme interpreter  
  
> "local" mode looks similar, but the list of flags is sorted by package. Users  
> can configure which flags are displayed and which packages are displayed.  
> Checkboxes can have three states (+, -, blank). A + or - will create an entry  
in  
> package.use, a blank checkbox will remove any reference to that flag on the  
> associated line in package.use.  
  
In my opinion, it would be nice to also allow this for global USE flags. The  
reason I'm suggesting this is because it would also solve the problems with the  
plans for package-specific USE defaults (if you don't have foo in your USE  
flags, and you emerge bar and baz, bar will be installed with USE=foo, but baz  
will be installed with USE=-foo, because they have different defaults). The  
problem this causes is that currently, you cannot use ufed to add -foo to your  
USE flags, because foo is not enabled globally by default, so you'd have to  
edit make.conf manually. (These plans are not yet implemented in portage, as  
far as I know.) While I'm not particularly fond of these plans myself, that's  
no excuse to make ufed unusable for some users, and this would (I think) solve  
it rather nicely.  
  
As for the rest of your comments about filtering, I'll mostly skip that for 
now, to try to avoid mixing it with editing package.use. But as for the rest of 
your comments about package.use, I don't know how to implement things yet, but 
thank you very much, because it has been very useful in getting a general idea 
of the interface. There are some problems: as an example, vanilla is a global 
USE flag, but should never be enabled globally - even though there's nothing to 
suggest so, this will break your system to the point where you cannot emerge 
glibc anymore. It should be enabled strictly on a per-package basis, for 
packages where it is supported. This doesn't seem to be possible with your 
suggestion. However, if instead of one local mode, several local modes (one for 
each entry in package.use) can be chosen, each local mode can display the 
global as well as local flags used by that specific package. The interface to 
select, add and remove modes/packages would be similar to elinks's bookmarks 
menu. It requires support from portage to display all flags used by one 
specific package, but that's not too big a deal. (Ideally, ufed could find this 
out itself, but it would require duplicating a whole lot of the portage code, 
which is really not worth the effort.) Would this be an acceptable solution? 
 
(Again, even if this is a good idea, sorry, it'll take a while before I can get 
it in ufed.) 
Comment 6 Justus Ranvier 2005-09-05 14:57:05 UTC
> (Again, even if this is a good idea, sorry, it'll take a while before I can get 
> it in ufed.) 

Thank you for listening to my ramblings. I didn't expect you to implement
anything right away; I just wanted to make some suggestions in case they turned
out to be useful.
Comment 7 James Earl Spahlinger 2009-07-24 06:29:54 UTC
At some point between 2005 and now UFED has been modified to have 0 useflags, thus negating the issue on this bug as related to UFED. 
Comment 8 James Earl Spahlinger 2009-07-24 06:34:47 UTC
I missread this, ufed works with useflags, I thought the bug was about the package having too many useflags! I'm sorry, my mistake.
Comment 9 Sven Eden 2012-12-30 10:26:02 UTC
Over seven years after this bug has been opened, it was assigned to me. And it is a shame I didn't see it earlier, because I really do like the idea to have some filtering.

As Harald already wrote it needs a lot of planning and further definition of what is to be filtered.

From what I understood the following functionality *can* be implemented in USE-Flags:

1. Add a filter to only show USE-Flags of installed packages.
  My first idea is to implement the "knowledge" of the relevant USE-flags around /var/db/pkg/<category>/<package>-<version>/USE.
But is this safe?

2. Add a filter to filter out local/global USE-Flags.
  This can be done by a simple assumption: USE-Flags that affect exactly one package are "local", all others are considered "global".
I make this assumption because of the behavior of myself. I tend to put USE-Flags into /etc/portage/package.use only if they do not affect more than this one package.

Note about /etc/portage/package.use:
I found the discussion interesting. It is my belief, that ufed *must* leave that file alone. ufed is a tool to maintain the USE entry in /etc/[portage/]make.conf, and nothing else. The reason is as simply as that: There is no safe way to handle special cases like the ones listed by Harald in comment 3.

Finally I have to ask for some patience. Those two filters are nothing to implement in an instant, of course.
Comment 10 Sven Eden 2013-01-08 10:47:40 UTC
*** Bug 367953 has been marked as a duplicate of this bug. ***
Comment 11 Roman Žilka 2013-01-22 14:10:49 UTC
(In reply to comment #9)
> 1. Add a filter to only show USE-Flags of installed packages.
>   My first idea is to implement the "knowledge" of the relevant USE-flags
> around /var/db/pkg/<category>/<package>-<version>/USE.
> But is this safe?

As of ufed-9999-4c71fca78a4ff4afc8d8380c6221acecf58f123a I can see [L] at "vim-pager" which is used in vim which I have installed. "vim-pager" is turned off in my install. However, I have a [g] at "vanilla" which, however, is used in glibc (which I have installed) and again turned off in my install. How does ufed currently determine which installed-package-relevant USE flags to show with a capital G/L?

Also, let's say packageX-1.0 is installed with flagY in /var/db/pkg/foo-bar/packageX-1.0/USE. Over time flagY may change description/semantics. Or status from global to local (or vice versa). It may become masked and/or removed from make.conf. Or it was a local flag for packageX when packageX-1.0 was emerged, but now packageX-1.0 is obsolete (but still installed), doesn't even have an ebuild in the tree, and PackageX-2.0 doesn't use flagY at all. FlagY doesn't have a description in use.local.flags anymore. And I'm hearing that tomorrow packageABC is about to introduce flagY as its new local flag.

I suppose ufed won't be able to check any time soon for the latest version of a package. So we have to make do with the USE files under /var/db/pkg, AFAIK. The local<-->global change and the description change are thus undetectable for the program, but this should be documented.

As for the case when flagY exists in /var/db/pkg/foo-bar/packageX-1.0/USE, but is neither a global flag, nor a local flag for foo-bar/packageX, I suggest that ufed marks the flag as [L] or [O] ('obsolete') and put something like "(Obsolete, but used by an installed package)" or "(foo-bar/packageX) (Obsolete, but used by an installed package)" in the description. Note that in this illustration flagY may/may not be in make.conf.

FlagY may also become eliminated from both use[.local].desc, but still be recorded as masked and used by some installed package. In this case flagY should not be turn-off-able as it otherwise would be, unless also present in make.conf.
Comment 12 Roman Žilka 2013-01-22 14:19:59 UTC
(In reply to comment #9)
> 2. Add a filter to filter out local/global USE-Flags.
>   This can be done by a simple assumption: USE-Flags that affect exactly one
> package are "local", all others are considered "global".

Personally, I'm strongly opposed to this. Globals are in use.desc and locals in use.local.desc - this denomination is a decision of the package maintainers and arch folks.

By the way, thank you, Sven, for your work. I imagine your aggregated TODO list is somewhat long, but I'll be happy to wait an adequate amount of time for it to be implemented - wholly or partially.
Comment 13 Sven Eden 2013-01-22 15:34:29 UTC
(In reply to comment #12)
> (In reply to comment #9)
> > 2. Add a filter to filter out local/global USE-Flags.
> >   This can be done by a simple assumption: USE-Flags that affect exactly one
> > package are "local", all others are considered "global".
> 
> Personally, I'm strongly opposed to this. Globals are in use.desc and locals
> in use.local.desc - this denomination is a decision of the package
> maintainers and arch folks.

As of writing number 2, I had no clear Overview. But now I have and I'd like to update that point:

2. Add a filter to toggle the display of USE flag descriptions by their scope to either show only local (per package) descriptions (use.local.desc), global descriptions (use.desc) or both (default).

> 
> By the way, thank you, Sven, for your work. I imagine your aggregated TODO
> list is somewhat long, but I'll be happy to wait an adequate amount of time
> for it to be implemented - wholly or partially.

And thank you for your help and your feedback! Nothing is better and more valuable than feedback! Be it praise or criticism is not important. Being constructive is what counts.

I think that once the filters are working and everybody is content with the layout, a new release is due although not all positions from my TODO list are done then. ;)
Comment 14 Sven Eden 2013-01-22 15:36:20 UTC
Just forgot:

Both filters (all/local/global, all/installed/not installed) are WOIP.
Comment 15 Roman Žilka 2013-01-22 15:58:27 UTC
Sorry for being off-topic, but what is WOIP? :)
Comment 16 Sven Eden 2013-01-22 16:05:54 UTC
(In reply to comment #15)
> Sorry for being off-topic, but what is WOIP? :)

[WO]rk [I]n [P]rogress. Written this way anytime your fingers are too thick to hit 'I' alone. ;)
Seriously, I meant "WIP" aka "Work In Progress".
Comment 17 Sven Eden 2013-02-01 10:59:02 UTC
(In reply to comment #14)
> Both filters (all/local/global, all/installed/not installed) are WOIP.

Both filters have been added and assigned to F6 and F7.

Please check current app-portage/ufed-9999 whether you like the result.

ufed is not yet complete, the display of masked local flags (meaning a normal flag where it is masked or forced only for specific package) needs to be improved. (Example: "upnp" A regular flag but masked for kdelibs.)

And of course the documentation has to be updated.

Once everybody is fine with the design and the above two issues are sorted out, I guess ufed is ready for a new release.

Thank you very much for your patience!
Comment 18 Roman Žilka 2013-02-01 14:18:50 UTC
I'm on it!

The bottom help-line has F-keys and their functionality mixed up.

Do I understand that ufed can now recognize global used flags? When I choose all/all/installed, I can see some global flags with 'i'. Also, "(iwmmxt)" has 'i', but I have no relevant package installed.

F5 shows not only masked flags, but also +flag+ flags - bottom help-line needs updating.

+flag+ is a forced flag, right? If so, I shouldn't be able to disable a forced flag.

I can enable the masked waveout flag, but I suppose that's one of the unfinished items you mentioned.

When I mask (/etc/portage/profile/use.mask) "+x86emu+", it is correctly shown only through F5, but it's not marked as "(x86emu)". The local definitions in /etc/portage should override everything.
Comment 19 Sven Eden 2013-02-01 15:27:03 UTC
(In reply to comment #18)
> The bottom help-line has F-keys and their functionality mixed up.
Ah! I re-ordered the F-Keys but not in the texts. Blimey.
-> Fixed!

> Do I understand that ufed can now recognize global used flags? When I choose
> all/all/installed, I can see some global flags with 'i'. Also, "(iwmmxt)"
> has 'i', but I have no relevant package installed.
The global lines get the 'i' if at least one package that has this USE flag in its IUSE is installed.
As for "iwmmxt" the package is x11-libs/pixman.

I plan to add additional lines which list those affected packages. But it has to be done carefully and has to wait until after the next release. The reason is simple: Some use flags affect an insanely high number of packages. Even if filtered for installed packages only. Just think about all packages in kde-* having a "handbook" use flag. (So maybe listing affected packages is generally a stupid idea...)

> 
> F5 shows not only masked flags, but also +flag+ flags - bottom help-line
> needs updating.
> 
> +flag+ is a forced flag, right? If so, I shouldn't be able to disable a
> forced flag.

A forced flag is implicitly masked. 
Example:
 ~ # USE="-minimal" emerge --ask --oneshot media-gfx/iscan-plugin-gt-f500
...
[ebuild  N    ~] media-gfx/iscan-plugin-gt-f500-1.0.0.1  USE="(minimal)" 122 kB

That you can toggle this flag is one of the display issues I mentioned. "minimal" is not a masked or forced flag by default, just for this specific package, so of course you can change it.

I still have to think about a way to display it in a way that does not confuse any more.

> I can enable the masked waveout flag, but I suppose that's one of the
> unfinished items you mentioned.

No, that is actually a bug. This flag has only masked descriptions (one) so it must not be changeable.

-> fixed

> When I mask (/etc/portage/profile/use.mask) "+x86emu+", it is correctly
> shown only through F5, but it's not marked as "(x86emu)". The local
> definitions in /etc/portage should override everything.

x86emu is forced for sys-apps/v86d, not masked. If you want to see it as masked (x86emu) you need to put "x86emu" into /etc/portage/profiles/use.mask and "sys-apps/v86d -x86emu" into  /etc/portage/profile/package.use.force.

-----

Thank you very much for your feedback!
Comment 20 Roman Žilka 2013-02-01 16:27:26 UTC
(In reply to comment #19)
> (In reply to comment #18)
> > Do I understand that ufed can now recognize global used flags? When I choose
> > all/all/installed, I can see some global flags with 'i'. Also, "(iwmmxt)"
> > has 'i', but I have no relevant package installed.
> The global lines get the 'i' if at least one package that has this USE flag
> in its IUSE is installed.
> As for "iwmmxt" the package is x11-libs/pixman.

You're right, I do have pixman.:) My bad.

Also, it's cool that ufed processes IUSE of installed packages.

> I plan to add additional lines which list those affected packages. But it
> has to be done carefully and has to wait until after the next release. The
> reason is simple: Some use flags affect an insanely high number of packages.
> Even if filtered for installed packages only. Just think about all packages
> in kde-* having a "handbook" use flag. (So maybe listing affected packages
> is generally a stupid idea...)

Maybe it would be enough for starters, if ufed just printed the number of installed packages in this case. Or perhaps even print the number of installed packages per package category groups having the same prefix. By this I mean:
xfce-*/*: 7 installed packages
x11-*/*: 0
www-*/*: 2
(...)

But that could grow fairly lengthy too, hmm...

> > When I mask (/etc/portage/profile/use.mask) "+x86emu+", it is correctly
> > shown only through F5, but it's not marked as "(x86emu)". The local
> > definitions in /etc/portage should override everything.
> 
> x86emu is forced for sys-apps/v86d, not masked. If you want to see it as
> masked (x86emu) you need to put "x86emu" into /etc/portage/profiles/use.mask
> and "sys-apps/v86d -x86emu" into  /etc/portage/profile/package.use.force.

OK. Wow, complex stuff this be.

> Thank you very much for your feedback!

Long live ufed!
Comment 21 Roman Žilka 2013-04-13 14:34:25 UTC
With udev-0.90 out this seems closable now.
Comment 22 Sven Eden 2013-04-15 08:09:34 UTC
Implemented in app-portage/ufed-0.90 since rc1.