Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 634824 - app-portage/gentoolkit - equery uses shows incorrect status for forced flags
Summary: app-portage/gentoolkit - equery uses shows incorrect status for forced flags
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2017-10-19 21:11 UTC by faminebadger
Modified: 2019-08-19 07:38 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
remove check for flags from usemasked or useforced (equery_uses_forced_flag_bug.patch,440 bytes, patch)
2019-07-28 10:56 UTC, nnbrain8
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description faminebadger 2017-10-19 21:11:29 UTC
In the process of trying to debug an ffmpeg incompatibility with kodi, I spotted some strange equery uses output:

> famine@gentoo ~ (0) $ equery u kodi
> [ Legend : U - final flag setting for installation]
> [        : I - package is installed with flag     ]
> [ Colors : set, unset                             ]
>  * Found these USE flags for media-tv/kodi-17.4-r1:
>  U I
>  + + X                        : Add support for X11
> ...
>  - - sftp                     : Support browsing files over SFTP
>  - + system-ffmpeg            : Use system ffmpeg instead of the bundled one
>  - - systemd                  : Enable use of systemd-specific libraries and features like socket activation or session tracking
> ...
>  - - zeroconf                 : Support for DNS Service Discovery (DNS-SD)
> famine@gentoo ~ (0) $ 

The important point here is that system-ffmpeg is shown as not on by default, but installed with it on anyway.

This isn't true - system-ffmpeg is not only on by default in the kodi package, I also have it set in my make.conf for general use.

I debugged it a bit, and I can see that the python code for filtering use flags appears to be doing the wrong thing with package.use.force.

Here is (a summary of) the debug session:

> Python 3.4.6 (default, Sep 22 2017, 14:35:47) 
> [GCC 6.4.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from gentoolkit.flag import get_flags
> >>> final, hidden, mask, force = get_all_cpv_use("media-tv/kodi-17.4-r1")
> >>> print("system-ffmpeg" in final)
> True
> >>> "system-ffmpeg" in final
> True
> >>> "system-ffmpeg" in hidden
> False
> >>> "system-ffmpeg" in mask
> False
> >>> "system-ffmpeg" in force
> True
> >>> result=filter_flags(final, hidden, mask, force)
> >>> "system-ffmpeg" in result
> False
> >>> 

And indeed, it is forced on by package.use.force (system wide):

> famine@gentoo ~ (0) $ egrep '(kodi|ffmpeg)' /usr/portage/**/*.force
> /usr/portage/profiles/base/package.use.force:# Only ever use system ffmpeg, allow kodi bundled ffmpeg
> /usr/portage/profiles/base/package.use.force:media-tv/kodi system-ffmpeg
> famine@gentoo ~ (0) $

So - for some reason the function filter_flags has removed a flag that is in fact enabled by package.use.force

Looking at that function, it does indeed seem to treat package.use.force as a mask, whereas my understanding of this file is that it may contain both "+" and "-" type use lines:
>	# dbl check if any from usemasked  or useforced are still there
>	masked = usemasked + useforced
>	for a in use[:]:
>		if a in masked:
>			use.remove(a)
>	return use
Comment 1 faminebadger 2018-01-07 06:30:02 UTC
Shame, I went to the effort of investigating this right through to pointing out the exact lines of code that aren't working, and nobody has even bothered to look at it.
Comment 2 Franz Trischberger 2018-02-08 11:52:00 UTC
Seems here is another report:
https://bugs.gentoo.org/646874
Would be great to see a fix for this issue, especially since there already is a hint on the source!
Comment 3 Brian Dolbec gentoo-dev 2018-02-08 18:06:50 UTC
Currently there is no one actively working on gentoolkit.  Myself, I have been swamped with work, family and other things in Gentoo, and getting even farther behind...

If someone is able to supply a patch, that would help us out a tremendous amount.
Comment 4 faminebadger 2018-04-02 00:10:59 UTC
Months later, no action here.

The output of equery use continues to be complete bollocks.  I don't trust it at all now.

The below is output for an uninstalled package so I can try and see what it will do before I waste an age compiling, but it's just crap.

hangouts/proprietary-codes/suid it says are installed off, despite package not being installed and none of these use flags being mentioned in any of my configuration files.  God knows what it means by this.

system-libvpx it says will be set off, despite it being set *ON* in my make.conf.

emerge --info agrees with me that hangouts/proprietary-codes/suid are not mentioned and system-libvpx is on.

Disinformation is worse than no information, sigh.  This feature should be disabled if it won't be fixed.

> famine@gentoo ~ (0) $ equery u -i chromium
> [ Legend : U - final flag setting for installation]
> [        : I - package is installed with flag     ]
> [ Colors : set, unset                             ]
>  * Found these USE flags for www-client/chromium-66.0.3359.45:
>  U I
>  - - component-build    : Split build into more shared libraries to speed up linking. Mostly intended for debugging
>                           and development, NOT RECOMMENDED for general use.
>  + + cups               : Add support for CUPS (Common Unix Printing System)
>  - - custom-cflags      : Build with user-specified CFLAGS (unsupported)
>  - - gnome-keyring      : Enable support for storing passwords via gnome-keyring
>  + - hangouts           : Enable support for Google Hangouts features such as screen sharing
>  - - jumbo-build        : Combine source files to speed up build process.
>  - - kerberos           : Add kerberos support
>  + - proprietary-codecs : Enable proprietary codecs like H.264, MP3
>  + + pulseaudio         : Add support for PulseAudio sound server
>  + - suid               : Build the SUID sandbox, which is only needed on CONFIG_USER_NS=n kernels
>  + + system-ffmpeg      : Use system ffmpeg instead of the bundled one
>  + + system-icu         : Use system icu instead of the bundled one
>  - + system-libvpx      : Use system libvpx instead of the bundled one
>  - - tcmalloc           : Use bundled tcmalloc instead of system malloc
>  - - widevine           : Unsupported closed-source DRM capability (required by Netflix VOD)
> famine@gentoo ~ (0) $
Comment 5 nnbrain8 2019-07-28 10:56:29 UTC
Created attachment 584904 [details, diff]
remove check for flags from usemasked or useforced

I encountered this issue and found that simply deleting the lines below that check package.use.force seems to solve the issue. (see patch)

>	# dbl check if any from usemasked  or useforced are still there
>	masked = usemasked + useforced
>	for a in use[:]:
>		if a in masked:
>			use.remove(a)

This patch is a bit of a hack and I'm not sure if it causes any other problems, but I haven't noticed any issues so far.
Comment 6 Zac Medico gentoo-dev 2019-07-28 23:59:16 UTC
I just checked, and the flags going into the filter_flags function may have a + prefix which will cause unintended behavior:

['+xslt', 'python_targets_python2_7', 'dvd', 'udisks', '+system-ffmpeg', 'pulseaudio', 'nonfree', 'mariadb', 'gles', 'samba', 'vaapi', 'dbus', 'bluetooth', 'lirc', 'zeroconf', 'libusb', 'upnp', 'libressl', 'alsa', 'caps', 'cec', '+css', '+opengl', 'vdpau', 'bluray', 'kernel_linux', 'debug', 'mysql', 'test', 'upower', 'airplay', '+udev', 'webserver', 'nfs', 'java', 'sftp', 'lcms', 'systemd']
Comment 7 Zac Medico gentoo-dev 2019-07-29 00:55:55 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=8eeb4ee67da8ce2143f26f07545e666b3a4ad610

commit 8eeb4ee67da8ce2143f26f07545e666b3a4ad610
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-07-29 00:48:07 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-07-29 00:50:23 +0000

    filter_flags: handle default IUSE correctly with reduce_flag
    
    Fix comparisons to use the result of reduce_flag.
    
    Bug: https://bugs.gentoo.org/690786
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 pym/gentoolkit/flag.py | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
Comment 8 Zac Medico gentoo-dev 2019-07-29 01:07:30 UTC
(In reply to faminebadger from comment #0)
> So - for some reason the function filter_flags has removed a flag that is in
> fact enabled by package.use.force

Apparently that was the intended behavior of the function, though it did not correctly handle comparisons for flags with default IUSE.

(In reply to Zac Medico from comment #7)
> The bug has been referenced in the following commit(s):
> 
> https://gitweb.gentoo.org/proj/gentoolkit.git/commit/
> ?id=8eeb4ee67da8ce2143f26f07545e666b3a4ad610
> 
> commit 8eeb4ee67da8ce2143f26f07545e666b3a4ad610
> Author:     Zac Medico <zmedico@gentoo.org>
> AuthorDate: 2019-07-29 00:48:07 +0000
> Commit:     Zac Medico <zmedico@gentoo.org>
> CommitDate: 2019-07-29 00:50:23 +0000
> 
>     filter_flags: handle default IUSE correctly with reduce_flag
>     
>     Fix comparisons to use the result of reduce_flag.
>    
>     Bug: https://bugs.gentoo.org/690786
>     Signed-off-by: Zac Medico <zmedico@gentoo.org>
> 
>  pym/gentoolkit/flag.py | 19 +++++++++----------
>  1 file changed, 9 insertions(+), 10 deletions(-)

This causes the code to behave as intended, so that equery uses will not display any forced or masked flags.
Comment 9 Zac Medico gentoo-dev 2019-07-29 01:09:29 UTC
(In reply to nnbrain8 from comment #5)
> Created attachment 584904 [details, diff] [details, diff]
> remove check for flags from usemasked or useforced
> 
> I encountered this issue and found that simply deleting the lines below that
> check package.use.force seems to solve the issue. (see patch)
> 
> >	# dbl check if any from usemasked  or useforced are still there
> >	masked = usemasked + useforced
> >	for a in use[:]:
> >		if a in masked:
> >			use.remove(a)
> 
> This patch is a bit of a hack and I'm not sure if it causes any other
> problems, but I haven't noticed any issues so far.

Lets not change default behavior of the function. We can add a parameter that allows the behavior to be overridden, or else define a new function to give the desired result.
Comment 10 Zac Medico gentoo-dev 2019-07-29 01:13:01 UTC
Perhaps it's best if equery uses does not display forced/masked flags, since it's generally not intended for the user to modify these flags. Otherwise, we should probably update the display so that forced/masked flags are distinguishable from regular flags.
Comment 11 Larry the Git Cow gentoo-dev 2019-08-19 05:06:39 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e5b2a1086997f3bda772910edf5199a3827b5137

commit e5b2a1086997f3bda772910edf5199a3827b5137
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-08-19 04:00:56 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-08-19 05:06:14 +0000

    app-portage/gentoolkit: Bump to version 0.4.6
    
    Bug: https://bugs.gentoo.org/463952
    Bug: https://bugs.gentoo.org/634824
    Package-Manager: Portage-2.3.71, Repoman-2.3.17
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 app-portage/gentoolkit/Manifest                |  1 +
 app-portage/gentoolkit/gentoolkit-0.4.6.ebuild | 74 ++++++++++++++++++++++++++
 2 files changed, 75 insertions(+)