Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 631764 - app-portage/eix eix-test-obsolete reporting false positives
Summary: app-portage/eix eix-test-obsolete reporting false positives
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Mikle Kolyada (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-22 20:17 UTC by faminebadger
Modified: 2020-04-01 10:48 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description faminebadger 2017-09-22 20:17:38 UTC
eix-test-obsolete doesn't appear to be working as intended.

For example, I just had to add =media-libs/freetype-2.8.1 to my /etc/portage/package.mask because of https://bugs.gentoo.org/631284.

eix-test-obsolete then reports:

Redundant in /etc/portage/package.mask:

... considered as REDUNDANT_IF_MASK_NO_CHANGE
[I] media-libs/freetype (2.8(2)@22/09/17): A high-quality and portable font engine

That makes no sense - freetype-2.8 is not mentioned in my mask file at all, and the version that is mentioned in my mask file (2.8.1) is not at all redundant and did correctly force emerge to downgrade.

This report of obsolescence is completely misleading, and at worst would lead me to incorrectly remove required entries from my config files in future.

The man page is quite clear:  "Applies if /etc/portage/package.mask contains entries which do not change the mask status for the versions in question."  As stated, there is no version mentioned in package.mask which is not changing the mask status of that version.
Comment 1 faminebadger 2017-09-22 20:30:36 UTC
Additionally, the redundancy detection in package.use is completely incorrect too.

Witness the following example using sys-libs/zlib (but I have many more false positives reported):

Redundant in /etc/portage/package.use:
...
[I] sys-libs/zlib (1.2.11@22/09/17): Standard (de)compression library
...
Found 9 matches

Equery reports the use flags as:

gentoo ~ (1) # equery u sys-libs/zlib
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-libs/zlib-1.2.11:
 U I
 + + abi_x86_32  : 32-bit (x86) libraries
 + + minizip     : include the minizip library for quick and dirty zip extraction
 - - static-libs : Build static versions of dynamic libraries as well
gentoo ~ (0) # equery u sys-libs/zlib

So, something must be redundant - right?

Wrong - it's mentioned twice in my package.use:

gentoo ~ (0) # grep sys-libs/zlib /etc/portage/package.use 
>=sys-libs/zlib-1.2.8-r1 abi_x86_32
>=sys-libs/zlib-1.2.8-r1 minizip
gentoo ~ (0) # 

And commenting out both lines causes equery to change to:

gentoo ~ (0) # equery u sys-libs/zlib
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-libs/zlib-1.2.11:
 U I
 - + abi_x86_32  : 32-bit (x86) libraries
 - + minizip     : include the minizip library for quick and dirty zip extraction
 - - static-libs : Build static versions of dynamic libraries as well


So, those flags are NOT redundant, they are doing something!
Comment 2 faminebadger 2017-09-22 20:31:17 UTC
These sort of completely incorrect reports really make my not trust this tool at all - which is a real shame, because it's exactly the sort of tool I want to be able to do maintenance on my config files.
Comment 3 Martin Väth 2017-09-24 20:27:55 UTC
> "Applies if /etc/portage/package.mask contains entries which do not change the  mask status for the versions in question."

: $ eix --print REDUNDANT_IF_MASK_NO_CHANGE
: all-uninstalled
: $ man eix:
:              all-uninstalled
: The redundancy must occurs at least for all uninstalled versions. [...]

According from your description, you have an entry to mask freetype-2.8.1, but apparently no entry to mask freetype-2.7.1-r2, so the status of all uninstalled versions is not changed. Maybe you want to set REDUNDANT_IF_MASK_NO_CHANGE to some-uninstalled for the way how you use masks.

> did correctly force emerge to downgrade.

That's irrelevant: eix -T knows nothing about dependencies. It only reports exactly what is specified by the REDUNDANT_IF_* variables.

> redundancy detection in package.use is completely incorrect too

It reports only exactly what is specified by the REDUNDANT_IF_* variables, in your example probably due to REDUNDANT_IF_DOUBLE_USE:

: Applies if /etc/portage/package.use matches twice for the versions in questions

By default, there is nothing checked for package.use (REDUNDANT_IF_DOUBLE_USE and REDUNDANT_IF_IN_USE are both "false" by default), so I guess you must have set it locally to some value.
Comment 4 faminebadger 2017-09-29 14:43:15 UTC
> It reports only exactly what is specified by the REDUNDANT_IF_* variables,
> in your example probably due to REDUNDANT_IF_DOUBLE_USE:

I think you're correct on this one, I had that set to "some" and on re-reading what it does, I've disabled it again and package.use messages have now gone away.

> According from your description, you have an entry to mask freetype-2.8.1,
> but apparently no entry to mask freetype-2.7.1-r2, so the status of all
> uninstalled versions is not changed. Maybe you want to set
> REDUNDANT_IF_MASK_NO_CHANGE to some-uninstalled for the way how you use
> masks.

I have REDUNDANT_IF_MASK_NO_CHANGE set to "some", which the documentation is quite clear means that it should flag any version in the database, installed or otherwise, whose mask status is unchanged by an entry in package.mask.

However, once again, there is no redundant entry in package.mask - the only version mentioned is effecting a mask change.

It's quite clear that either the documentation is incorrectly explaining the behaviour, or the program isn't working properly.

> That's irrelevant: eix -T knows nothing about dependencies. It only reports
> exactly what is specified by the REDUNDANT_IF_* variables.

You missed my point.  I cited the downgrade as evidence that the mask status had changed on the later version, I'm not claiming anything about dependencies.
Comment 5 Martin Väth 2017-09-30 13:12:02 UTC
(In reply to faminebadger from comment #4)
> I have REDUNDANT_IF_MASK_NO_CHANGE set to "some"

The logic is a bit confusing because \not\forall is logically the same as \exists\not, and \not\exists is logically the same as \forall\not, and the description is no so clear where the negation applies to
(I think that I also confused it in my previous posting):

REDUNDANT_IF_MASK_NO_CHANGE="some" eix -T ...
will output the package if there is _some_ version which is _not_ changed, i.e. if _not all_ versions change their status.

REDUNDANT_IF_MASK_NO_CHANGE="all" eix -T ...
will output the package if _all_ versions are _not_ changed, i.e. if there is _not some_ version which changed its status

REDUNDANT_IF_MASK_NO_CHANGE="all-uninstalled eix -T...
will output the package if _all uninstalled_ versions are _not_ changed, i.e. if there is _not some uninstalled_ version which changed its status.

Usually, one wants the latter (which is why it is the default). This also shouldn't give you the "false" positive in your example.
Comment 6 faminebadger 2017-09-30 18:44:35 UTC
Thanks for the explanation - it helps.

I think I understand what's going on here, and it's a case of extremely poor documentation :)

I had exactly the same logical understanding of *some* version *not* changed as you did, however I understood the list of versions under consideration to be all those mentioned in package.mask, but you seem to be suggesting that's not the case.

i.e. Imagine a hypothetical package media-libs/wibble had versions -1, -2, and -3, where package.mask had >=media-libs/wibble-2 in it.

My understanding of "some" was: *some* version *not* changed from the set [wibble-2, wibble-3] (i.e. the set of versions mentioned in the file).

You seem to be saying "some" is: *some* version *not* changed from the set [wibble-1, wibble-2, wibble-3] (i.e. the set of all versions of all packages mentioned in the file).

I don't understand why anyone would want the latter behaviour - it's pointless, and nearly always guarantees false positives (hence my original problem with it).

This is exactly what happens with my real case - only version 2.8.1 of media-libs/freetype is mentioned in package.mask, and hence I expect *some* *not* changed to match nothing, but in fact it matches something not even mentioned in the version range in package.mask.

Is this really what is intended for this tool?  It's a next to useless mode!

Also, changing to "all-uninstalled" is no good for detecting a version that is redundantly unmasked by package.unmask, since it then becomes the installed version.  Hence why I still maintain "some" is broken - because none of the other settings for the variable work in the way that any normal sane user wants (and indeed understands from the man page description of "some") - that is simply "tell me if I have a line in my package.(un)mask that I don't need".
Comment 7 Martin Väth 2017-10-02 22:22:42 UTC
> I don't understand why anyone would want the latter behaviour - it's pointless
[...]
> "all-uninstalled" is no good for [...] package.unmask

That's why REDUNDANT_IF_UNMASK_NO_CHANGE has the default value "all-installed" while REDUNDANT_IF_MASK_NO_CHANGE has the default value "all-uninstalled".

It is simply the same logic applied to all REDUNDANT_IF... variables.
I agree that for many of these variables only 1 value really makes sense
(well, usually at least 2 values: "false" if you want to switch off the test).

The whole thing was an attempt to apply a relatively simple mechanism - always the same mechanism - to many cases without being too intrusive (and thus costly) for the eix marking/masking code. As
eix --dump-defaults |grep REDUNDANT_IF
will show, for most possible values there is a variable for which just this value makes sense (so that it is used in the defaults). Hence, the mechanism could not be simplified even more. (Historically, whenever I missed something for some REDUNDANT_IF... variable, I had added another value which is then available for every other REDUNDANT_IF... variable, independent of whether it makes sense for any of the other variables.)

> extremely poor documentation 

Yes, it does not describe all cases in detail. Nevertheless it is already much too long. Theoretically, the whole documentation should be reworked completely, but unfortunately this is a Sisyphean task, see e.g. https://github.com/vaeth/eix/issues/41
Comment 8 faminebadger 2017-10-15 10:47:53 UTC
all-installed and all-uninstalled still don't do the job, really, if you want what I would argue is the "normal" use case of detecting a line you don't need in package.mask.

I want it to be detected for both installed and uninstalled packages - i.e. say something that I used to have installed I subsequently remove, then I want to be informed about the now redundant line.

It seems from your responses that there really is no setting for these variables that "just works" - they all do some subset of what you'd want, but nothing does exactly what you'd want.

That's a shame, but seems to be by design rather than a bug as I first assumed.
Comment 9 Martin Väth 2017-10-15 14:10:37 UTC
(In reply to faminebadger from comment #8)
>
> say something that I used to have installed I subsequently remove, then I
> want to be informed about the now redundant line.

If I understood your setting correctly, this is not a job for REDUNDANT_IF_MASK_NO_CHANGE but for REDUNDANT_IF_IN_MASK.
In fact, the default value REDUNDANT_IF_IN_MASK="-some" should cover exactly your  situation: If I understood correctly, after the removal you have no version of the package installed, but (at least) one version of the package is covered by package.mask.

Note, however, that some people dislike this setting, because e.g. they prefer to mask (all versions of) sys-apps/systemd and do not consider this setting redundant. I do not think that it is possible to consider one as redundant and the other not, because it requires to know the _reason_ for the mask. For such cases, I recommend to keep REDUNDANT_IF_IN_MASK="-some" and to explicitly exclude sys-apps/systemd from this test by making use of package.nowarn.
Comment 10 faminebadger 2017-10-16 21:01:00 UTC
It still simply doesn't work.

I've just tested putting REDUNDANT_IF_UNMASK_NO_CHANGE back to its default value of all-installed, per your suggestions, and it is still telling me I have redundant entries that I do not.

Here's the output:

> famine@gentoo ~ (0) $ eix --dump |grep REDUNDANT_IF_UNMASK
> REDUNDANT_IF_UNMASK_NO_CHANGE="all-installed"
> famine@gentoo ~ (0) $ eix-test-obsolete 
> ...
> Redundant in /etc/portage/package.unmask:
> 
> ... considered as REDUNDANT_IF_UNMASK_NO_CHANGE
> [I] app-emulation/wine-any (2.17-r1(2.17)@15/10/17): Free implementation of Windows(tm) on Unix, with optional external patchsets
> [I] app-emulation/wine-desktop-common (20150204@16/06/17): Various desktop menu items and icons for wine
> [I] app-emulation/wine-gecko (2.47-r1(2.47)@16/06/17): A Mozilla Gecko based version of Internet Explorer for Wine
> [I] app-emulation/wine-mono (4.7.1(4.7.1)@22/09/17): Wine Mono is a replacement for the .NET runtime and class libraries in Wine
> [I] app-eselect/eselect-wine (1.2.2@22/09/17): Manage active wine version
> [I] virtual/wine (0-r6@29/09/17): Virtual for Wine that supports multiple variants and slotting
> Found 6 matches
> ...
> famine@gentoo ~ (0) $ egrep '(app-emulation/wine|app-eselect/eselect-wine)' /etc/portage/package.unmask |egrep -v '^#' |sort
> >=app-emulation/wine-any-2.9
> >=app-emulation/wine-desktop-common-20150204
> >=app-emulation/wine-gecko-2.47-r1
> >=app-emulation/wine-mono-4.7.0
> >=app-eselect/eselect-wine-1.1
> famine@gentoo ~ (0) $ egrep '(app-emulation/wine|app-eselect/eselect-wine)' /etc/portage/package.unmask |egrep -v '^#' |eix '-|*' --format '<installedversions:NAMEVERSION>' |sort
> app-emulation/wine-any-2.17-r1
> app-emulation/wine-desktop-common-20150204
> app-emulation/wine-gecko-2.47-r1
> app-emulation/wine-mono-4.7.1
> app-eselect/eselect-wine-1.2.2
> famine@gentoo ~ (0) $ 

And all of those packages are masked by default, meaning my mask line *is* important and not redundant.

While I appreciate your time and efforts in trying to help, I think this conversation isn't going anywhere - the tool simply seems incapable of doing what I (and I would argue most "normal" users) want, and frankly I'm fed up trying to get it to do so.

I think I'll write a custom tool of my own to do the job.
Comment 11 faminebadger 2017-10-16 21:11:54 UTC
Sigh - looks like I'm wrong - someone's unmasked those packages whilst this conversation has been happening, so they're no longer the good example they were.

You can pretty much ignore the above comment then - I'll try with your recommended settings for a while and see if I observe any more problems.

I think I've definitely proven that it's all too confusing and ill explained for your average user though!
Comment 12 Manfred Knick 2019-06-30 13:46:00 UTC
Perhaps related:    (If not, I'd be happy to excise this as a separate bug)

         ? Wrong Error-Message selected ?

Running 'eix-update && eix-test-obsolete' :

First, concedes that
   "The names of all installed packages are in the database."

Later on, it criticises
   "Following installed Packages are not in database:"

    [?] app-text/opensp (1.5.2-r6@28.06.2019 -> 1.5.2-r3):  < ... >


# equery list -p app-text/opensp

   [-P-] [  ] app-text/opensp-1.5.2-r3:0
   [IP-] [ ~] app-text/opensp-1.5.2-r6:0     <-----

Unmasking " =app-text/opensp-1.5.2-r6:0 ~amd64 ",
eix-test-obsolete feels satisfied 
and reports no complaints any more.
Comment 13 Martin Väth 2019-07-01 20:40:02 UTC
(In reply to Manfred Knick from comment #12)
> Perhaps related

No, and this works as intended:

>    "The names of all installed packages are in the database."

Indeed, app-text/opensp is in the database.

>    "Following installed Packages are not in database:"

In current versions the text is clearer:
"Installed packages with a version not in the database (or masked):"

>     [?] app-text/opensp (1.5.2-r6@28.06.2019 -> 1.5.2-r3):  < ... >

Indeed, app-text/opensp-1.5.2-r6 is probably masked according to your information.