Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 695172 - Please clarify QA policy regarding USE flags with underscores
Summary: Please clarify QA policy regarding USE flags with underscores
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Council
Classification: Unclassified
Component: unspecified (show other bugs)
Hardware: All All
: Normal normal
Assignee: Gentoo Council
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-20 07:59 UTC by Arfrever Frehtes Taifersar Arahesis
Modified: 2019-11-10 19:24 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
List of USE flags with an underscore as of 2008-09-11 (flags.txt,3.79 KB, text/plain)
2019-09-26 17:49 UTC, Ulrich Müller
Details
List of USE flags with an underscore as of 2008-09-11 (flags.txt,339 bytes, text/plain)
2019-09-26 18:02 UTC, Ulrich Müller
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arfrever Frehtes Taifersar Arahesis 2019-09-20 07:59:34 UTC
https://projects.gentoo.org/pms/7/pms.html#x1-200003.1.4 contains:

"Underscores should be considered reserved for USE_EXPAND, as described in section 11.1.1."

This ambiguous sentence was used as pretext for adding a new check in pkgcheck tool and filing dozens of bugs.

Package managers have no problems with handling of non-USE_EXPAND USE flags with underscores.

If any sentence is needed at all, I suggest re-phrasing to:

"One of possible uses of underscores is for USE flags related to USE_EXPAND, as described in section 11.1.1."
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-09-20 09:12:28 UTC
It's not ambiguous.  I have verified its meaning before adding the check to pkgcheck.  The problem is not technical 'problems' but confusion by flags that look like USE_EXPAND but don't work that way.
Comment 2 Ulrich Müller gentoo-dev 2019-09-20 19:50:42 UTC
The wording has already been changed from "must" to "should" in r64, and I believe it is fine as it is.
Comment 3 Arfrever Frehtes Taifersar Arahesis 2019-09-20 23:50:10 UTC
I would like to request appeal from Council.

There are no technical reasons to disallow underscores in arbitrary USE flags.

Output of `emerge` properly distinguishes USE flags not related to USE_EXPAND and USE flags related to USE_EXPAND.
(If another tool does not yet print USE flags in nice format, then it can be improved, but existence of such a tool is not a reason to disallow underscores in arbitrary USE flags.)


Suggested possible resolutions:

1. Dropping sentence.

2. Re-phrasing sentence to something like:
"One of possible uses of underscores is for USE flags related to USE_EXPAND, as described in section 11.1.1."

3. Re-phrasing sentence to something like:
"USE flags related to USE_EXPAND should be described in profiles/desc/*.desc instead of profiles/use.desc or metadata.xml."

4. Keeping sentence unchanged, but requesting disabling new "QA" check.


(Bugs filed as result of new "QA" check are all bugs with numbers from bug #695028 to bug #695170.)
Comment 4 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2019-09-25 23:45:43 UTC
I want to join Arfrever's request. 
This just imposes way too much hassle to developers (who now have to deal with "QA" bugs in order to change the USE flags in their packages) as well as users who have set such USE flags in make.conf or package.use for nor real visible benefit.
Comment 5 Ulrich Müller gentoo-dev 2019-09-26 11:03:52 UTC
This is really about tree policy in the Gentoo repository. The PMS wording has not changed since 2007, and has been approved by the council (first for EAPI 0, in the 2008-09-11 meeting).

By PMS section 1.3, https://projects.gentoo.org/pms/7/pms.html#x1-80001.3:
"When reporting issues, remember that this document is not the appropriate place for pushing through changes to the tree or the package manager, except where those changes are bugs."

So, if you want to discuss tree policy, then please start a thread in the gentoo-dev mailing list.
Comment 6 Mike Gilbert gentoo-dev 2019-09-26 13:55:20 UTC
There's already a thread about the issue on gentoo-dev. It hasn't come to any real resolution, so I would also like the council to make a decision.
Comment 7 Ulrich Müller gentoo-dev 2019-09-26 14:51:01 UTC
As I said in comment #2, the wording was changed in 2007 from "Underscores are reserved for USE_EXPAND" to "Underscores should be considered reserved for USE_EXPAND":
https://gitweb.gentoo.org/proj/pms.git/commit/?id=1eb32b0025ec2d6f8f47756bb65fe2b7d0109540

There is no rationale in the commit message, but I presume that back then we had a few normal USE flags with a underscore in their name (which would be kept). On the other hand, adding new flags with an underscore in their name should be avoided.

I don't understand why this wording is now being opposed, after it has been in place for 12 years. Also I think it is perfectly fine that adding new flags with an underscore is forbidden by policy (and verified by a QA check). After all, it *is* a collision of name spaces between normal flags and USE_EXPAND ones.
Comment 8 Mike Gilbert gentoo-dev 2019-09-26 16:03:32 UTC
(In reply to Ulrich Müller from comment #7)

My main objection is the relatively new QA check that is flagging *existing* USE flags that contain underscores.

If those could be grandfathered in, and we simply prohibit *new* flags with underscores, that is perfectly acceptable to me.
Comment 9 Mike Gilbert gentoo-dev 2019-09-26 16:05:14 UTC
Adding QA in case anyone else on that team wants to weigh-in.
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-09-26 16:51:50 UTC
(In reply to Mike Gilbert from comment #8)
> (In reply to Ulrich Müller from comment #7)
> 
> My main objection is the relatively new QA check that is flagging *existing*
> USE flags that contain underscores.
> 
> If those could be grandfathered in, and we simply prohibit *new* flags with
> underscores, that is perfectly acceptable to me.

What is your solution?  Are you suggesting that pkgcheck should carry hardcoded list of special historical exceptions?  All of them, or only those whose maintainers oppose fixing this?
Comment 11 Mike Gilbert gentoo-dev 2019-09-26 17:03:17 UTC
(In reply to Michał Górny from comment #10)
> What is your solution?

That's not my problem to solve. The limitations of your tools should not force maintainers to change USE flag names.
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-09-26 17:09:22 UTC
I'm sorry if you belong to this class of developers who believe that Gentoo is a playground where every developer just cares about his toys, and limits his interest in global issues to 'do whatever, as long as you don't bother me'.  Sadly, this gets us nowhere.

So please either put some effort to contribute constructively into finding a real solution, or don't stand in the way of imperfect but reasonably working solutions.  If renaming a few flags is what it takes, it's still better than pretending that the issue does not exist and letting it grow even further.
Comment 13 Mike Gilbert gentoo-dev 2019-09-26 17:19:48 UTC
I would like to formally request that the QA team vote on a policy regarding USE flags with underscores for existing packages.

My stance is that changing existing USE flag names is harmful to users and should be avoided.
Comment 14 Ulrich Müller gentoo-dev 2019-09-26 17:42:32 UTC
(In reply to Mike Gilbert from comment #13)
> I would like to formally request that the QA team vote on a policy regarding
> USE flags with underscores for existing packages.

Cutoff date for this would be when? Approval date of the policy, i.e., all USE flags existing on 2008-09-11 would be grandfathered?
Comment 15 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-09-26 17:45:47 UTC
(In reply to Ulrich Müller from comment #14)
> (In reply to Mike Gilbert from comment #13)
> > I would like to formally request that the QA team vote on a policy regarding
> > USE flags with underscores for existing packages.
> 
> Cutoff date for this would be when? Approval date of the policy, i.e., all
> USE flags existing on 2008-09-11 would be grandfathered?

If you want to go for that, I'd suggest making a list first.  It would silly if it turned out that none of the original flags remain after the vote.
Comment 16 Ulrich Müller gentoo-dev 2019-09-26 17:49:29 UTC
Created attachment 591126 [details]
List of USE flags with an underscore as of 2008-09-11

Here's the list as of 2008-09-11, extracted from use.desc and use.local.desc.
Comment 17 Ulrich Müller gentoo-dev 2019-09-26 18:02:46 UTC
Created attachment 591128 [details]
List of USE flags with an underscore as of 2008-09-11

Sorry, I made a mistake when extracting the list. Here is the correct one.

So in fact, the overlap with the flags existing today is small:

   SECURITY_HAZARD
   mpi_njtree
   pam_nuauth
   static_pairalign
Comment 18 Tim Harder gentoo-dev 2019-09-26 21:48:57 UTC
(In reply to Michał Górny from comment #10)
> (In reply to Mike Gilbert from comment #8)
> > (In reply to Ulrich Müller from comment #7)
> > 
> > My main objection is the relatively new QA check that is flagging *existing*
> > USE flags that contain underscores.
> > 
> > If those could be grandfathered in, and we simply prohibit *new* flags with
> > underscores, that is perfectly acceptable to me.
> 
> What is your solution?  Are you suggesting that pkgcheck should carry
> hardcoded list of special historical exceptions?  All of them, or only those
> whose maintainers oppose fixing this?

From a technical, pkgcheck-dev perspective, it is possible to flag newly added flags or other changes before a dev pushes them to git (e.g. see how DroppedStableKeywords is implemented).

However, enshrining such policies as official QA standards would require either forcing pkgcheck usage or developing the same checks in repoman as those checks most be run individually by developers before pushing to git.
Comment 19 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-09-27 05:56:05 UTC
(In reply to Tim Harder from comment #18)
> However, enshrining such policies as official QA standards would require
> either forcing pkgcheck usage or developing the same checks in repoman as
> those checks most be run individually by developers before pushing to git.

...and presumes that the developer will actually a. run either, b. respect the results, without giving us any technical way of monitoring the situation.  Now, let's look at git log how many commits are made with --force...
Comment 20 Arfrever Frehtes Taifersar Arahesis 2019-09-27 08:51:57 UTC
(In reply to Mike Gilbert from comment #13)
> I would like to formally request that the QA team vote on a policy regarding
> USE flags with underscores for existing packages.
> 
> My stance is that changing existing USE flag names is harmful to users and
> should be avoided.

I would like to formally suggest an alternative policy, based not on date of introduction of USE flags, but their possibility to imitate USE_EXPAND-related USE flags.

Potential phrasing:
"Packages should not have USE flags not related to USE_EXPAND which imitate USE flags related to USE_EXPAND."

Technically it can be implemented by detecting situation where a package has many USE flags which have common prefix which matches e.g.:

.*_(card|device|feature|module|platform|plugin|target)s?_


Currently USE_EXPAND contains:
  2 elements matching .*_CARDS
  3 elements matching .*_DEVICES
  2 elements matching .*_FEATURES
  6 elements matching .*_MODULES
  1 element  matching .*_PLATFORMS
  4 elements matching .*_PLUGINS
  2 elements matching .*_TARGET
  7 elements matching .*_TARGETS
Comment 21 Arfrever Frehtes Taifersar Arahesis 2019-10-13 17:26:23 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #3)
> I would like to request appeal from Council.
> 
> There are no technical reasons to disallow underscores in arbitrary USE
> flags.
> 
> Output of `emerge` properly distinguishes USE flags not related to
> USE_EXPAND and USE flags related to USE_EXPAND.
> (If another tool does not yet print USE flags in nice format, then it can be
> improved, but existence of such a tool is not a reason to disallow
> underscores in arbitrary USE flags.)

The appeal to Council about decision of PMS project has been never withdrawn.

USE flags with underscores have been always working in practice.
I think that underscores should be allowed in both pre-existing USE flags and potential future USE flags.

Outside of PMS, there could be a policy to not use USE flags which could be confusing to users (e.g. multiple USE flags with "${something}_targets_" prefix).

If, for aesthetic reasons, a separator not used for anything else was desired, then a character currently not allowed in USE flags should be chosen, instead of stealing "_".
":" (colon) would be a good choice and is also used in Exherbo.
Example: "video_cards:intel"   (Name in Exherbo: "video_drivers:intel")
Comment 22 Ulrich Müller gentoo-dev 2019-11-10 19:24:50 UTC
The council has accepted the following motion in its 2019-10-13 meeting:
"Underscores should be considered reserved for USE_EXPAND. They must not be used in any new 'normal' USE flags. Existing flags are technically valid; phasing them out has low priority."