Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 639724 - Make repoman complain about KEYWORDS spanning multiple lines (ekeyword can't handle it)
Summary: Make repoman complain about KEYWORDS spanning multiple lines (ekeyword can't ...
Status: RESOLVED WONTFIX
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Repoman (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-04 07:22 UTC by Sergei Trofimovich (RETIRED)
Modified: 2022-07-12 03:18 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergei Trofimovich (RETIRED) gentoo-dev 2017-12-04 07:22:17 UTC
The problem was noticed by kent\n today when I missed to keyword one simple package:


$ ekeyword hppa dev-perl/Archive-Zip/Archive-Zip-1.590.0.ebuild
$ Archive-Zip-1.590.0: no updates

Unfortunately it's a lie caused by the fact KEYWORDS takes 2 lines:

https://gitweb.gentoo.org/repo/gentoo.git/commit/dev-perl/Archive-Zip?id=ccbf60935d12f3e82a292a02036c26a3d05fb090

It's arguably an ekeyword bug but it's hard to expect ekeyword to handle
anything more complicate that direct single-line assignment.

There is at least 7 ebuilds in tree like that:

git grep -E 'KEYWORDS="[^"]+$' | cat
dev-perl/Archive-Zip/Archive-Zip-1.590.0.ebuild:KEYWORDS="alpha amd64 arm ~arm64 ~hppa ia64 ppc ppc64 sparc x86 ~x86-fbsd ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~sparc-solaris
dev-perl/Test-MockModule/Test-MockModule-0.110.0.ebuild:KEYWORDS="alpha amd64 arm ~arm64 ~hppa ia64 ppc ppc64 sparc x86 ~x86-fbsd ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~sparc-solaris
media-libs/jasper/jasper-9999.ebuild:   KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~mips ~ppc ~ppc64 ~s390 \
media-video/libav/libav-12.1.ebuild:[[ ${PV} == *9999 ]] || KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~mips ~ppc ~ppc64
media-video/libav/libav-12.2.ebuild:[[ ${PV} == *9999 ]] || KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~mips ~ppc ~ppc64
media-video/libav/libav-12.ebuild:[[ ${PV} == *9999 ]] || KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~mips ~ppc ~ppc64
media-video/libav/libav-9999.ebuild:[[ ${PV} == *9999 ]] || KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~mips ~ppc ~ppc64
Comment 1 Larry the Git Cow gentoo-dev 2017-12-04 07:30:21 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=32409c853c19870cc61dd4d29dafb40caf2feca2

commit 32409c853c19870cc61dd4d29dafb40caf2feca2
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2017-12-04 07:27:30 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2017-12-04 07:30:14 +0000

    dev-perl/Test-MockModule: make KEYWORDS to span single line, unbreaks 'ekeywords'
    
    Bug: https://bugs.gentoo.org/639724
    Package-Manager: Portage-2.3.16, Repoman-2.3.6

 dev-perl/Test-MockModule/Test-MockModule-0.110.0.ebuild | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a645be76416c91bd9e3a161065b3a508f345de26

commit a645be76416c91bd9e3a161065b3a508f345de26
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2017-12-04 07:25:56 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2017-12-04 07:30:14 +0000

    dev-perl/Archive-Zip: make KEYWORDS to span single line, unbreaks 'ekeywords'
    
    Bug: https://bugs.gentoo.org/639724
    Package-Manager: Portage-2.3.16, Repoman-2.3.6

 dev-perl/Archive-Zip/Archive-Zip-1.590.0.ebuild | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)}
Comment 2 Ulrich Müller gentoo-dev 2017-12-04 10:16:22 UTC
(In reply to Sergei Trofimovich from comment #0)
> It's arguably an ekeyword bug but it's hard to expect ekeyword to handle
> anything more complicate that direct single-line assignment.

It is definitely an ekeyword bug. And I don't think that we have any policy that would restrict variable assignments to a single line.

Also, what about conditional assignment, which I have seen frequently in live ebuilds?

if [[ ${PV##*.} == 9999 ]]; then
        KEYWORDS=""
else
        KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~ppc ~ppc64 ~s390 ~sh ~sparc ~x
86 ~amd64-fbsd ~x86-fbsd ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-maco
fi

ekeyword will horribly fail on this one. For example, trying to stabilise the non-live ebuild for amd64 will result in amd64 being added to *both* lines. And next time you sync the live ebuild it may end up with a stable keyword if you don't pay attention.

IMHO we shouldn't impose arbitrary restrictions on ebuilds because of one buggy tool. Fix the tool, or live with the occasional outage.
Comment 3 Ulrich Müller gentoo-dev 2017-12-04 10:20:28 UTC
(In reply to Ulrich Müller from comment #2)
> if [[ ${PV##*.} == 9999 ]]; then
>         KEYWORDS=""
> else
>         KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~ppc ~ppc64 ~s390 ~sh
> ~sparc ~x
> 86 ~amd64-fbsd ~x86-fbsd ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos
> ~x86-maco
> fi

Sorry, copy and paste failure. The second line should read:

        KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-fbsd ~x86-fbsd ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos"
Comment 4 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2017-12-04 12:10:48 UTC
I don't think its viable however to expect ekeyword to parse bash code effectively with full awareness of bashes nuances, and know which field to tweak.

It just makes more sense to have a QA Policy that indicates that ebuilds are less accessible to a wide variety of tools (eg: naive greps) when KEYWORDS is split over multiple lines, and formally discourage it.

As for the -9999 case, the recommendation should be that KEYWORDS is simply not defined in the -9999 side of things.

That way no matter what ekeyword does to the existing KEYWORDS statements, they'll only affect what happens on the non-development versions.

if [[ "${PV##*.}" != "9999" ]]; then
KEYWORDS="..."
fi


(Note: I also left-align this KEYWORDS assignment just in case it helps some tool)
Comment 5 Sergei Trofimovich (RETIRED) gentoo-dev 2017-12-04 21:08:07 UTC
(In reply to Ulrich Müller from comment #2)
> (In reply to Sergei Trofimovich from comment #0)
> > It's arguably an ekeyword bug but it's hard to expect ekeyword to handle
> > anything more complicate that direct single-line assignment.
> 
> It is definitely an ekeyword bug. And I don't think that we have any policy
> that would restrict variable assignments to a single line.

That's why qa@ is CCed: qa@ can decide to have (or not have) such a policy.
It qa@ decided not to I'm fine with that. But the decision must be stated
clearly.

> Also, what about conditional assignment, which I have seen frequently in
> live ebuilds?
> 
> if [[ ${PV##*.} == 9999 ]]; then
>         KEYWORDS=""
> else
>         KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~ppc ~ppc64 ~s390 ~sh
> ~sparc ~x
> 86 ~amd64-fbsd ~x86-fbsd ~amd64-linux ~x86-linux ~ppc-macos ~x64-macos
> ~x86-maco
> fi
> 
> ekeyword will horribly fail on this one. For example, trying to stabilise
> the non-live ebuild for amd64 will result in amd64 being added to *both*
> lines. And next time you sync the live ebuild it may end up with a stable
> keyword if you don't pay attention.

From practical standpoint current ekeyword's behaviour is perfectly fine.
It gets the job done. KEYWORDS are populated and ebuild is promoted to
stable.

From aestetic standpoint arguably there should be no multiple KEYWORDS
assignment at all. In this case KEYWORDS="" is redundant but not harmful
to version at hand.

> Fix the tool, or live with the occasional outage.

How the fixed tool should behave in this case in your opinion?

KEYWORDS changes is not something that is done one-off or manually.
Unique snowflake ebuild will trip over every arch tester who has to
commit keyword changes.
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-04 22:20:15 UTC
I don't like long lines either but given that ekeyword is commonly used by arch testers, I agree with this policy. At least until someone 'fixes' ekeyword to handle line wrapping sanely, and we can rely on the new version being reasonably deployed.
Comment 7 Sergei Trofimovich (RETIRED) gentoo-dev 2020-02-18 13:18:43 UTC
(In reply to Sergei Trofimovich from comment #5)
> (In reply to Ulrich Müller from comment #2)
> > (In reply to Sergei Trofimovich from comment #0)
> > > It's arguably an ekeyword bug but it's hard to expect ekeyword to handle
> > > anything more complicate that direct single-line assignment.
> > 
> > It is definitely an ekeyword bug. And I don't think that we have any policy
> > that would restrict variable assignments to a single line.
> 
> That's why qa@ is CCed: qa@ can decide to have (or not have) such a policy.
> It qa@ decided not to I'm fine with that. But the decision must be stated
> clearly.

Can we have an explicit statement that reflects final qa@ team's decision?
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-02-18 16:04:43 UTC
I'll make a policy guide patch later today and put it under vote.
Comment 9 Larry the Git Cow gentoo-dev 2020-02-21 11:07:50 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/policy-guide.git/commit/?id=0834171b83aead95ae39e51272a9f89bb785e28b

commit 0834171b83aead95ae39e51272a9f89bb785e28b
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2020-02-18 17:05:52 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2020-02-21 11:07:40 +0000

    ebuild-format: Require KEYWORDS to be on one line
    
    Bug: https://bugs.gentoo.org/639724
    Closes: https://github.com/gentoo/policy-guide/pull/13
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 ebuild-format.rst | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
Comment 10 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-07-12 03:18:31 UTC
repoman support has been removed per bug 835013.

Please file a new bug (or, I suppose, reopen this one) if you feel this check is still applicable to pkgcheck and doesn't already exist.