Created attachment 539294 [details] repoman-bug.tar.gz Here is the full contents of the repository that triggers error (also attached): $ cat metadata/layout.conf masters = gentoo manifest-hashes = SHA256 SHA512 WHIRLPOOL update-changelog = true thin-manifests = true sign-commits = true sign-manifests = false cache-formats = md5-dict pms $ cat net-im/a/a-0.ebuild # Copyright 1999-2018 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 EAPI=6 DESCRIPTION="D" HOMEPAGE="https://none.none" IUSE="" # this triggers problem in next processed ebuild KEYWORDS="" # this would make the error in 'b' package to go away #KEYWORDS="~amd64" LICENSE="MIT" SLOT="0" $ cat net-im/b/b-0.ebuild # Copyright 1999-2018 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 EAPI=6 DESCRIPTION="D" HOMEPAGE="https://none.none" LICENSE="MIT x86? ( MIT ) amd64? ( MIT )" SLOT="0" KEYWORDS="~x86 ~amd64" IUSE="" $ cat profiles/repo_name repoman-bug Here is how repoman from top-level overlay directory incorrectly flags 'net-im/b' ebuild: $ time repoman full -d RepoMan scours the neighborhood... IUSE.missing [fatal] 1 net-im/b/b-0.ebuild: LICENSE: USE flag 'x86' referenced in conditional 'x86?' is not in IUSE KEYWORDS.missing 1 net-im/a/a-0.ebuild LICENSE.syntax [fatal] 1 net-im/b/b-0.ebuild: LICENSE: USE flag 'x86' referenced in conditional 'x86?' is not in IUSE Note: if I uncomment '#KEYWORDS="~amd64"' in 'net-im/a' the error in 'net-im/b' will go away.
git bisect says this commit introduced the regression: bfe7892202b85c46ff127048ca1cc752c54b186c is the first bad commit commit bfe7892202b85c46ff127048ca1cc752c54b186c Author: Zac Medico <zmedico@gentoo.org> Date: Fri Apr 6 13:05:25 2018 -0700 _pkg_str: add _db attribute (bug 640318) In order to propagate information from dbapi to Package instance, it's useful for each _pkg_str instance to have a _db attribute which references the dbapi instance that instantiated it. Use a new dbapi._iuse_implicit_cnstr method to delegate implicit IUSE logic to the dbapi instance, in order to make the behavior customizable at the dbapi level for the purposes of bug 640318. This patch consists only of refactoring, with no behavioral changes. Bug: https://bugs.gentoo.org/640318 $ git describe bfe7892202b85c46ff127048ca1cc752c54b186c portage-2.3.28-11-gbfe789220
It seems to be only a random coincidence that bfe7892202b85c46ff127048ca1cc752c54b186c triggers your test case, the issue it that that this code instantiates a config instance without any implicit IUSE: https://gitweb.gentoo.org/proj/portage.git/tree/repoman/pym/repoman/profile.py?h=repoman-2.3.9#n70 > # Use an empty profile for checking dependencies of > # packages that have empty KEYWORDS. > prof = ProfileDesc('**', 'stable', '', '') > profiles.setdefault(prof.arch, []).append(prof)
Patch posted for review: https://archives.gentoo.org/gentoo-portage-dev/message/930c818cfcdddfe96f51418ad88c630d https://github.com/gentoo/portage/pull/337
won't it brake the "legal" checks for used-but-undeclared in IUSE flags in 9999 ebuilds?
(In reply to Vadim A. Misbakh-Soloviov (mva) from comment #4) > won't it brake the "legal" checks for used-but-undeclared in IUSE flags in > 9999 ebuilds? Yes, but the check has always been fundamentally flawed, since it used implicit IUSE settings from a random profile. In order to solve this problem, we could create a profile that's tailored specifically for repoman to use when checking ebuilds with empty KEYWORDS. The 'base' profile is not suitable, since it has no USE_EXPAND_VALUES_ARCH setting. The USE_EXPAND_VALUES_ARCH setting is located in the 'base/arch' profile, and that profile is not suitable because it does not inherit the IUSE_IMPLICIT setting from the 'base' profile.
(In reply to Zac Medico from comment #5) > In order to solve this problem, we could create a profile that's tailored > specifically for repoman to use when checking ebuilds with empty KEYWORDS. I suppose it could walk all of the available profile directories from all of the relevant repositories, extract the implicit IUSE variables, and merge them all together into one unified implict IUSE configuration. It should ignore any negative settings, since the intention is have the largest possible set of implicit IUSE flags.
(In reply to Zac Medico from comment #6) > (In reply to Zac Medico from comment #5) > > In order to solve this problem, we could create a profile that's tailored > > specifically for repoman to use when checking ebuilds with empty KEYWORDS. > > I suppose it could walk all of the available profile directories from all of > the relevant repositories, extract the implicit IUSE variables, and merge > them all together into one unified implict IUSE configuration. It should > ignore any negative settings, since the intention is have the largest > possible set of implicit IUSE flags. but wouldn't that be really slow? Exspecially if derived at each run of repoman. Pre-generating it and including it in the tree means that it may be out of sync depending what has been committed since the last generation. I suppose it could be generated on demand stored locally only, Then re-used for subsequent runs before the next sync and re-generation. Developers could add it to repo.postsync.d hooks.
(In reply to Brian Dolbec from comment #7) > but wouldn't that be really slow? Exspecially if derived at each run of > repoman. We'll have to benchmark it, it shouldn't take very long to read and process all of the make.defaults files.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=e691f07bf0572fad7686a54d628c1a29aec4ebe4 commit e691f07bf0572fad7686a54d628c1a29aec4ebe4 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2018-07-12 09:48:32 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2018-07-14 23:31:34 +0000 dbapi: fix repoman implicit IUSE (bug 660982) Account for repoman modifications of the portdbapi self.settings reference, and treat all flags as valid for the empty profile because it does not have any implicit IUSE settings. Bug: https://bugs.gentoo.org/660982 Reviewed-by: Brian Dolbec <dolsen@gentoo.org> pym/_emerge/Package.py | 5 ++++- pym/portage/dbapi/__init__.py | 16 ++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-)
(In reply to Zac Medico from comment #8) > (In reply to Brian Dolbec from comment #7) > > but wouldn't that be really slow? Exspecially if derived at each run of > > repoman. > > We'll have to benchmark it, it shouldn't take very long to read and process > all of the make.defaults files. With this patch it only takes about 0.5 seconds on my slow laptop: https://archives.gentoo.org/gentoo-portage-dev/message/c7698811bb5ac32dd148e0fb167ed3a8 https://github.com/gentoo/portage/pull/342
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=ef604f402a93234a4b6e3ef433678c8fa04c746b commit ef604f402a93234a4b6e3ef433678c8fa04c746b Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2018-07-15 22:51:48 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2018-07-16 06:13:02 +0000 repoman: populate implicit IUSE for empty profile (bug 660982) For the empty profile that's used to check dependencies of ebuilds that have empty KEYWORDS, populate implicit IUSE from all of the make.defaults files found in the relevant repositories (this should take less than 1 second on most hardware). Since the IUSE.missing check cannot be performed without implicit IUSE settings, this makes the IUSE.missing check work for ebuilds with empty KEYWORDS. Bug: https://bugs.gentoo.org/660982 pym/portage/dbapi/__init__.py | 10 ++---- repoman/pym/repoman/modules/scan/depend/profile.py | 36 ++++++++++++++++++++++ 2 files changed, 39 insertions(+), 7 deletions(-)
