After dumping the effective values: default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+/make.defaults:ARCH="amd64" default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+/make.defaults:ARCH="amd64" default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+/make.defaults:ARCH="amd64" default/linux/arm/17.0/armv7a/prefix/kernel-3.2+/make.defaults:ARCH="arm" default/linux/arm64/17.0/prefix/kernel-3.2+/make.defaults:ARCH="arm64" default/linux/x86/17.0/prefix/kernel-2.6.16+/make.defaults:ARCH="x86" default/linux/x86/17.0/prefix/kernel-2.6.32+/make.defaults:ARCH="x86" default/linux/x86/17.0/prefix/kernel-3.2+/make.defaults:ARCH="x86" However, profiles.desc state: amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+ dev amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+ exp amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+ exp x86-linux default/linux/x86/17.0/prefix/kernel-3.2+ exp x86-linux default/linux/x86/17.0/prefix/kernel-2.6.32+ exp x86-linux default/linux/x86/17.0/prefix/kernel-2.6.16+ exp arm-linux default/linux/arm/17.0/armv7a/prefix/kernel-3.2+ exp arm64-linux default/linux/arm64/17.0/prefix/kernel-3.2+ exp Not sure if this is technically forbidden but it is confusing as hell to developers (why does pkgcheck complain about ~amd64 keyword on a prefix profile?!).
Are you resolving parents in these paths or something? I don't have a default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+/make.defaults Anyway, this was done because of compatibility. Things like `use amd64`, and also the ACCEPT_KEYWORDS is set based on this (~${ARCH}). The former was used frequently a couple of years ago, when support for amd64 was introduced, and we didn't want to turn all of those into or-ed conditions.
(In reply to Michał Górny from comment #0) > After dumping the effective values: > > default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+/make.defaults: > ARCH="amd64" > default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+/make.defaults: > ARCH="amd64" > default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+/make.defaults: > ARCH="amd64" > default/linux/arm/17.0/armv7a/prefix/kernel-3.2+/make.defaults:ARCH="arm" > default/linux/arm64/17.0/prefix/kernel-3.2+/make.defaults:ARCH="arm64" > default/linux/x86/17.0/prefix/kernel-2.6.16+/make.defaults:ARCH="x86" > default/linux/x86/17.0/prefix/kernel-2.6.32+/make.defaults:ARCH="x86" > default/linux/x86/17.0/prefix/kernel-3.2+/make.defaults:ARCH="x86" > > > However, profiles.desc state: > > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+ dev > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+ exp > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-3.2+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-2.6.32+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-2.6.16+ exp > arm-linux default/linux/arm/17.0/armv7a/prefix/kernel-3.2+ exp > arm64-linux default/linux/arm64/17.0/prefix/kernel-3.2+ exp > > > > Not sure if this is technically forbidden but it is confusing as hell to > developers (why does pkgcheck complain about ~amd64 keyword on a prefix > profile?!). This has been a pain for us. That's your suggestion? We want to distinguish Prefix and vanilla as the profiles cannot be interchanged, but at the same time the keywords should be kept the same.
(In reply to Michał Górny from comment #0) > After dumping the effective values: > > default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+/make.defaults: > ARCH="amd64" > default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+/make.defaults: > ARCH="amd64" > default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+/make.defaults: > ARCH="amd64" > default/linux/arm/17.0/armv7a/prefix/kernel-3.2+/make.defaults:ARCH="arm" > default/linux/arm64/17.0/prefix/kernel-3.2+/make.defaults:ARCH="arm64" > default/linux/x86/17.0/prefix/kernel-2.6.16+/make.defaults:ARCH="x86" > default/linux/x86/17.0/prefix/kernel-2.6.32+/make.defaults:ARCH="x86" > default/linux/x86/17.0/prefix/kernel-3.2+/make.defaults:ARCH="x86" > > > However, profiles.desc state: > > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+ dev > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+ exp > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-3.2+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-2.6.32+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-2.6.16+ exp > arm-linux default/linux/arm/17.0/armv7a/prefix/kernel-3.2+ exp > arm64-linux default/linux/arm64/17.0/prefix/kernel-3.2+ exp > > > > Not sure if this is technically forbidden but it is confusing as hell to > developers (why does pkgcheck complain about ~amd64 keyword on a prefix > profile?!). This has been a pain for us. What's your suggestion? We want to distinguish Prefix and vanilla as the profiles cannot be interchanged, but at the same time the keywords should be kept the same.
I am somewhat aware that the historical distinction has been painful. It has also happened to FreeBSD keywords back in the day (e.g. when I add to duplicate all amd64 conditionals to amd64_fbsd). My primary question is: does that mean that we no longer do keywording for amd64-linux prefix? If so, I think it would be nice to make it more explicit somewhere. I don't know whether this is really wrong or not, and I don't have any better ideas. My main concern is that some tools may be making assumptions based on this, and it could trigger undefined behavior (like, when some tools look at $ARCH and some look at profiles.desc arch, and you get surprising results when they are mismatched).
Thinking about it more, profiles.desc probably doesn't affect normal Portage operation. After all, eselect-profile makes make.profile symlink and Portage uses just that. So this probably affects only repoman/pkgcheck. @dev-portage, @radhermit, do you happen to know offhand if and where Portage/repoman and pkgcore/pkgcheck use arch value from profiles.desc?
from a quick grep, looks like : portage/repoman/lib/repoman/profile.py
In repoman, this _gen_arches function is used to generate relevant arches for an ebuild, which are later intersected with the profiles.desc arch values: https://gitweb.gentoo.org/proj/portage.git/tree/repoman/lib/repoman/modules/scan/depend/_gen_arches.py?h=repoman-2.3.21
(In reply to Zac Medico from comment #7) > In repoman, this _gen_arches function is used to generate relevant arches > for an ebuild, which are later intersected with the profiles.desc arch > values: > > https://gitweb.gentoo.org/proj/portage.git/tree/repoman/lib/repoman/modules/ > scan/depend/_gen_arches.py?h=repoman-2.3.21 Could you explain what implications does this have? Are Prefix profiles checked correctly when ebuilds have ~amd64 keyword? Or when they have ~amd64-linux keyword?
(In reply to Michał Górny from comment #8) > (In reply to Zac Medico from comment #7) > > In repoman, this _gen_arches function is used to generate relevant arches > > for an ebuild, which are later intersected with the profiles.desc arch > > values: > > > > https://gitweb.gentoo.org/proj/portage.git/tree/repoman/lib/repoman/modules/ > > scan/depend/_gen_arches.py?h=repoman-2.3.21 > > Could you explain what implications does this have? Are Prefix profiles > checked correctly when ebuilds have ~amd64 keyword? Or when they have > ~amd64-linux keyword? The ARCH value from the profile is never considered. For an ebuild with ~amd64 keyword, it generates a hypothetical ACCEPT_KEYWORDS="amd64 ~amd64" setting that it uses to resolve the ebuild's dependencies for all profiles that list amd64 as their arch in profiles.desc. The behavior for an ebuild with ~amd64-linux keyword is analogous to the behavior for ~amd64 describe above. It uses ACCEPT_KEYWORDS="amd64-linux ~amd64-linux" for all profiles that list amd64-linux as their arch in profiles.desc.
(In reply to Zac Medico from comment #9) > The ARCH value from the profile is never considered. Actually, the arch value is considered in the sense that it's added to use.force here: https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/dep/dep_check.py?h=portage-2.3.99#n775
(In reply to Michał Górny from comment #0) > After dumping the effective values: > > default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+/make.defaults: > ARCH="amd64" > default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+/make.defaults: > ARCH="amd64" > default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+/make.defaults: > ARCH="amd64" > default/linux/arm/17.0/armv7a/prefix/kernel-3.2+/make.defaults:ARCH="arm" > default/linux/arm64/17.0/prefix/kernel-3.2+/make.defaults:ARCH="arm64" > default/linux/x86/17.0/prefix/kernel-2.6.16+/make.defaults:ARCH="x86" > default/linux/x86/17.0/prefix/kernel-2.6.32+/make.defaults:ARCH="x86" > default/linux/x86/17.0/prefix/kernel-3.2+/make.defaults:ARCH="x86" > > > However, profiles.desc state: > > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-3.2+ dev > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.32+ exp > amd64-linux default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-3.2+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-2.6.32+ exp > x86-linux default/linux/x86/17.0/prefix/kernel-2.6.16+ exp > arm-linux default/linux/arm/17.0/armv7a/prefix/kernel-3.2+ exp > arm64-linux default/linux/arm64/17.0/prefix/kernel-3.2+ exp > > > > Not sure if this is technically forbidden but it is confusing as hell to > developers (why does pkgcheck complain about ~amd64 keyword on a prefix > profile?!). After some serious thoughts, I think we can tag these profiles as amd64, x86, arm and arm64. All the problems solved. > We want to distinguish Prefix and vanilla as the profiles cannot be interchanged, but at the same time the keywords should be kept the same. Previous I did not think it a good idea. But hey, multilib and non-multilib profiles are not interchangeable, nor do hardened and vanilla. What do you think?
(In reply to Benda Xu from comment #11) > > Previous I did not think it a good idea. But hey, multilib and non-multilib > profiles are not interchangeable, nor do hardened and vanilla. > > What do you think? As long as its clear from the profile path to both users and tools that the profile is intended for prefix use, then it should be fine. Especially given they're tagged "(exp)" so users will get a stern warning when they attempt to use it, or may not even see it listed. What's holding this back from happening?
If I interpret this correctly, it implicates that we drop all *-linux keywords. I have nothing really against this, but it will implicate a lot of packages suddenly being "available", yet not working at all. It seems impractical to mask all packages currently not keyworded, and add masks for any new package entering the tree.
(In reply to Fabian Groffen from comment #13) > If I interpret this correctly, it implicates that we drop all *-linux > keywords. > > I have nothing really against this, but it will implicate a lot of packages > suddenly being "available", yet not working at all. It seems impractical to > mask all packages currently not keyworded, and add masks for any new package > entering the tree. We are already doing that by landing amd64 prefix to unstable profiles from exp. So far so good. In the age of EAPI 7, most of the packages work out of the box for Prefix.
(In reply to Benda Xu from comment #14) > (In reply to Fabian Groffen from comment #13) > > If I interpret this correctly, it implicates that we drop all *-linux > > keywords. > > > > I have nothing really against this, but it will implicate a lot of packages > > suddenly being "available", yet not working at all. It seems impractical to > > mask all packages currently not keyworded, and add masks for any new package > > entering the tree. > > We are already doing that by landing amd64 prefix to unstable profiles from > exp. So far so good. In the age of EAPI 7, most of the packages work out > of the box for Prefix. I guess the main downside of "sharing a keyword with prefix", is well, on non-prefixed systems, the keyword being set indicates "somebody tested this combo". Exposing this to prefix users gives them a little deception, because just because its keyworded ~amd64, it doesn't mean somebody ever got it working on prefix. I'm not sure I like that outcome, but like, I care more about the graph being consistent. If I can hold my end up of that bargain, whether or not prefix actually works is prefix's problem :)
(In reply to Benda Xu from comment #14) > (In reply to Fabian Groffen from comment #13) > > If I interpret this correctly, it implicates that we drop all *-linux > > keywords. > > > > I have nothing really against this, but it will implicate a lot of packages > > suddenly being "available", yet not working at all. It seems impractical to > > mask all packages currently not keyworded, and add masks for any new package > > entering the tree. > > We are already doing that by landing amd64 prefix to unstable profiles from > exp. So far so good. In the age of EAPI 7, most of the packages work out > of the box for Prefix. Yes, I can also say that at least on prefix-standalone, it's becoming quite rare to find packages that don't work on prefix out of the box. I have installations with 1000+ packages and I think dropping *-linux keywords and reusing the regular keywords should be just fine. However, it would be great if we could somehow make at least bootstrapping more stable by using packages with stable keywords only at that stage, and ensuring those work for bootstrapping.
Has the prefix team decided how they want to handle this yet? The change that makes pkgcheck ignore broken *-linux prefix keywords for visibility checks will probably be reverted in the next release leading to CI exploding with NonsolvableDepsInDev results.
I think the dominating thought is to stop using the *-linux variants in favour of the non-Prefix keywords. It seems I'm the only one who's in doubt about whether this is a good idea or not, but it would greatly relieve keyword stress on "myself", e.g. not to care about *-linux any more, so that's kind of balancing the scales for me. I don't quite fancy doing the work though. I think we could certainly start with adapting the profiles, switching the keywords, and let someone figure out how to explain to ordinary Gentoo Linux users they shouldn't try and select the prefix profiles from eselect profile list. In a way they already figure out switching e.g. to musl is a bad idea, so perhaps this is a non-issue. I'm hoping we can phase out the keywords from the tree gradually though, or someone else to script this up and take the risk for it.
Ping. This is making pkgcheck unusable. If you can't manage to finally fix it, I'm going to mark Prefix profiles exp in order to silence the spam.
I think the RAP profiles want to use ~amd64 keyword, while the rpath-profiles want to use ~amd64-linux keywords at this point. The latter is exp AFAICT, and I will put it on my TODO list to see if I can safely switch ARCH to amd64-linux there. (The answer is likely yes, would satisfy PMS reqs then IMO.) For the RAP profiles, Benda really should make a decision and act accordingly. General remark, rpath-profiles are NOT to be removed/deprecated. They are actively in use.
From my own experience running repoman and pkgcheck, the problem is that in rpath prefix, the main repository is a combination of prefix local packages and packages in ::gentoo. That means that ::gentoo effectively acts like an overlay for ::gentoo_prefix. However, that is a problem when a package in ::gentoo keyworded as e.g. ~amd64-linux relies on dependencies that exist only in ::gentoo_prefix. The solution is to either merge everything in ::gentoo_prefix back into ::gentoo or make the tools ignore these errors as was done before. If the prefix rpath profiles can be marked exp while keeping prefix standalone profiles as dev, that's probably also just fine, since standalone prefix uses regular keywords and shouldn't cause any missing dep warnings. But long term, I think that merging ::gentoo_prefix and ::gentoo would be best, or at least making ::gentoo_prefix a normal overlay to ::gentoo and not the other way around.
What you say is already the case: - all Prefix profiles are 'exp', except the few RAP profiles being 'dev' - migrations from the prefix-overlay to the main tree are ongoing, it is just a slow process
Indeed, for some reason I thought that rpath prefix on linux was also marked dev along with RAP, probably because the warnings are for bad dependency in dev and with keywords as ~amd64-linux... I wonder why RAP triggers such warnings then. I see these warnings also for packages with optional dependencies that are not keyworded for prefix, while the main package is, like packages that optionally depend on Qt. In any case, if we can simply supress warnings for *-linux keywords rather than move the profiles back to exp, that would be better in my opinion.
I will migrate the prefix-standalone profiles to the ${ARCH} keyword.
Ping.
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=b75855e68ea743a6eea54d0b7ad947ed567de6ae is also masking live ebuild... :(
(In reply to Thomas Deutschmann from comment #26) > https://gitweb.gentoo.org/repo/gentoo.git/commit/ > ?id=b75855e68ea743a6eea54d0b7ad947ed567de6ae is also masking live ebuild... > :( While I don't agree with the mask, I did request mgorny to not include live ebuilds in the mask so someone should change it to only mask releases. People using live ebuilds should know how to handle themselves without masks being required.
The bug is open since May last year, let's call it a timeout and unleash the hell. I suppose marking the profiles exp won't harm existing users, won't it?
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fe71f903d5afafcaa77619a96a550f3f817cdede commit fe71f903d5afafcaa77619a96a550f3f817cdede Author: Michał Górny <mgorny@gentoo.org> AuthorDate: 2021-01-31 21:42:06 +0000 Commit: Michał Górny <mgorny@gentoo.org> CommitDate: 2021-01-31 22:32:51 +0000 profiles.desc: Mark prefix profiles exp due to broken $ARCH The problem with inconsistent ${ARCH} value has not been resolved since 2020-05-01 and is resulting in a large number of false positives with pkgcheck. Switch the two Linux Standalone dev profiles (kernel-3.2+) to exp to hide them from normal output. Bug: https://bugs.gentoo.org/720224 Signed-off-by: Michał Górny <mgorny@gentoo.org> profiles/profiles.desc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
(In reply to Michał Górny from comment #28) > The bug is open since May last year, let's call it a timeout and unleash the > hell. I suppose marking the profiles exp won't harm existing users, won't > it? Sorry for not acting in time. Will process it ASAP.
