Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 919751 - emerge @golang-rebuild re-emerges the just upgraded version of dev-lang/go
Summary: emerge @golang-rebuild re-emerges the just upgraded version of dev-lang/go
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 923490 (view as bug list)
Depends on:
Blocks: 930802
  Show dependency tree
 
Reported: 2023-12-12 10:32 UTC by Adrian Bassett
Modified: 2024-04-28 00:32 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
0001-cnf-sets-remove-golang-rebuild-set.patch (0001-cnf-sets-remove-golang-rebuild-set.patch,984 bytes, patch)
2024-01-11 16:27 UTC, William Hubbs
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Bassett 2023-12-12 10:32:05 UTC
As per summary 'emerge @golang-rebuild' includes a re-emerge of the just-upgraded version of dev-lang/go

 # emerge -v @golang-rebuild

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 4.62 s (backtrack: 0/20).

[ebuild   R    ] dev-lang/go-1.21.5:0/1.21.5::gentoo  CPU_FLAGS_X86="sse2" 0 KiB
...[other dev-lang/go dependencies]...

Presumably this is un-intentional and should be excluded?  

Reproducible: Always
Comment 1 Alfred Wingate 2023-12-12 11:31:12 UTC
I decided to assign this to portage as this could be weird behaviour with having two sets named same colliding.

dev-lang/go installs golang-rebuild set which would rebuild anything which inherits go related eclasses (wouldn't rebuild dev-lang/go).

BUT portage installs a golang-rebuild set as well which rebuilds anything with dev-lang/go in its *DEPEND (would rebuild dev-lang/go).

So this may be case where the portage installed one is preferred.
Comment 2 William Hubbs gentoo-dev 2023-12-12 19:31:17 UTC
I believe that dev-lang/go should install the set, so if the portage set
is preferred, we need to remove the definition from portage and add it
to dev-lang/go instead. My thinking is you don't need the set if go
isn't installed.

That being said, @golang-rebuild doesn't need to rebuild dev-lang/go.

Thanks,

William
Comment 3 Alfred Wingate 2023-12-12 19:53:10 UTC
The install in portage was the same as in go before.

The rationale for using dev-lang/go in BDEPEND as the rebuild reason is explained here.
https://github.com/gentoo/portage/commit/38c479d46dc91be66877d857a3682534eb1b5f12
Comment 4 Zac Medico gentoo-dev 2023-12-12 20:00:06 UTC
This was triggered by https://github.com/gentoo/portage/pull/1203. We could add an exclude_atoms option for VariableSet, and use it to exclude dev-lang/go:0 from  @golang-rebuild.

The same issue does not trigger with @rust-rebuild because it uses virtual/rust.
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-12-12 20:08:46 UTC
We might be able to do a bit better given the dep is coming from Go itself and just ignore it if it's a self-dependency.
Comment 6 Zac Medico gentoo-dev 2023-12-12 20:29:25 UTC
It would be neat if @golang-rebuild could check for a broken slot operator dep on dev-lang/go:= in BDEPEND, so it would only trigger rebuild when there's a new subslot of dev-lang/go to build against.
Comment 7 William Hubbs gentoo-dev 2023-12-12 21:01:03 UTC
It would, but that would mean that subslots would need to mean something
for BDEPEND.

I think subslots currently only work for RDEPEND. Is this correct? Would
it take an eapi bump to make them work for BDEPEND?
Comment 8 Zac Medico gentoo-dev 2023-12-12 21:48:34 UTC
There was BDEPEND subslot expansion added in the original BDEPEND commit where it calls _eval_deps(deps["BDEPEND"], [running_vardb]):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=02df0addf8f8477590176807d747b4cbb57b7c32

commit 02df0addf8f8477590176807d747b4cbb57b7c32
Author: Michał Górny <mgorny@gentoo.org>
Date:   Fri Mar 16 09:30:41 2018 +0100

    Support BDEPEND (CBUILD dependencies) in EAPI 7


It doesn't trigger slot-operator rebuilds in existing emerge commands because BDEPEND is only pulled in for built packages when --with-bdeps=y is enabled, and it's considered an "optional" dependency in this case which is not strong enough to trigger a rebuild.

There's no reason @golang-rebuild couldn't use it to trigger rebuilds though, with no need for an EAPI bump.
Comment 9 William Hubbs gentoo-dev 2023-12-13 19:04:57 UTC
Hi Zac,

Ok, what do I need to do in the golang-rebuild definition in dev-lang/go
to make this work?
I can update the definition there so you can remove it from portage;
that way we only have the @golang-rebuild set active when go is
installed. There are several other packages in the tree that install set
definitions, so I'm ok with doing this in dev--lang/go.

Thanks,

William
Comment 10 Zac Medico gentoo-dev 2023-12-13 21:20:06 UTC
First, we can fix the set configuration preference issue noted in comment #4 by renaming /usr/share/portage/config/sets/go.conf so that it comes after /usr/share/portage/config/sets/portage.conf. A rename to zzz-go.conf or something like that will work. This shows the current order:

> $ python -c 'import os; print(sorted(os.listdir("/usr/share/portage/config/sets")))'
> ['go.conf', 'portage.conf', 'xorg.conf']

(In reply to William Hubbs from comment #9)
> Ok, what do I need to do in the golang-rebuild definition in dev-lang/go
> to make this work?

First the go-module eclass needs to use a slot operator dependency like >=dev-lang/go-1.18:= so that portage will fill in the slot/subslot in the /var/db/pkg/*/*/BDEPEND value.

> I can update the definition there so you can remove it from portage;
> that way we only have the @golang-rebuild set active when go is
> installed. There are several other packages in the tree that install set
> definitions, so I'm ok with doing this in dev--lang/go.

We'll need a new PackageSet type that is kind of like VariableSet but knows how to check whether or not a slot operator atom like >=dev-lang/go-1.18:0/1.21.4= is satisfied by the installed version of dev-lang/go.

It would only take a few lines of code to make VariableSet do this, so we could possible modify VariableSet to do it or create a class that inherits VariableSet and overrides the _filter method behavior.
Comment 11 Larry the Git Cow gentoo-dev 2024-01-10 20:46:27 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=24228f0576765cb52ed0431bf66e30628f40b3b0

commit 24228f0576765cb52ed0431bf66e30628f40b3b0
Author:     William Hubbs <williamh@gentoo.org>
AuthorDate: 2024-01-10 20:33:40 +0000
Commit:     William Hubbs <williamh@gentoo.org>
CommitDate: 2024-01-10 20:43:30 +0000

    dev-lang/go: add 1.20.13, 1.21.6
    
    Bug: https://bugs.gentoo.org/919751
    Signed-off-by: William Hubbs <williamh@gentoo.org>

 dev-lang/go/Manifest           |   2 +
 dev-lang/go/files/go-sets.conf |   6 +-
 dev-lang/go/go-1.20.13.ebuild  | 210 +++++++++++++++++++++++++++++++++++++++++
 dev-lang/go/go-1.21.6.ebuild   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 425 insertions(+), 3 deletions(-)
Comment 12 William Hubbs gentoo-dev 2024-01-10 20:47:52 UTC
@zmedico:
You can feel free to remove the @golang-rebuild definition from
sys-apps/portage; it is now defined the same way in dev-lang/go.

Thanks,

William
Comment 13 William Hubbs gentoo-dev 2024-01-11 16:27:37 UTC
Created attachment 881989 [details, diff]
0001-cnf-sets-remove-golang-rebuild-set.patch

This patch removes the golang-rebuild set from portage.

Thanks,

William
Comment 14 Larry the Git Cow gentoo-dev 2024-01-11 17:50:12 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=84a16b558bf7d371434fe401373d697672ba9881

commit 84a16b558bf7d371434fe401373d697672ba9881
Author:     William Hubbs <williamh@gentoo.org>
AuthorDate: 2024-01-11 17:49:10 +0000
Commit:     William Hubbs <williamh@gentoo.org>
CommitDate: 2024-01-11 17:49:10 +0000

    go-module.eclass: update go version in BDEPEND and add subslot
    
    Bug: https://bugs.gentoo.org/919751
    Signed-off-by: William Hubbs <williamh@gentoo.org>

 eclass/go-module.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-02-01 20:49:05 UTC
*** Bug 923490 has been marked as a duplicate of this bug. ***
Comment 16 Gary E. Miller 2024-02-01 21:40:57 UTC
"unconfirmed" ??
Comment 17 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-02-01 21:42:54 UTC
(In reply to Gary E. Miller from comment #16)
> "unconfirmed" ??

We don't really use UNCONFIRMED vs CONFIRMED, but if it makes people happy, sure.
Comment 18 Gary E. Miller 2024-02-01 21:47:43 UTC
Isn't it your job Sam to make me happy?  😀😀😀

Thanks, as always!
Comment 19 Scott Howard 2024-02-05 05:48:20 UTC
As a user, it took me a while to figure out what sets.conf should say, so I'm sharing it here... I added 4 lines to my /etc/portage/sets.conf in order to work around the issue:

[golang-rebuild-better]
class = portage.sets.dbapi.VariableSet
variable = INHERITED
includes = golang-base golang-build golang-vcs golang-vcs-snapshot go-module

For now I can use emerge -1 @golang-rebuild-better to rebuild go dependencies after I upgrade go, without re-rebuilding go.
Comment 20 Zac Medico gentoo-dev 2024-02-25 23:02:58 UTC
(In reply to William Hubbs from comment #13)
> Created attachment 881989 [details, diff] [details, diff]
> 0001-cnf-sets-remove-golang-rebuild-set.patch
> 
> This patch removes the golang-rebuild set from portage.
> 
> Thanks,
> 
> William

In https://github.com/gentoo/portage/pull/1286 I've also added a local @golang-rebuild set in lib/portage/tests/sets/base/test_variable_set.py so the corresponding test case will still work.
Comment 21 Larry the Git Cow gentoo-dev 2024-02-27 02:27:23 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=1d3e3843f2a51c581d344540c5c6ee266afa30d2

commit 1d3e3843f2a51c581d344540c5c6ee266afa30d2
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-25 22:57:43 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-26 23:17:55 +0000

    cnf: sets: Migrate @golang-rebuild to dev-lang/go
    
    Bug: https://bugs.gentoo.org/919751
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 cnf/sets/portage.conf                            |  6 ------
 lib/portage/tests/resolver/ResolverPlayground.py | 10 +++++++++-
 lib/portage/tests/sets/base/test_variable_set.py |  8 ++++++--
 3 files changed, 15 insertions(+), 9 deletions(-)