Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 919751 - emerge @golang-rebuild re-emerges the just upgraded version of dev-lang/go
Summary: emerge @golang-rebuild re-emerges the just upgraded version of dev-lang/go
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: AMD64 Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 923490 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-12-12 10:32 UTC by Adrian Bassett
Modified: 2024-11-16 12:09 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
0001-cnf-sets-remove-golang-rebuild-set.patch (0001-cnf-sets-remove-golang-rebuild-set.patch,984 bytes, patch)
2024-01-11 16:27 UTC, William Hubbs
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Bassett 2023-12-12 10:32:05 UTC
As per summary 'emerge @golang-rebuild' includes a re-emerge of the just-upgraded version of dev-lang/go

 # emerge -v @golang-rebuild

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 4.62 s (backtrack: 0/20).

[ebuild   R    ] dev-lang/go-1.21.5:0/1.21.5::gentoo  CPU_FLAGS_X86="sse2" 0 KiB
...[other dev-lang/go dependencies]...

Presumably this is un-intentional and should be excluded?  

Reproducible: Always
Comment 1 Alfred Wingate 2023-12-12 11:31:12 UTC
I decided to assign this to portage as this could be weird behaviour with having two sets named same colliding.

dev-lang/go installs golang-rebuild set which would rebuild anything which inherits go related eclasses (wouldn't rebuild dev-lang/go).

BUT portage installs a golang-rebuild set as well which rebuilds anything with dev-lang/go in its *DEPEND (would rebuild dev-lang/go).

So this may be case where the portage installed one is preferred.
Comment 2 William Hubbs gentoo-dev 2023-12-12 19:31:17 UTC
I believe that dev-lang/go should install the set, so if the portage set
is preferred, we need to remove the definition from portage and add it
to dev-lang/go instead. My thinking is you don't need the set if go
isn't installed.

That being said, @golang-rebuild doesn't need to rebuild dev-lang/go.

Thanks,

William
Comment 3 Alfred Wingate 2023-12-12 19:53:10 UTC
The install in portage was the same as in go before.

The rationale for using dev-lang/go in BDEPEND as the rebuild reason is explained here.
https://github.com/gentoo/portage/commit/38c479d46dc91be66877d857a3682534eb1b5f12
Comment 4 Zac Medico gentoo-dev 2023-12-12 20:00:06 UTC
This was triggered by https://github.com/gentoo/portage/pull/1203. We could add an exclude_atoms option for VariableSet, and use it to exclude dev-lang/go:0 from  @golang-rebuild.

The same issue does not trigger with @rust-rebuild because it uses virtual/rust.
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-12-12 20:08:46 UTC
We might be able to do a bit better given the dep is coming from Go itself and just ignore it if it's a self-dependency.
Comment 6 Zac Medico gentoo-dev 2023-12-12 20:29:25 UTC
It would be neat if @golang-rebuild could check for a broken slot operator dep on dev-lang/go:= in BDEPEND, so it would only trigger rebuild when there's a new subslot of dev-lang/go to build against.
Comment 7 William Hubbs gentoo-dev 2023-12-12 21:01:03 UTC
It would, but that would mean that subslots would need to mean something
for BDEPEND.

I think subslots currently only work for RDEPEND. Is this correct? Would
it take an eapi bump to make them work for BDEPEND?
Comment 8 Zac Medico gentoo-dev 2023-12-12 21:48:34 UTC
There was BDEPEND subslot expansion added in the original BDEPEND commit where it calls _eval_deps(deps["BDEPEND"], [running_vardb]):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=02df0addf8f8477590176807d747b4cbb57b7c32

commit 02df0addf8f8477590176807d747b4cbb57b7c32
Author: Michał Górny <mgorny@gentoo.org>
Date:   Fri Mar 16 09:30:41 2018 +0100

    Support BDEPEND (CBUILD dependencies) in EAPI 7


It doesn't trigger slot-operator rebuilds in existing emerge commands because BDEPEND is only pulled in for built packages when --with-bdeps=y is enabled, and it's considered an "optional" dependency in this case which is not strong enough to trigger a rebuild.

There's no reason @golang-rebuild couldn't use it to trigger rebuilds though, with no need for an EAPI bump.
Comment 9 William Hubbs gentoo-dev 2023-12-13 19:04:57 UTC
Hi Zac,

Ok, what do I need to do in the golang-rebuild definition in dev-lang/go
to make this work?
I can update the definition there so you can remove it from portage;
that way we only have the @golang-rebuild set active when go is
installed. There are several other packages in the tree that install set
definitions, so I'm ok with doing this in dev--lang/go.

Thanks,

William
Comment 10 Zac Medico gentoo-dev 2023-12-13 21:20:06 UTC
First, we can fix the set configuration preference issue noted in comment #4 by renaming /usr/share/portage/config/sets/go.conf so that it comes after /usr/share/portage/config/sets/portage.conf. A rename to zzz-go.conf or something like that will work. This shows the current order:

> $ python -c 'import os; print(sorted(os.listdir("/usr/share/portage/config/sets")))'
> ['go.conf', 'portage.conf', 'xorg.conf']

(In reply to William Hubbs from comment #9)
> Ok, what do I need to do in the golang-rebuild definition in dev-lang/go
> to make this work?

First the go-module eclass needs to use a slot operator dependency like >=dev-lang/go-1.18:= so that portage will fill in the slot/subslot in the /var/db/pkg/*/*/BDEPEND value.

> I can update the definition there so you can remove it from portage;
> that way we only have the @golang-rebuild set active when go is
> installed. There are several other packages in the tree that install set
> definitions, so I'm ok with doing this in dev--lang/go.

We'll need a new PackageSet type that is kind of like VariableSet but knows how to check whether or not a slot operator atom like >=dev-lang/go-1.18:0/1.21.4= is satisfied by the installed version of dev-lang/go.

It would only take a few lines of code to make VariableSet do this, so we could possible modify VariableSet to do it or create a class that inherits VariableSet and overrides the _filter method behavior.
Comment 11 Larry the Git Cow gentoo-dev 2024-01-10 20:46:27 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=24228f0576765cb52ed0431bf66e30628f40b3b0

commit 24228f0576765cb52ed0431bf66e30628f40b3b0
Author:     William Hubbs <williamh@gentoo.org>
AuthorDate: 2024-01-10 20:33:40 +0000
Commit:     William Hubbs <williamh@gentoo.org>
CommitDate: 2024-01-10 20:43:30 +0000

    dev-lang/go: add 1.20.13, 1.21.6
    
    Bug: https://bugs.gentoo.org/919751
    Signed-off-by: William Hubbs <williamh@gentoo.org>

 dev-lang/go/Manifest           |   2 +
 dev-lang/go/files/go-sets.conf |   6 +-
 dev-lang/go/go-1.20.13.ebuild  | 210 +++++++++++++++++++++++++++++++++++++++++
 dev-lang/go/go-1.21.6.ebuild   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 425 insertions(+), 3 deletions(-)
Comment 12 William Hubbs gentoo-dev 2024-01-10 20:47:52 UTC
@zmedico:
You can feel free to remove the @golang-rebuild definition from
sys-apps/portage; it is now defined the same way in dev-lang/go.

Thanks,

William
Comment 13 William Hubbs gentoo-dev 2024-01-11 16:27:37 UTC
Created attachment 881989 [details, diff]
0001-cnf-sets-remove-golang-rebuild-set.patch

This patch removes the golang-rebuild set from portage.

Thanks,

William
Comment 14 Larry the Git Cow gentoo-dev 2024-01-11 17:50:12 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=84a16b558bf7d371434fe401373d697672ba9881

commit 84a16b558bf7d371434fe401373d697672ba9881
Author:     William Hubbs <williamh@gentoo.org>
AuthorDate: 2024-01-11 17:49:10 +0000
Commit:     William Hubbs <williamh@gentoo.org>
CommitDate: 2024-01-11 17:49:10 +0000

    go-module.eclass: update go version in BDEPEND and add subslot
    
    Bug: https://bugs.gentoo.org/919751
    Signed-off-by: William Hubbs <williamh@gentoo.org>

 eclass/go-module.eclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-02-01 20:49:05 UTC
*** Bug 923490 has been marked as a duplicate of this bug. ***
Comment 16 Gary E. Miller 2024-02-01 21:40:57 UTC
"unconfirmed" ??
Comment 17 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-02-01 21:42:54 UTC
(In reply to Gary E. Miller from comment #16)
> "unconfirmed" ??

We don't really use UNCONFIRMED vs CONFIRMED, but if it makes people happy, sure.
Comment 18 Gary E. Miller 2024-02-01 21:47:43 UTC
Isn't it your job Sam to make me happy?  😀😀😀

Thanks, as always!
Comment 19 Scott Howard 2024-02-05 05:48:20 UTC
As a user, it took me a while to figure out what sets.conf should say, so I'm sharing it here... I added 4 lines to my /etc/portage/sets.conf in order to work around the issue:

[golang-rebuild-better]
class = portage.sets.dbapi.VariableSet
variable = INHERITED
includes = golang-base golang-build golang-vcs golang-vcs-snapshot go-module

For now I can use emerge -1 @golang-rebuild-better to rebuild go dependencies after I upgrade go, without re-rebuilding go.
Comment 20 Zac Medico gentoo-dev 2024-02-25 23:02:58 UTC
(In reply to William Hubbs from comment #13)
> Created attachment 881989 [details, diff] [details, diff]
> 0001-cnf-sets-remove-golang-rebuild-set.patch
> 
> This patch removes the golang-rebuild set from portage.
> 
> Thanks,
> 
> William

In https://github.com/gentoo/portage/pull/1286 I've also added a local @golang-rebuild set in lib/portage/tests/sets/base/test_variable_set.py so the corresponding test case will still work.
Comment 21 Larry the Git Cow gentoo-dev 2024-02-27 02:27:23 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=1d3e3843f2a51c581d344540c5c6ee266afa30d2

commit 1d3e3843f2a51c581d344540c5c6ee266afa30d2
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-02-25 22:57:43 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-02-26 23:17:55 +0000

    cnf: sets: Migrate @golang-rebuild to dev-lang/go
    
    Bug: https://bugs.gentoo.org/919751
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 cnf/sets/portage.conf                            |  6 ------
 lib/portage/tests/resolver/ResolverPlayground.py | 10 +++++++++-
 lib/portage/tests/sets/base/test_variable_set.py |  8 ++++++--
 3 files changed, 15 insertions(+), 9 deletions(-)
Comment 22 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-06-15 10:25:19 UTC
FWIW, I do plan on getting back to this, it's just been lower priority compared to various other stuff I've been juggling. I'm aware it's annoying for users. Thanks!
Comment 23 Gary E. Miller 2024-06-15 19:04:00 UTC
(In reply to Sam James from comment #22)
> FWIW, I do plan on getting back to this, it's just been lower priority
> compared to various other stuff I've been juggling. I'm aware it's annoying
> for users. Thanks!

I'm in no hurry.  This is hardly an annoyance, unless you watch your emerges in progress.