Go 1.18.5 was released and widely reported as a suggested update, due to critical security bugs it fixed. There's a pkg_postinst() notice in dev-lang/go warning folks to run @golang-rebuild to address this. There are a couple of issues with golang-rebuild, but this bug is really only about one of them: 1. Because @golang-rebuild only looks at what packages report themselves as depending on go, it will always report those packages need to be rebuilt. It would be nice if @golang-rebuild only reported packages that differ from the currently-installed version of go, so it would be possible to blindly run it and get "the right thing." (You could always have a --force flag to force a rebuild of all go packages, of course.) 2. The more serious issue is: trusting package maintainers to properly set BDEPEND; I wrote some quick and dirty shell code to find all the go binaries on my system and validate that they were built with the current version of go installed; much to my surprise, this could found that sys-libs/libpcap-2.64 has a go binary in it (/sbin/captree), and that package is not reported as needing to be rebuilt by @golang-rebuild. Given that 1.18.5 was relevant in a security context, @golang-rebuild should catch these things, instead of trusting package maintainers to add the proper BDEPEND. (Alternatively, I could see that a QA step on ebuilds being added to fail ebuilds that produce go binaries, but don't list it as a BDEPEND.) I'm listing the severity as major, due to the security implications, but I'm not sure based on Gentoo community standards whether that's appropriate. This may be related to bug 827974, I'm not sure. I'm happy to share the shell code I wrote to validate this... but not sure if it'd be useful. Reproducible: Always Actual Results: underworld /usr/portage/dev-lang/go # emerge -p @golang-rebuild These are the packages that would be merged, in order: Calculating dependencies ... done! [ebuild R ] dev-go/go-md2man-2.0.0 [ebuild R ] app-containers/docker-proxy-0.8.0_p20210525 [ebuild R ] app-containers/docker-cli-20.10.12 [ebuild R ] app-containers/runc-1.0.3 [ebuild R ] app-containers/containerd-1.5.11 [ebuild R ] app-containers/docker-20.10.12-r1 Expected Results: Given this: [preed@underworld ~]$ file /sbin/captree /sbin/captree: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=2K7T6Kv2VG8PaATZVZfa/lLLdSX7kLZAXRbJ57Gfa/_rNL91Z8MhXGSlkr1gih/0eCT2f4S3U2yQ-ExI-W-, stripped [preed@underworld ~]$ go version /sbin/captree /sbin/captree: go1.18.2 [preed@underworld ~]$ equery b /sbin/captree * Searching for /sbin/captree ... sys-libs/libcap-2.64 (/sbin/captree) I would expect to see libpcap listed in the above output. It's not.
The set definitely should catch everything that depends on dev-lang/go. That's what's covered in bug 827974. As for libcap, that seems to be because the ebuild only has: BDEPEND="tools? ( dev-lang/go )" instead of inheriting the eclass, so it doesn't get added to the @golang-rebuild set (which is indeed the same issue from the other bug again).
(In reply to John Helmert III from comment #1) > The set definitely should catch everything that depends on dev-lang/go. > That's what's covered in bug 827974. Gotcha. Well, let me know if you want to see the code I used to fix this; as I said, it uses the Go tooling to assert the dependency, so it doesn't rely on other ebuild maintainers to always get it right. (It also does the "smart" thing wrt also suggesting packages that aren't built with the current Go compiler, not all packages declaring a Go dependency.) I see you're active over in bug 827974 too, so feel free to declare this a duplicate if you like.
(In reply to J. Paul Reed from comment #2) > I see you're active over in bug 827974 too, so feel free to declare this a > duplicate if you like. I'll defer to the portage maintainers ;)
On point 2., libcap does depend on Go with USE=tools, and that's the package which installs captree. If you know of (actual) missing dependencies, you should file bugs for them.
And yes -- I'd love to see your code, thanks for filing this!
Created attachment 800299 [details] golang_rebuild_check.sh (In reply to Sam James from comment #5) > And yes -- I'd love to see your code, thanks for filing this! Here ya go! :-) This is in shell, obviously, but I'd be happy to port it to Python and submit it as a patch if you point me at where that should live, roughly.
(In reply to Sam James from comment #4) > On point 2., libcap does depend on Go with USE=tools, and that's the package > which installs captree. If you know of (actual) missing dependencies, you > should file bugs for them. FWIW, USE=tools is set in my make.conf, and yet @golang-rebuild doesn't catch libpcap; that may be a (non-libpcap ebuild) bug? [preed@underworld systools]$ emerge --info | grep tools USE="24bpp 256-color X Xaw3d a52 aac aacs acl acpi aio alac alsa amd64 apache2 apng archive aspell bash-completion branding btrfs bundled-libs bzip2 cache cairo caps cdda cddb cdio cdparanoia cdr center-tilde cg cgi clang cli client clipboard cmake colordiff compat contrib cpudetection cron crypt cryptsetup cscope css curl cvs dbus dga dri dts dvd dvdr ecc egl eglfs elogind enchant encode examples exceptions exif expat extensions extras faac faad fat fbcon fdk ffmpeg fftw flac flatfile float fontconfig fonts fortran fortune ftp games gconf gcrypt gd gdbm geoip geoip2 gif gimp git gles gles1 gles2 glib gmp gnome gnome-keyring gnutls go gpg gphoto2 gps grammar graphite gstreamer gtk gtk3 gui gzip heif http hunspell i18n iconv icu id3tag idn inotify intl ipc ipv6 irc ithreads javascript jemalloc jit jpeg jpeg2k json kmod lame latex lcms libcaca libcanberra libffi libglvnd libinput libnotify libsamplerate libtirpc libxml2 lock logrotate logwatch lto lua lz4 lzma lzo mad magic mbox mercurial mhash mime minizip mjpeg mmap mms mng mp3 mp4 mpeg mplayer mpx mta multilib musepack mysql mysqli nas ncurses netpbm network nls nptl nsplugin ntp nvidia objc objc++ objc-gc offensive ogg ogg123 oniguruma opengl openmp openssl optimize opus otf otr pam pango parse-clocks pcap pcf-8bit pcf-unicode pci pcre pcre16 pcre32 pdf pdo perl pgo pkcs11 plugins png pnm policykit posix postfix postproc postscript printsupport privsep python qml qt5 quicktime rar raw rcs rdp readline reencrypt regex reiserfs replaygain rpc rpm rtc rtf s3 screensaver sdl sdlgfx seccomp server sftp sha512 shadow sharedmem shm simplexml smp smtp sna sndfile sound sox speex spell split-usr sqlite ssh ssh-agent ssl startup-notification subversion suexec svg syslog system-boost system-ffmpeg system-jpeg system-libs system-sqlite systray taglib tcl tcpdump threads thunar tiff timezone tk tools truetype ...
The attached function appears to call "go version" on a hard-coded list of directories (/usr/bin /bin /sbin). That's problematic: go binaries could be installed anywhere. I'm not sure it makes sense to implement this as a portage set. All existing sets utilize package metadata to build a list of packages to be rebuilt. None of them call external commands to scan installed files. If package metadata cannot be relied upon, it might make more sense to implement a separate tool outside the package manager for this, similar to revedep-rebuild, perl-cleaner, etc.
(In reply to Mike Gilbert from comment #8) > The attached function appears to call "go version" on a hard-coded list of > directories (/usr/bin /bin /sbin). That's problematic: go binaries could be > installed anywhere. Yeah, I only did that because: underworld ~/systools # time go version $(find / -maxdepth 1 -not -name '.' -not -name '..' -not -wholename '/' -not -wholename /proc) real 5m52.990s user 1m39.123s sys 0m53.421s underworld ~/systools # time go version /usr/bin /bin /sbin ... real 0m0.495s user 0m0.107s sys 0m0.078s But obviously, '/' is more correct. You probably don't want/need packages in /home and probably some other directories (/proc?), so maybe there's a more (LSB-informed?) way to get a good set of directories? > I'm not sure it makes sense to implement this as a portage set. All existing > sets utilize package metadata to build a list of packages to be rebuilt. > None of them call external commands to scan installed files. > > If package metadata cannot be relied upon, it might make more sense to > implement a separate tool outside the package manager for this, similar to > revedep-rebuild, perl-cleaner, etc. It's funny you say that... that's sorta what I thought as well (and, in fact, what I did, since this became part of a set of custom scripts I use to maintain my Gentoo box.)
(In reply to J. Paul Reed from comment #7) > (In reply to Sam James from comment #4) > > On point 2., libcap does depend on Go with USE=tools, and that's the package > > which installs captree. If you know of (actual) missing dependencies, you > > should file bugs for them. > > FWIW, USE=tools is set in my make.conf, and yet @golang-rebuild doesn't > catch libpcap; that may be a (non-libpcap ebuild) bug? No, it doesn't get picked up by @golang-rebuild because not all go reverse dependencies get pulled into that set.
(also, libpcap != libcap.) This is looking to me like bug 827974 again.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=38c479d46dc91be66877d857a3682534eb1b5f12 commit 38c479d46dc91be66877d857a3682534eb1b5f12 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-09-10 06:22:44 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-09-28 23:56:08 +0000 cnf: sets: convert @golang-rebuild into VariableSet This allows rebuilding *all* Go packages correctly, rather than purely going off inherit. There's a few reasons to do this: 1. Even if (and it's a big if) we suppose that all Go packages should inherit a Go eclass, there will be packages in user repositories which don't do that; 2. Eclasses are, by their nature, repository-specific. This solution is a generic approach independent of the eclass layout in ::gentoo. Bug: https://bugs.gentoo.org/827974 Bug: https://bugs.gentoo.org/865115 Signed-off-by: Sam James <sam@gentoo.org> Closes: https://github.com/gentoo/portage/pull/898 Signed-off-by: Sam James <sam@gentoo.org> cnf/sets/portage.conf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=bb09a2d4db4cd0f85f8ae8ceaddc05ae2585aba3 commit bb09a2d4db4cd0f85f8ae8ceaddc05ae2585aba3 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-09-10 06:22:39 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-09-28 23:56:08 +0000 portage: sets: VariableSet: parse *DEPEND in includes/excludes VariableSet takes a metadata variable and checks its contents based on 'includes' or 'excludes' from the set definition. Unfortunately, until now, it didn't parse/expand the chosen variable and would only match it literally (it'd also not check effective metadata, just what's listed in the ebuild -- no *DEPEND from an eclass, for example). If variable is *DEPEND, actually parse includes/excludes so we can effecitvely match say, includes="dev-lang/go" against ">=dev-lang/go-1.18" in an ebuild. Bug: https://bugs.gentoo.org/827974 Bug: https://bugs.gentoo.org/865115 Signed-off-by: Sam James <sam@gentoo.org> lib/portage/_sets/dbapi.py | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=f034ac4a678a6da8d854f82a52d4fe523bf6cb8f commit f034ac4a678a6da8d854f82a52d4fe523bf6cb8f Author: Sam James <sam@gentoo.org> AuthorDate: 2022-09-29 02:37:14 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-09-30 20:35:00 +0000 cnf: sets: add @rust-rebuild set Rust is statically linked like Go and this is useful for us to mention in GLSAs (and possibly dev-lang/rust{,-bin}'s pkg_postinst). Bug: https://bugs.gentoo.org/827974 Bug: https://bugs.gentoo.org/865115 Signed-off-by: Sam James <sam@gentoo.org> Closes: https://github.com/gentoo/portage/pull/915 Signed-off-by: Sam James <sam@gentoo.org> cnf/sets/portage.conf | 6 ++++++ 1 file changed, 6 insertions(+)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=00f62c1578506cb2d94b3ccf922cb40fa128387a commit 00f62c1578506cb2d94b3ccf922cb40fa128387a Author: Sam James <sam@gentoo.org> AuthorDate: 2022-10-01 02:59:09 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-10-01 02:59:59 +0000 sys-apps/portage: add 3.0.38 Closes: https://bugs.gentoo.org/827974 Closes: https://bugs.gentoo.org/864259 Closes: https://bugs.gentoo.org/865115 Closes: https://bugs.gentoo.org/871570 Closes: https://bugs.gentoo.org/872392 Closes: https://bugs.gentoo.org/872440 Closes: https://bugs.gentoo.org/873088 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.38.ebuild | 273 +++++++++++++++++++++++++++++++++ 2 files changed, 274 insertions(+)
