| Summary: | =app-portage/portage-utils-0.82: qlop -r misses some packages that are currently merged when running as non-root user in the portage group | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Till Schäfer <till2.schaefer> |
| Component: | Tools | Assignee: | Fabian Groffen <grobian> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | ||
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: | tail of emerge.log | ||
Created attachment 598416 [details]
tail of emerge.log
is this during the entire process of merging, or only near the end of it, e.g. when it's cleaning up? the firefox merge was still going for another half an hour or so. ETA of genlop is heavily biased by very old firefox releases, that took only a few minutes to merge. and qlop was ran as root? no, as a user in to the portage group. Ok, that probably is the reason why then. yes, you are right. It is always reproducible with a non-root user in the portage group. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=1e5b2c8d3fb58335990e411af98498f249b54980 commit 1e5b2c8d3fb58335990e411af98498f249b54980 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-12-27 20:38:44 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-12-27 20:38:44 +0000 qlop: some changes to -r (running) mode - warn when qlop needs to defer to log heuristics (#701968) - print running packages most recent first - suppress identical running packages (#701392) - ignore batches in emerge.log that appear to be backwards in time Bug: https://bugs.gentoo.org/701968 Bug: https://bugs.gentoo.org/701392 Signed-off-by: Fabian Groffen <grobian@gentoo.org> qlop.c | 50 +++++++++++++++++++++++++++++++++++++++++++------- tests/qlop/list01.good | 2 +- tests/qlop/list10.good | 4 ++-- tests/qlop/sync.log | 12 ++++++------ 4 files changed, 52 insertions(+), 16 deletions(-) The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ccbfbff3ffb51a9e7783458c9dad3e6d2cb49e29 commit ccbfbff3ffb51a9e7783458c9dad3e6d2cb49e29 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-12-28 09:42:05 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-12-28 09:42:23 +0000 app-portage/portage-utils-0.83: version bump with bugfixes Drop overshadowed testing versions. Closes: https://bugs.gentoo.org/701968 Closes: https://bugs.gentoo.org/701470 Bug: https://bugs.gentoo.org/701402 Closes: https://bugs.gentoo.org/701392 Closes: https://bugs.gentoo.org/701386 Closes: https://bugs.gentoo.org/700850 Closes: https://bugs.gentoo.org/699558 Package-Manager: Portage-2.3.79, Repoman-2.3.16 Signed-off-by: Fabian Groffen <grobian@gentoo.org> app-portage/portage-utils/Manifest | 3 +- .../portage-utils/portage-utils-0.74-r1.ebuild | 38 ---------- .../portage-utils/portage-utils-0.81.ebuild | 87 ---------------------- ...utils-0.82.ebuild => portage-utils-0.83.ebuild} | 0 4 files changed, 1 insertion(+), 127 deletions(-) The current solution seems suboptimal to me, since it was possible to run qlop -r with pre-0.80 version and a portage group user without any workarounds. AFAIK this versions also relied on watching the current running processes. The same applies to genlop. The current workaround re-introduces many of the fixed problems in the from the previous versions. Thus, maybe there is a better solution. PS: The error message contains a small typo: "lop: ..." (missing q). Feel free to close the bug again, if I am wrong with something. The "lop" thing is because the applet is called "lop", but I don't like it either. What genlop and older portage-utils do, is to look at the process names, and match sandbox. Sandbox has a nifty feature to embed the package name in its process name when it runs. This one is extracted and that's what's being used. There's some things not nice about that approach: 1) it assumes sandbox is in use, and that sandbox remains doing what it does, while there have been some people voicing the opinion that sandbox is outdated and should go. Its performance isn't too good in some aspects either. 2) an ongoing "emerge" is only visible when the sandbox stage runs, e.g. the compile and install. While this is the majority, it isn't the full thing. FEATURES=-sandbox essentially numbs the sandbox detection approach. However, since /proc/x/cmdline is world-readable, qlop could try to scan for sandbox there, and use matches it finds, for the majority of people will still have sandbox for a while. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=7cdf692beb93c596b4bf4f5f7b5bb8bcc69a27ea commit 7cdf692beb93c596b4bf4f5f7b5bb8bcc69a27ea Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-12-29 09:54:28 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-12-29 09:54:28 +0000 q: improve applet searching somewhat, change warn app name - use a single loop iteration to match an applet - set full name of applet (qlop iso lop) in argv0 for warn() to match the running command - use name of aliased applet when invoked by an alias (belongs -> qfile) Bug: https://bugs.gentoo.org/701968 Signed-off-by: Fabian Groffen <grobian@gentoo.org> applets.h | 1 - q.c | 37 +++++++++++++++++++++++++++---------- 2 files changed, 27 insertions(+), 11 deletions(-) The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1278e75ff758df9e3a7f11d57d36dc378461fe2e commit 1278e75ff758df9e3a7f11d57d36dc378461fe2e Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2020-10-04 20:14:05 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2020-10-04 20:14:05 +0000 app-portage/portage-utils-0.90: version bump for qlop fixes Closes: https://bugs.gentoo.org/745798 Closes: https://bugs.gentoo.org/731122 Closes: https://bugs.gentoo.org/713536 Closes: https://bugs.gentoo.org/705748 Closes: https://bugs.gentoo.org/701968 Package-Manager: Portage-3.0.4, Repoman-3.0.1 Signed-off-by: Fabian Groffen <grobian@gentoo.org> app-portage/portage-utils/Manifest | 2 +- ...portage-utils-0.88.ebuild => portage-utils-0.90.ebuild} | 14 -------------- 2 files changed, 1 insertion(+), 15 deletions(-) |
qlop -r misses to report some packages. I have not ran any other merge in parallel, but I am using jobs=2. $ qlop -r $ genlop -c Currently merging 26 out of 70 * www-client/firefox-71.0 current merge time: 48 minutes and 36 seconds. ETA: any time now. Reproducible: Always # emerge --info portage-utils Portage 2.3.79 (python 3.6.9-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-9.2.0, glibc-2.29-r2, 5.3.14-gentoo-r1 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.3.14-gentoo-r1-x86_64-Intel-R-_Core-TM-_i7-3770_CPU_@_3.40GHz-with-gentoo-2.6 KiB Mem: 15920316 total, 4681376 free KiB Swap: 16801788 total, 15166972 free Timestamp of repository gentoo: Wed, 04 Dec 2019 10:46:32 +0000 Head commit of repository gentoo: 3262acc414c460b831f96f93d81111125a8c1ada sh bash 4.4_p23-r1 ld GNU ld (Gentoo 2.32 p2) 2.32.0 app-shells/bash: 4.4_p23-r1::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.28.2-r1::gentoo dev-lang/python: 2.7.16::gentoo, 3.6.9::gentoo dev-util/cmake: 3.14.6::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.41.2::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.13.4-r2::gentoo, 1.15.1-r2::gentoo, 1.16.1-r1::gentoo sys-devel/binutils: 2.32-r1::gentoo sys-devel/gcc: 9.2.0-r2::gentoo sys-devel/gcc-config: 2.1::gentoo sys-devel/libtool: 2.4.6-r3::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 4.19::gentoo (virtual/os-headers) sys-libs/glibc: 2.29-r2::gentoo Repositories: gentoo location: /usr/portage sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo.git priority: -1000 sync-git-verify-commit-signature: true shared_overlay location: /opt/conf/common/var/db/repos/shared_overlay masters: gentoo priority: 100 local_overlay location: /var/db/repos/local_overlay masters: gentoo priority: 200 Installed sets: @system ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe -ftree-vectorize -ggdb" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.6/conf /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.3/ext-active/ /etc/php/cgi-php7.3/ext-active/ /etc/php/cli-php7.3/ext-active/ /etc/portage/package.accept_keywords/99-autounmask /etc/portage/package.unmask/99-autounmask /etc/portage/package.use/99-autounmask /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe -ftree-vectorize -ggdb" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y --autounmask=y --autounmask-write --autounmask-continue --jobs=2 --load-average=8 --backtrack=100" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned sandbox sfperms sign splitdebug strict strict-keepdir unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j8" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acl acpi activities aes alsa amd64 apng avx bash-completion berkdb bluetooth branding brotli bzip2 cairo cdaudio cdda cddb cdparanoia cdr chm cli consolekit crypt cups cxx dbus declarative djvu dnssec dri dts dvd dvdr emboss encode epub evdev exif f16c fam ffmpeg flac fortran gdbm gif glib gpg gpm gstreamer gtk gzip iconv icu id3tag idn imagemagick inotify ipv6 irc jpeg jpeg2k kde kipi kwallet lame latex lcms ldap libnotify libsamplerate libtirpc lvm lzma mad matroska mmx mmxext mng mp3 mp4 mpeg mplayer mtp multilib musicbrainz ncurses nls nptl ntp ogg opengl openmp opus otr pam pango pclmul pcre pdf phonon plasma png policykit popcnt ppds pulseaudio qml qt5 quicktime rar readline real rss sdl seccomp semantic-desktop spell spice split-usr sse sse2 sse3 sse4_1 sse4_2 ssl ssse3 startup-notification svg syslog taglib tcpd theora threads thumbnail tiff truetype twolame udev udisks uefi unicode upower usb v4l v4l2 vaapi vcd vim-syntax visualization vorbis vpx wavpack widgets wma wmf wxwidgets x264 x265 xattr xcb xcomposite xinerama xml xv xvid xvidv xvmc xz zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx f16c mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= app-portage/portage-utils-0.82::gentoo was built with the following: USE="nls openmp qmanifest qtegrity -libressl -static" ABI_X86="(64)"