When using the -v flag when running qlop, it results in qlop being unable to calculate ETAs for running emerges: > home ~ # qlop -r > 2021-08-13T11:00:54 >>> dev-python/QtPy... (38 of 612) ETA: 16s > home ~ # qlop -rv > 2021-08-13T11:00:54 >>> dev-python/QtPy-1.9.0-r5... (38 of 612) ETA: unknown I suspect qlop is trying to match against the specific version of the package when I use -v, but according to the manpage, -v is only intended to be a verbose flag, and the flag also controls whether qlop tells you whether it's using the average or longest run to calculate the ETA. Reproducible: Always Steps to Reproduce: 1. Start an update emerge that lasts long enough for you to execute the next step while it's happening. 2. During the emerge in step 1, run "qlop -rv" as root in another terminal. Actual Results: qlop prints "unknown" for the ETA. Expected Results: qlop should print the actual ETA, along with whether it's using the average run or the longest run in its calculation. I am currently updating my whole system so this emerge --info might have some outdated packages, but I don't believe this will affect this bug. Portage 3.0.20 (python 3.8.6-final-0, default/linux/amd64/17.1, gcc-9.3.0, glibc-2.33-r1, 5.4.80-gentoo-r1 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.4.80-gentoo-r1-x86_64-Intel-R-_Core-TM-_i7-5820K_CPU_@_3.30GHz-with-glibc2.2.5 KiB Mem: 24604524 total, 10781932 free KiB Swap: 8191996 total, 8165884 free Timestamp of repository gentoo: Wed, 11 Aug 2021 00:45:01 +0000 sh bash 5.1_p8 ld GNU ld (Gentoo 2.34 p6) 2.34.0 ccache version 3.7.12 [disabled] app-shells/bash: 5.1_p8::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.34.0::gentoo dev-lang/python: 2.7.18-r4::gentoo, 3.6.12::gentoo, 3.7.9::gentoo, 3.8.6::gentoo, 3.9.0::gentoo dev-lang/rust: 1.53.0::gentoo dev-util/ccache: 3.7.12::gentoo dev-util/cmake: 3.17.4-r1::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.24::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.13.4-r2::gentoo, 1.16.3-r1::gentoo sys-devel/binutils: 2.34-r2::gentoo, 2.35.2::gentoo sys-devel/gcc: 6.5.0-r3::gentoo, 8.4.0-r1::gentoo, 9.3.0-r2::gentoo, 10.3.0-r2::gentoo sys-devel/gcc-config: 2.4::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers) sys-libs/glibc: 2.33-r1::gentoo Repositories: gentoo location: /usr/portage sync-type: webrsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-webrsync-verify-signature: yes sph-local location: /opt/portage-overlay masters: gentoo priority: 0 anomen location: /var/lib/layman/anomen masters: gentoo priority: 50 dotnet location: /var/lib/layman/dotnet masters: gentoo priority: 50 gambas-overlay location: /var/lib/layman/gambas-overlay masters: gentoo priority: 50 jorgicio location: /var/lib/layman/jorgicio masters: gentoo priority: 50 lmiphay location: /var/lib/layman/lmiphay masters: gentoo priority: 50 menelkir location: /var/lib/layman/menelkir masters: gentoo priority: 50 nest location: /var/lib/layman/nest masters: gentoo priority: 50 netfab-overlay location: /var/lib/layman/netfab-overlay masters: gentoo priority: 50 palemoon location: /var/lib/layman/palemoon masters: gentoo priority: 50 pentoo location: /var/lib/layman/pentoo masters: gentoo priority: 50 stuff location: /var/lib/layman/stuff masters: gentoo priority: 50 vapoursynth location: /var/lib/layman/vapoursynth masters: gentoo priority: 50 wine location: /var/lib/layman/wine masters: gentoo priority: 50 x11 location: /var/lib/layman/x11 masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/avfs/extfs /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.4/ext-active/ /etc/php/cgi-php7.4/ext-active/ /etc/php/cli-php7.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--ask-enter-invalid --autounmask --autounmask-keep-masks y" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="https://ftp.snt.utwente.nl/pub/os/linux/gentoo/" LANG="en_GB.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en en-GB" MAKEOPTS="-j10" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac aacplus acl acpi alsa amd64 amr bluray bzip2 cairo cdda cddb cdio cdparanoia cli crypt cups dbus dri dts dvd flac fluidsynth fontconfig fortran gdbm gpm gtk ibus iconv icu ipv6 jpeg libglvnd libnotify libtirpc lzma mad mmx modplug mp3 mtp multilib ncurses nls nptl ogg opengl openmp pam pcre png qt3support qt5 readline seccomp sound split-usr sse sse2 ssl startup-notification svg tcl tcpd theora tk truetype unicode v4l vdpau vim-syntax vorbis vulkan xattr xv xvmc zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 f16c fma3 pclmul popcnt sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="python3_8" QEMU_SOFTMMU_TARGETS="x86_64 arm i386 mips mipsel ppc sparc" QEMU_USER_TARGETS="aarch64 alpha arm i386 m68k mips mipsel ppc sparc x86_64" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS ================================================================= Package Settings ================================================================= app-portage/portage-utils-0.92::gentoo was built with the following: USE="nls openmp qmanifest qtegrity -static" ABI_X86="(64)"
It acts indeed on the version itself, for this the -p (--predict) option was added to 0.92, as a replacement of the averaging technique of -r (it uses -a in the back). I understand this is something that only happens during emerging of new packages, but could you tell me if -p for e.g. QtPy comes close to the last emerge time of the package (QtPy-1.9.0-r5)?
Without a running merge going: > home ~ # qlop -p QtPy > QtPy: prediction 12s > home ~ # qlop -tv QtPy > 2021-01-07T08:43:04 >>> dev-python/QtPy-1.9.0-r2: 17s > 2021-08-11T10:19:04 >>> dev-python/QtPy-1.9.0-r2: 19s > 2021-08-13T11:00:54 >>> dev-python/QtPy-1.9.0-r5: 15s > 2021-08-14T23:48:57 >>> dev-python/QtPy-1.9.0-r5: 15s So it seems close, but strangely faster than the actual time it took to merge before. I do notice that it's impossible to use both the -r and the -p switches together, though, to predict how much longer a running merge will take. Is this something that would be possible to add?
it is the next step (instead of using -a, use -p with -r) but first it needs perfecting can you run qlop -p dev-python/QtPy-1.9.0-r5 without the version part, qlop "predicts" it will be the next version, and since you went down from average 18 to average 15 seconds, it assumes a linear continuation of that, and ends up with 12 seconds. If you ask it to predict a known version/revision it should base its prediction on hard evidence (the exact same version being merged already) and thus in your case produce 15.
No difference in output: > home ~ # qlop -p dev-python/QtPy-1.9.0-r5 > dev-python/QtPy: prediction 12s > home ~ # qlop -p =dev-python/QtPy-1.9.0-r5 > dev-python/QtPy: prediction 12s
can you do grep dev-python/QtPy /var/log/emerge.log and send me the result? I'd like to look at this in some detail
Created attachment 733138 [details] Output of "grep dev-python/QtPy /var/log/emerge.log" The output from the requested command, "grep dev-python/QtPy /var/log/emerge.log".
I created a pull request to fix this: https://github.com/gentoo/portage-utils/pull/15
thanks, sounds sensical
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=fc71a04651ab56b0133c61d839169a0cabf6f092 commit fc71a04651ab56b0133c61d839169a0cabf6f092 Author: Jaak Ristioja <jaak@ristioja.ee> AuthorDate: 2022-02-11 15:31:05 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2022-02-12 16:14:11 +0000 qlop: Fixed --running --verbose returning unknown ETA It seems that this was broken during the introduction of --predict in commit 13402fbd8c51f7feedcc85f2f0815768ec45ee7a which caused keys of the merge_averages and unmerge_averages sets to include version information, whereas --running expects these not to include version information. Bug: https://bugs.gentoo.org/807975 Signed-off-by: Jaak Ristioja <jaak@ristioja.ee> Signed-off-by: Fabian Groffen <grobian@gentoo.org> qlop.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
pushed
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ec3307e507ccfc092371f8e16ea40ff50557ca9f commit ec3307e507ccfc092371f8e16ea40ff50557ca9f Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2022-06-14 07:12:19 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2022-06-14 07:15:51 +0000 app-portage/portage-utils-0.94: version bump Closes: https://bugs.gentoo.org/851138 Closes: https://bugs.gentoo.org/837188 Closes: https://bugs.gentoo.org/836590 Closes: https://bugs.gentoo.org/833942 Closes: https://bugs.gentoo.org/807975 Signed-off-by: Fabian Groffen <grobian@gentoo.org> app-portage/portage-utils/Manifest | 1 + .../portage-utils/portage-utils-0.94.ebuild | 67 ++++++++++++++++++++++ 2 files changed, 68 insertions(+)
