`qlop -g <pkgname>` only shows the first 49 merges of a package, even though I have more in my history. Example: ``` $ qlop -g wine wine: Sat Jul 7 11:16:38 2012: 599 seconds wine: Wed Jul 18 15:04:13 2012: 2035 seconds wine: Tue Aug 7 18:25:06 2012: 805 seconds wine: Sun Aug 19 03:18:44 2012: 1336 seconds wine: Sun Sep 9 19:35:04 2012: 637 seconds wine: Sat Sep 15 16:46:39 2012: 655 seconds wine: Wed Oct 3 11:26:37 2012: 704 seconds wine: Thu Oct 18 20:29:11 2012: 676 seconds wine: Tue Oct 30 15:07:38 2012: 673 seconds wine: Sat Nov 10 11:43:54 2012: 683 seconds wine: Mon Dec 10 21:06:05 2012: 1238 seconds wine: Sat Dec 22 22:16:26 2012: 1839 seconds wine: Sat Dec 29 00:19:41 2012: 699 seconds wine: Sun Jan 20 00:30:41 2013: 709 seconds wine: Thu Jan 31 23:26:16 2013: 858 seconds wine: Thu Feb 7 15:07:01 2013: 1820 seconds wine: Sat Mar 9 02:18:17 2013: 805 seconds wine: Sat Mar 9 12:40:56 2013: 871 seconds wine: Sat Mar 9 13:20:52 2013: 786 seconds wine: Sat Mar 9 14:16:54 2013: 774 seconds wine: Thu Jul 18 23:31:09 2013: 779 seconds wine: Fri Jul 19 00:01:40 2013: 542 seconds wine: Wed Dec 11 12:01:32 2013: 495 seconds wine: Sat Feb 1 15:57:25 2014: 683 seconds wine: Sun Feb 16 03:44:22 2014: 460 seconds wine: Fri Mar 21 13:28:57 2014: 1423 seconds wine: Wed Apr 9 11:31:17 2014: 509 seconds wine: Sat May 3 18:42:19 2014: 1052 seconds wine: Tue Jul 8 07:12:47 2014: 581 seconds wine: Wed Jul 23 11:22:22 2014: 483 seconds wine: Tue Oct 14 15:25:39 2014: 513 seconds wine: Sun Jan 18 00:10:45 2015: 539 seconds wine: Thu Jan 29 06:21:05 2015: 689 seconds wine: Sun Feb 1 04:22:37 2015: 502 seconds wine: Sat Feb 7 13:04:32 2015: 985 seconds wine: Mon Jul 13 03:29:50 2015: 923 seconds wine: Fri Jul 31 06:04:54 2015: 583 seconds wine: Fri Jul 31 09:03:43 2015: 704 seconds wine: Fri Aug 7 21:03:56 2015: 691 seconds wine: Tue Aug 11 03:58:20 2015: 740 seconds wine: Mon Feb 8 05:40:06 2016: 1370 seconds wine: Tue Mar 29 04:15:09 2016: 926 seconds wine: Mon May 2 08:09:12 2016: 1997 seconds wine: Sat May 14 09:04:53 2016: 1061 seconds wine: Sun May 22 19:41:57 2016: 1028 seconds wine: Wed May 25 03:45:56 2016: 1035 seconds wine: Thu Jun 2 11:02:11 2016: 1167 seconds wine: Mon Jun 13 22:41:12 2016: 2662 seconds wine: Mon Jul 4 02:26:46 2016: 1318 seconds wine: 49 times $ qlop -l wine λ qlop -l wine Sat Jul 7 11:26:37 2012 >>> app-emulation/wine-1.5.8 Wed Jul 18 15:38:08 2012 >>> app-emulation/wine-1.5.9 Tue Aug 7 18:38:31 2012 >>> app-emulation/wine-1.5.10 Sun Aug 19 03:41:00 2012 >>> app-emulation/wine-1.5.11 Sun Sep 9 19:45:41 2012 >>> app-emulation/wine-1.5.12 Sat Sep 15 16:57:34 2012 >>> app-emulation/wine-1.5.13 Wed Oct 3 11:38:21 2012 >>> app-emulation/wine-1.5.14 Thu Oct 18 20:40:27 2012 >>> app-emulation/wine-1.5.15-r1 Tue Oct 30 15:18:51 2012 >>> app-emulation/wine-1.5.16-r1 Sat Nov 10 11:55:17 2012 >>> app-emulation/wine-1.5.17 Mon Dec 10 21:26:43 2012 >>> app-emulation/wine-1.5.19 Sat Dec 22 22:47:05 2012 >>> app-emulation/wine-1.5.20 Sat Dec 29 00:31:20 2012 >>> app-emulation/wine-1.5.20 Sun Jan 20 00:42:30 2013 >>> app-emulation/wine-1.5.21 Thu Jan 31 23:40:34 2013 >>> app-emulation/wine-1.5.22 Thu Feb 7 15:37:21 2013 >>> app-emulation/wine-1.5.23-r1 Sat Mar 9 02:31:42 2013 >>> app-emulation/wine-1.5.25 Sat Mar 9 12:55:27 2013 >>> app-emulation/wine-1.4.1 Sat Mar 9 13:33:58 2013 >>> app-emulation/wine-1.2.3 Sat Mar 9 14:29:48 2013 >>> app-emulation/wine-1.5.25 Thu Jul 18 23:44:08 2013 >>> app-emulation/wine-1.6_rc4 Fri Jul 19 00:10:42 2013 >>> app-emulation/wine-1.5.31 Sat Sep 21 14:27:37 2013 >>> app-emulation/wine-1.5.31 Wed Dec 11 12:09:47 2013 >>> app-emulation/wine-1.5.31 Sat Feb 1 16:08:48 2014 >>> app-emulation/wine-1.7.11 Sun Feb 16 03:52:02 2014 >>> app-emulation/wine-1.7.11 Fri Mar 21 13:52:40 2014 >>> app-emulation/wine-1.7.14 Wed Apr 9 11:39:46 2014 >>> app-emulation/wine-1.7.16 Sat May 3 18:59:51 2014 >>> app-emulation/wine-1.7.17 Tue Jul 8 07:22:28 2014 >>> app-emulation/wine-1.7.21 Wed Jul 23 11:30:25 2014 >>> app-emulation/wine-1.7.21 Tue Oct 14 15:34:12 2014 >>> app-emulation/wine-1.7.28 Sun Jan 18 00:19:44 2015 >>> app-emulation/wine-1.7.33 Thu Jan 29 06:32:34 2015 >>> app-emulation/wine-1.7.33 Sun Feb 1 04:30:59 2015 >>> app-emulation/wine-1.7.28 Sat Feb 7 13:20:57 2015 >>> app-emulation/wine-1.7.33 Mon Jul 13 03:45:13 2015 >>> app-emulation/wine-1.7.46 Fri Jul 31 06:14:37 2015 >>> app-emulation/wine-1.7.47 Fri Jul 31 09:15:27 2015 >>> app-emulation/wine-1.7.47 Fri Aug 7 21:15:27 2015 >>> app-emulation/wine-1.7.47 Tue Aug 11 04:10:40 2015 >>> app-emulation/wine-1.7.47 Mon Feb 8 06:02:56 2016 >>> app-emulation/wine-1.8 Tue Mar 29 04:30:35 2016 >>> app-emulation/wine-1.9.5 Mon May 2 08:42:29 2016 >>> app-emulation/wine-1.9.8 Sat May 14 09:22:34 2016 >>> app-emulation/wine-1.9.9 Sun May 22 19:59:05 2016 >>> app-emulation/wine-1.9.9 Wed May 25 04:03:11 2016 >>> app-emulation/wine-1.9.9 Thu Jun 2 11:21:38 2016 >>> app-emulation/wine-1.9.11 Mon Jun 13 23:25:34 2016 >>> app-emulation/wine-1.9.11 Mon Jul 4 02:48:44 2016 >>> app-emulation/wine-1.9.11 Thu Jul 14 01:27:50 2016 >>> app-emulation/wine-1.9.13 Wed Aug 3 07:25:10 2016 >>> app-emulation/wine-1.9.15 Sat Aug 13 22:35:45 2016 >>> app-emulation/wine-1.9.16 Sun Aug 28 14:47:56 2016 >>> app-emulation/wine-1.9.17 Sun Sep 11 12:00:30 2016 >>> app-emulation/wine-1.9.18 Sun Sep 11 17:00:05 2016 >>> app-emulation/wine-1.9.18 Mon Sep 12 14:24:38 2016 >>> app-emulation/wine-1.9.18 Wed Sep 21 19:59:29 2016 >>> app-emulation/wine-1.9.19-r1 Mon Oct 3 15:27:25 2016 >>> app-emulation/wine-1.9.10-r3 Mon Oct 10 03:48:55 2016 >>> app-emulation/wine-1.9.20 Mon Oct 17 20:25:56 2016 >>> app-emulation/wine-1.9.10-r3 Mon Oct 17 20:37:09 2016 >>> app-emulation/wine-1.9.20 Mon Nov 7 20:52:45 2016 >>> app-emulation/wine-1.9.10-r3 Wed Nov 9 01:19:20 2016 >>> app-emulation/wine-1.9.20 Wed Nov 9 01:31:31 2016 >>> app-emulation/wine-1.9.20 Wed Nov 9 01:43:45 2016 >>> app-emulation/wine-1.9.20 Mon Nov 28 13:24:21 2016 >>> app-emulation/wine-1.9.15-r1 Tue Dec 6 22:39:21 2016 >>> app-emulation/wine-1.9.20 Sat Dec 17 22:23:36 2016 >>> app-emulation/wine-9999 Sun Dec 18 14:03:41 2016 >>> app-emulation/wine-1.9.20 Sun Dec 18 19:16:23 2016 >>> app-emulation/wine-9999 ``` This bug does not affect other tools e.g. `genlop -t`. Reproducible: Always Steps to Reproduce: 1. Emerge a package more than 50 times 2. Use `qlop -g <pkgname>` Actual Results: `qlop -g` should show/consider all merges Expected Results: `qlop -g` only considers the first 49 merges
qlop doesn't have a limit of 50 entries. more likely your log file is confusing it. it is not an easy to parse file. please compress & attach yours.
@SpanKY: Which log file exactly are you talking about? I have a collection of files in /var/log/portage/elog that all begin with ‘app-emulation:wine’, 236 in total. Or is there some other central log I should be looking for?
(In reply to Niklas Haas from comment #2) qlop doesn't look at any elog files, it looks at the central log file that portage maintains. by default, that file lives at /var/log/emerge.log.
/var/log/emerge.log is too large to attach here even after XZ compression. I've uploaded it here: https://snow.haasn.xyz/~nand/log.xz
I'm seeing the same problem: ~ # qlop -l llvm Sun May 27 08:45:02 2012 >>> sys-devel/llvm-2.9-r2 Thu Sep 6 00:22:35 2012 >>> sys-devel/llvm-3.0-r2 Sun Nov 25 15:33:33 2012 >>> sys-devel/llvm-3.1-r2 Sun Mar 9 23:00:36 2014 >>> sys-devel/llvm-3.3-r3 Sat Dec 27 00:15:52 2014 >>> sys-devel/llvm-3.4.2 Thu Jan 29 04:39:33 2015 >>> sys-devel/llvm-3.5.0 Sun Apr 26 05:54:31 2015 >>> sys-devel/llvm-3.5.0 Sat Jul 2 15:49:16 2016 >>> sys-devel/llvm-3.7.1-r2 Sat Oct 29 12:44:10 2016 >>> sys-devel/llvm-3.7.1-r3 Tue Dec 6 05:59:26 2016 >>> sys-devel/llvm-3.7.1-r3 Sat Apr 8 21:53:18 2017 >>> sys-devel/llvm-3.9.1-r1 ~ # qlop -gH llvm llvm: Thu Sep 6 00:12:07 2012: 10 minutes, 28 seconds llvm: Sun Nov 25 15:19:45 2012: 13 minutes, 48 seconds llvm: Sun Mar 9 22:43:53 2014: 16 minutes, 43 seconds llvm: Fri Dec 26 23:42:10 2014: 33 minutes, 42 seconds llvm: Thu Jan 29 03:38:14 2015: 1 hour, 1 minute, 19 seconds llvm: Sun Apr 26 04:24:38 2015: 1 hour, 29 minutes, 53 seconds llvm: Sat Jul 2 12:54:24 2016: 2 hours, 54 minutes, 52 seconds llvm: Sat Oct 29 09:54:53 2016: 2 hours, 49 minutes, 17 seconds llvm: Tue Dec 6 02:34:36 2016: 3 hours, 24 minutes, 50 seconds llvm: 9 times Doesn't seem to be related with the number of merges. In this case the first and last merge are missing in the second output. Using portage-utils-0.64.
I have noticed the same issue, but with -g showing the first n (in this particular case, only those prior to some date. $qlop -l webkit-gtk Sun May 31 01:03:35 2015 >>> net-libs/webkit-gtk-2.4.8 Sun May 31 11:17:38 2015 >>> net-libs/webkit-gtk-2.4.8-r200 Tue Jun 9 02:15:39 2015 >>> net-libs/webkit-gtk-2.6.5 Wed Jun 24 09:47:51 2015 >>> net-libs/webkit-gtk-2.4.9 Wed Jun 24 19:53:25 2015 >>> net-libs/webkit-gtk-2.4.9-r200 Thu Jun 25 11:53:11 2015 >>> net-libs/webkit-gtk-2.6.6 Sun Sep 13 06:05:11 2015 >>> net-libs/webkit-gtk-2.8.5 Fri Dec 11 03:20:20 2015 >>> net-libs/webkit-gtk-2.4.9-r200 Wed Feb 24 21:54:03 2016 >>> net-libs/webkit-gtk-2.10.7 Wed Apr 13 11:48:53 2016 >>> net-libs/webkit-gtk-2.10.9 Thu Apr 14 05:14:33 2016 >>> net-libs/webkit-gtk-2.4.10 Fri May 13 17:41:37 2016 >>> net-libs/webkit-gtk-2.4.10 Sat May 14 10:25:46 2016 >>> net-libs/webkit-gtk-2.10.9 Wed Jul 13 07:10:24 2016 >>> net-libs/webkit-gtk-2.4.11 Tue Oct 4 17:52:01 2016 >>> net-libs/webkit-gtk-2.12.5 Sat Dec 3 20:59:28 2016 >>> net-libs/webkit-gtk-2.12.5 Sat Jan 14 02:26:09 2017 >>> net-libs/webkit-gtk-2.12.5 Tue Feb 14 10:35:34 2017 >>> net-libs/webkit-gtk-2.14.4 Sat Feb 18 06:56:29 2017 >>> net-libs/webkit-gtk-2.14.5 Thu Apr 13 23:46:45 2017 >>> net-libs/webkit-gtk-2.16.1 Tue May 16 06:52:02 2017 >>> net-libs/webkit-gtk-2.16.2 Thu Jun 1 22:42:38 2017 >>> net-libs/webkit-gtk-2.16.3 Mon Jun 26 18:48:52 2017 >>> net-libs/webkit-gtk-2.16.4 Fri Jun 30 02:35:13 2017 >>> net-libs/webkit-gtk-2.16.5 Thu Aug 3 20:24:57 2017 >>> net-libs/webkit-gtk-2.16.6 Fri Oct 27 01:06:17 2017 >>> net-libs/webkit-gtk-2.16.6 Sun Oct 29 23:24:13 2017 >>> net-libs/webkit-gtk-2.18.2 Sat Nov 25 04:43:48 2017 >>> net-libs/webkit-gtk-2.18.3 Mon Dec 11 03:44:53 2017 >>> net-libs/webkit-gtk-2.18.3 Wed Dec 27 18:36:17 2017 >>> net-libs/webkit-gtk-2.18.4 $ qlop -g webkit-gtk webkit-gtk: Sat May 30 12:36:39 2015: 44816 seconds webkit-gtk: Sun May 31 01:03:35 2015: 36843 seconds webkit-gtk: Mon Jun 8 12:02:47 2015: 51172 seconds webkit-gtk: Tue Jun 23 21:37:03 2015: 43848 seconds webkit-gtk: Wed Jun 24 09:47:51 2015: 36334 seconds webkit-gtk: Wed Jun 24 19:53:25 2015: 57586 seconds webkit-gtk: Sat Sep 12 14:39:50 2015: 55521 seconds webkit-gtk: Thu Dec 10 15:13:31 2015: 43609 seconds webkit-gtk: Tue Apr 12 14:50:41 2016: 75492 seconds webkit-gtk: Wed Apr 13 11:48:53 2016: 62740 seconds webkit-gtk: Fri May 13 04:08:48 2016: 48769 seconds webkit-gtk: Fri May 13 17:41:37 2016: 60249 seconds webkit-gtk: Tue Jul 12 17:08:55 2016: 50489 seconds webkit-gtk: 13 times rather then upload emerge.log to some other site, is there a "grep" which will include the relevant lines?
Are there by chance failed merges in there? Or multiple portage runs at the same time?
Possible, yes, but not likely for every case after July 2016. I wonder if something about either the last emerge shown by -g, or something in the log shortly after that, is messing up qlop so it stops reporting at that point. Again - what grep can I use to get the lines qlop is looking at (or is reading the code the only way?)
I don't have an answer ready, I need to look into this myself :) Have a look at the code, here: https://gitweb.gentoo.org/proj/portage-utils.git/tree/qlop.c#n76 From line 106 it's going to loop line-by-line over the logfile.
For some reason yet unknown to me, list and gauge use different code. Making both share the same code, will likely bring them in sync.
On second look, no. The problem is the gauge code gets confused by the log, because there is an emerge terminating (1404894355: Started emerge on: Jul 09, 2014 10:25:54 check for llvm-3.4.2) during the merging. The gauge code tries to track parallel running emerges in order to make sense of things, but in this case it didn't know there was another emerge already running when llvm started, hence it thought the main emerge ended, which is not true.
Additional question - does "ebuild" put anything into the emerge log? I have had numerous cases where an emerge failed, and instead of starting from scratch, I managed to correct the problem and then manually do the remaining ebuild steps. (often just install and qmerge - after emerge failed due to a sandbox violation, but after actually completing the compile step.)
As far as I am aware, only emerge logs into emerge.log, so ebuild X operations aren't visible. Yesterday I managed to fix one case where -g didn't see a merge, but it still isn't complete. My aim is to get -l and -g to align, if possible.
I managed to get one case covered, but the intrinsic problem is that the logfile isn't consistent, which is causing problems. There are 7719 "Started emerge" messages, and 7665 "terminating." messages.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=80d7de29b2c1f977e84bc653933545a6eb589694 commit 80d7de29b2c1f977e84bc653933545a6eb589694 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2018-04-04 13:13:15 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2018-04-04 13:13:15 +0000 show_merge_times: start counting parallel merges from start of the file If we start counting parallel merges only once we found something interesting, we might not be aware of other emerges at that time, drawing wrong conclusions when seeing a termination message. Bug: https://bugs.gentoo.org/603024 qlop.c | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-)}
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=c74a5abf3fd8db03adb531f95ecff5316d997ab3 commit c74a5abf3fd8db03adb531f95ecff5316d997ab3 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-02-27 20:41:45 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-02-27 20:50:21 +0000 qlop: rewrite from scratch This new implementation achieves a few things: - unified code path for all modes of operation (thus consistent results) - more flexibility to implement new features - simplification of the codebase Short version of what this commit changes: - existing flags -t -g have been replaced with -a and -t - -c has been been replaced with -r and no longer uses proc processing code (thus works everywhere) - addition of an ETA for running emerges (subject to improvements) - allow reading a file of atoms (e.g. /var/lib/portage/world) - summary mode -c to compute grand total, e.g. to compute world compile time Bug: https://bugs.gentoo.org/161244 Bug: https://bugs.gentoo.org/603024 Bug: https://bugs.gentoo.org/636334 Bug: https://bugs.gentoo.org/658824 Signed-off-by: Fabian Groffen <grobian@gentoo.org> man/include/qlop.desc | 25 +- man/include/qlop.optdesc.yaml | 23 +- man/qlop.1 | 66 +- qlop.c | 1479 +++++++++++++++++++++-------------------- tests/qlop/dotest | 20 +- tests/qlop/list01.good | 4 +- tests/qlop/list02.good | 6 +- tests/qlop/list03.good | 8 +- tests/qlop/list04.good | 2 +- tests/qlop/list05.good | 2 +- tests/qlop/list06.good | 4 +- tests/qlop/list07.good | 4 +- tests/qlop/list08.good | 4 +- tests/qlop/list09.good | 6 +- 14 files changed, 892 insertions(+), 761 deletions(-)
this is available onwards from 0.80_pre*
