Built from: commit 868b2b4b4f5140da7188389acf9f719fcf343ab2 Author: Fabian Groffen <grobian@gentoo.org> Date: Fri Apr 19 13:47:29 2019 +0200 gdb --args qcache -c dev-perl -i x86 ... (gdb) run Starting program: /usr/bin/qcache -c dev-perl -i x86 Program received signal SIGSEGV, Segmentation fault. 0x000055555555f135 in decode_arch () Thread 1 (process 20416): #0 0x000055555555f135 in decode_arch () No symbol table info available. #1 0x00005555555614ae in qcache_main () No symbol table info available. #2 0x000055555555e58f in q_main () No symbol table info available. #3 0x000055555555c53b in main () No symbol table info available. [1519099.314163] qcache[20410]: segfault at 0 ip 0000562b003b4135 sp 00007ffde5b46fd0 error 4 in q[562b003b0000+26000] [1519099.314171] Code: 00 00 00 00 90 41 55 41 54 55 48 89 fd 53 48 83 ec 08 0f b6 07 3c 7e 74 54 3c 2d 74 50 4c 8b 2d e9 37 03 00 41 bc ff ff ff ff <49> 8b 7d 00 48 85 ff 74 27 31 db eb 14 66 0f 1f 44 00 00 48 83 c3 More over, all the output for qcache --help talks about "given arch", but has no defined way to "give" an arch ( other than what I just tried ) I poked around at its calls to getenv, and it seems it reads ARCH from ENV, and DEBUG from ENV, and setting ARCH and DEBUG give output indicating it reads ARCH... but like, the output doesn't change respective to the ARCH in question like it should, and clearly invalid values of ARCH work just fine and give identical output to what would be expected on amd64 DEBUG=1 ARCH=vogon-poetry qcache -c dev-perl -n profile ////etc/make.profile/make.defaults profile ////etc/make.profile/../make.defaults profile ////etc/make.profile/../../../../base/make.defaults profile ////etc/make.profile/../../make.defaults profile ////etc/portage/make.profile/make.defaults profile //usr/share/portage/config/make.globals profile ////etc/make.conf profile ////etc/portage/make.conf profile ////etc/portage/"/var/lib/layman/make.conf" repos.conf.d scanner ///etc/portage/repos.conf parse ///etc/portage/repos.conf/gentoo.conf parse ///etc/portage/repos.conf/layman.conf parse ///etc/portage/repos.conf/local.conf parse ///etc/portage/repos.conf/lte.conf parse ///etc/portage/repos.conf/perl-eclass-testsuite.conf ROOT = / ACCEPT_LICENSE = * -@EULA INSTALL_MASK = /usr/lib/debug/.build-id/ PKG_INSTALL_MASK = ARCH = vogon-poetry CONFIG_PROTECT = /etc /etc /etc /usr/share/maven-bin-3.5/conf /usr/share/maven-bin-3.6/conf /usr/share/gnupg/qualified.txt /usr/share/config /usr/lib64/libreoffice/program/sofficerc CONFIG_PROTECT_MASK = /etc/env.d /etc/gconf /etc/env.d /etc/sandbox.d /etc/php/cli-php7.2/ext-active/ /etc/php/cgi-php7.2/ext-active/ /etc/php/apache2-php7.2/ext-active/ /etc/php/cli-php7.3/ext-active/ /etc/php/cgi-php7.3/ext-active/ /etc/php/apache2-php7.3/ext-active/ /etc/fonts/fonts.conf /etc/gentoo-release /etc/gconf /etc/terminfo /etc/dconf /etc/ca-certificates.conf /etc/texmf/web2c /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/revdep-rebuild NOCOLOR = 0 FEATURES = assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync ipc-sandbox network-sandbox strict-keepdir xattr clean-logs compress-build-logs binpkg-logs fixlafiles sandbox split-elog split-log userfetch usersandbox userpriv xattr sign ipc-sandbox parallel-install -ebuild-locks network-sandbox cgroup splitdebug compressdebug metadata-transfer EPREFIX = / EMERGE_LOG_DIR = /var/log PORTDIR = /usr/portage PORTAGE_BINHOST = PORTAGE_TMPDIR = /var/tmp PKGDIR = /usr/portage/packages Q_VDB = /var/db/pkg Q_EDB = /var/cache/edb dev-perl/AnyEvent-HTTPD dev-perl/Class-Mix dev-perl/Crypt-Eksblowfish dev-perl/Crypt-X509 dev-perl/Data-GUID dev-perl/Data-Types dev-perl/Date-Extract dev-perl/DateTime-Tiny dev-perl/Email-Address-List dev-perl/Filesys-SmbClient dev-perl/Google-Ads-AdWords-Client dev-perl/HTML-FormatText-WithLinks dev-perl/HTML-FormatText-WithLinks-AndTables dev-perl/IMAP-BodyStructure dev-perl/Lab-Zhinst dev-perl/Mac-Pasteboard dev-perl/Math-ModInt dev-perl/MongoDB dev-perl/Net-ARP dev-perl/Net-Trackback dev-perl/PPI-PowerToys dev-perl/Quota dev-perl/REST-Client dev-perl/RRD-Simple dev-perl/Rcs dev-perl/Role-Basic dev-perl/Schedule-Cron-Events dev-perl/Sort-Key-IPv4 dev-perl/Symbol-Global-Name dev-perl/TAP-Parser-SourceHandler-pgTAP dev-perl/Template-Plugin-Cycle dev-perl/Test-Strict dev-perl/Time-Stopwatch dev-perl/UUID-Tiny dev-perl/XML-Directory dev-perl/XML-Validator-Schema Other diagnostics: Prior to the SEGV occurring, strace says: openat(AT_FDCWD, "///etc/portage/repos.conf/local.conf", O_RDONLY) = 3 fstat(3, {st_dev=makedev(0x8, 0x6), st_ino=4203041, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=88, st_atime=1556104004 /* 2019-04-24T23:06:44.845276943+1200 */, st_atime_nsec=845276943, st_mtime=1440704345 /* 2015-08-28T07:39:05+1200 */, st_mtime_nsec=0, st_ctime=1446214286 /* 2015-10-31T03:11:26.383285586+1300 */, st_ctime_nsec=383285586}) = 0 read(3, "[local-portage]\npriority = 100\nlocation = /usr/local/portage\nsync-type =\nauto-sync = no\n", 4096) = 88 read(3, "", 4096) = 0 close(3) = 0 openat(AT_FDCWD, "///etc/portage/repos.conf/lte.conf", O_RDONLY) = 3 fstat(3, {st_dev=makedev(0x8, 0x6), st_ino=4199869, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=96, st_atime=1556104004 /* 2019-04-24T23:06:44.859276978+1200 */, st_atime_nsec=859276978, st_mtime=1468569135 /* 2016-07-15T19:52:15.814317172+1200 */, st_mtime_nsec=814317172, st_ctime=1468569135 /* 2016-07-15T19:52:15.859317145+1200 */, st_ctime_nsec=859317145}) = 0 read(3, "[kentnl-gentoo-lte]\npriority = 100\nlocation = /usr/local/gentoo-lted\nsync-type =\nauto-sync = no\n", 4096) = 96 read(3, "", 4096) = 0 close(3) = 0 openat(AT_FDCWD, "///etc/portage/repos.conf/perl-eclass-testsuite.conf", O_RDONLY) = 3 fstat(3, {st_dev=makedev(0x8, 0x6), st_ino=4195678, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=118, st_atime=1556104004 /* 2019-04-24T23:06:44.871277009+1200 */, st_atime_nsec=871277009, st_mtime=1487931826 /* 2017-02-24T23:23:46.260404061+1300 */, st_mtime_nsec=260404061, st_ctime=1487931826 /* 2017-02-24T23:23:46.319403988+1300 */, st_ctime_nsec=319403988}) = 0 read(3, "[perl-eclass-testsuite]\npriority = 100\nlocation = /home/kent/gentoo/perl-eclass-testsuite/\nsync-type =\nauto-sync = no\n", 4096) = 118 read(3, "", 4096) = 0 close(3) = 0 openat(AT_FDCWD, "/etc/portage/color.map", O_RDONLY) = -1 ENOENT (No such file or directory) And ltrace says: [0x5561d0c4abce] malloc(12) = 0x5561d1d651c0 [0x5561d0c4a32e] __errno_location() = 0x7f56c670b6c0 [0x5561d0c4a65e] __snprintf_chk(0x5561d1d651c0, 12, 1, -1) = 21 [0x5561d0c4a716] realloc(0x5561d1d651c0, 24) = 0x5561d1d651c0 [0x5561d0c4a65e] __snprintf_chk(0x5561d1d651c0, 24, 1, -1) = 21 [0x5561d0c4a0c7] realloc(0x5561d1d651c0, 48) = 0x5561d1d69be0 [0x5561d0c4a0f4] memcpy(0x5561d1d69bf5, ":location", 9) = 0x5561d1d69bf5 [0x5561d0c4af55] realloc(0x5561d1d69be0, 31) = 0x5561d1d69be0 [0x5561d0c2883f] iniparser_getstring(0x5561d1d651e0, 0x5561d1d69be0, 0, 0) = 0x5561d1d6a180 [0x5561d0c28850] strlen("/home/kent/gentoo/perl-eclass-testsuite/") = 40 [0x5561d0c49b59] malloc(41) = 0x5561d1d69fb0 [0x5561d0c4737f] memcpy(0x5561d1d69fb0, "/home/kent/gentoo/perl-eclass-testsuite/\0", 41) = 0x5561d1d69fb0 [0x5561d0c2889a] free(0x5561d1d69be0) = <void> [0x5561d0c288af] iniparser_freedict(0x5561d1d651e0, 5, 0x5561d1d69dc0, 0x5561d1d61010) = 0 [0x5561d0c283fb] free(0x5561d1d652e0) = <void> [0x5561d0c4623c] free(0x5561d1d64340) = <void> [0x5561d0c4623c] free(0x5561d1d648e0) = <void> [0x5561d0c4623c] free(0x5561d1d61880) = <void> [0x5561d0c4623c] free(0x5561d1d648b0) = <void> [0x5561d0c4623c] free(0x5561d1d64880) = <void> [0x5561d0c4623c] free(0x5561d1d66020) = <void> [0x5561d0c4623c] free(0x5561d1d61300) = <void> [0x5561d0c4623c] free(0x5561d1d612a0) = <void> [0x5561d0c2841a] free(0x5561d1d64820) = <void> [0x5561d0c28424] free(0x5561d1d647f0) = <void> [0x5561d0c28439] free(0x5561d1d613a0) = <void> [0x5561d0c28457] getenv("DEBUG") = nil [0x5561d0c28509] getenv("PORTAGE_QUIET") = nil [0x5561d0c43c01] fopen("/etc/portage/color.map", "r") = 0 [0x5561d0c4322e] strrchr("qcache", '/') = nil [0x5561d0c2a02c] strcmp("q", "qcache") = -99 [0x5561d0c2a02c] strcmp("qatom", "qcache") = -2 [0x5561d0c2a02c] strcmp("qcache", "qcache") = 0 [0x5561d0c2d35c] getopt_long(5, 0x7ffef80c4598, "p:c:idtansvqChV", 0x5561d0c5b240, nil) = 99 [0x5561d0c2d35c] getopt_long(5, 0x7ffef80c4598, "p:c:idtansvqChV", 0x5561d0c5b240, nil) = 105 [0x5561d0c2d35c] getopt_long(5, 0x7ffef80c4598, "p:c:idtansvqChV", 0x5561d0c5b240, nil) = -1 [0x5561d0c2b135] --- SIGSEGV (Segmentation fault) --- [0xffffffffffffffff] +++ killed by SIGSEGV +++
hmmm, that's crap, it just works for me. but that's probably because of: commit 04213a5dfc76db2caa821d41d1cb471fa3556707 Author: Fabian Groffen <grobian@gentoo.org> Date: Sat Apr 20 18:03:13 2019 +0200 libq/cache: rework to be vdb-like enumerator abstraction which isn't pushed yet.
this should be fixed in git now % ./qcache -c dev-perl -i x86 dev-perl/Alien-Gnuplot-1.33.0 ~x86 +amd64 dev-perl/Audio-DSP-0.20.0-r2 ~x86 +sparc +amd64 dev-perl/Business-Hours-0.120.0-r1 ~x86 +amd64 dev-perl/Business-ISBN-3.4.0 ~x86 ~arm +amd64 ~arm64 dev-perl/Business-ISBN-Data-20140910.3.0 ~x86 ~arm +amd64 ~arm64 dev-perl/Business-ISMN-1.132.0 ~x86 ~arm +amd64 ~arm64 dev-perl/Business-ISSN-1.3.0 ~x86 ~arm +amd64 ~arm64 dev-perl/CGI-FormBuilder-3.100.0 ~x86 +amd64 dev-perl/CGI-PSGI-0.150.0-r1 ~x86 +amd64 dev-perl/CSS-Minifier-XS-0.90.0-r1 ~x86 +amd64 dev-perl/CSS-Squish-0.100.0-r1 ~x86 +amd64 ~ppc dev-perl/Cairo-GObject-1.4.0-r1 ~x86 +amd64 dev-perl/Convert-Color-0.110.0 ~x86 +amd64 dev-perl/Crypt-DH-GMP-0.0.120 ~x86 +amd64 dev-perl/Data-ICal-0.220.0 ~x86 +amd64 dev-perl/Data-Page-2.20.0-r1 ~x86 ~ppc-aix +amd64 ~x86-solaris dev-perl/Data-Page-Pageset-1.20.0-r1 ~x86 +amd64 dev-perl/DateTime-Calendar-Julian-0.40.0 ~x86 ~arm +amd64 ~arm64 dev-perl/DateTime-Format-Natural-1.50.0 ~x86 +amd64 dev-perl/Devel-CallChecker-0.8.0 ~x86 ~ppc-macos ~amd64-linux ~sparc64-solaris ~x86-macos ~ppc-aix ~hppa ~alpha ~x64-solaris ~arm ~m68k-mint ~sparc-solaris ~ia64 ~sparc +amd64 ~x86-linux ~sh ~x86-solaris ~s390 ~x64-macos dev-perl/Devel-SmallProf-2.20.0-r3 ~x86 +sparc +amd64 dev-perl/DynaLoader-Functions-0.3.0 ~x86 ~ppc-macos ~amd64-linux ~sparc64-solaris ~x86-macos ~ppc-aix ~hppa ~alpha ~x64-solaris ~arm ~m68k-mint ~sparc-solaris ~ia64 ~sparc +amd64 ~x86-linux ~sh ~x86-solaris ~s390 ~x64-macos dev-perl/Encode-JIS2K-0.30.0 ~x86 ~arm +amd64 ~arm64 dev-perl/Feed-Find-0.70.0-r2 ~x86 +amd64 dev-perl/File-Slurp-Unicode-0.7.1-r1 ~x86 ~arm +amd64 dev-perl/Gtk3-0.34.0 ~x86 +amd64 dev-perl/HTML-Mason-PSGIHandler-0.530.0 ~x86 +amd64 dev-perl/HTML-Quoted-0.40.0 ~x86 +amd64 dev-perl/HTML-RewriteAttributes-0.50.0-r1 ~x86 +amd64 dev-perl/IO-Handle-Util-0.10.0 ~x86 ~ppc64 ~alpha ~ia64 ~sparc +amd64 ~ppc dev-perl/JavaScript-Minifier-XS-0.110.0-r1 ~x86 +amd64 dev-perl/LWP-UserAgent-Cached-0.60.0-r1 ~x86 +amd64 dev-perl/LWPx-ParanoidAgent-1.120.0 ~x86 +amd64 dev-perl/Lingua-Translit-0.280.0 ~x86 ~arm +amd64 ~arm64 dev-perl/Lirc-Client-2.20.0 ~x86 +amd64 dev-perl/List-UtilsBy-0.100.0 ~x86 ~ppc-macos ~amd64-linux ~sparc64-solaris ~x86-macos ~ppc64 ~ppc-aix ~m68k ~hppa ~alpha ~x64-solaris ~arm ~amd64-fbsd ~m68k-mint ~sparc-solaris ~ia64 ~sparc +amd64 ~x86-linux ~sh ~x86-solaris ~ppc ~arm64 ~s390 ~x64-macos dev-perl/MediaWiki-API-0.410.0 ~x86 ~x86-fbsd ~ppc64 ~hppa ~alpha ~arm ~ia64 ~sparc +amd64 ~ppc dev-perl/Modem-Vgetty-0.30.0-r2 ~x86 +amd64 dev-perl/Mojolicious-7.300.0 ~x86 +amd64 dev-perl/MooseX-Params-Validate-0.210.0 ~x86 +amd64 dev-perl/MouseX-Types-0.60.0-r1 ~x86 +hppa +amd64 ~ppc dev-perl/Net-OpenID-Common-1.200.0 ~x86 +amd64 dev-perl/Net-OpenID-Consumer-1.180.0 ~x86 +amd64 dev-perl/Net-Server-Mail-0.230.0 ~x86 +amd64 ~ppc dev-perl/PDL-Graphics-Gnuplot-2.11.0-r1 ~x86 +amd64 dev-perl/PDL-Transform-Color-1.3.0 ~x86 +amd64 dev-perl/Parallel-Prefork-0.180.0 ~x86 +amd64 dev-perl/Params-Classify-0.15.0 ~x86 ~ppc-macos ~amd64-linux ~sparc64-solaris ~xdev-perl/Server-Starter-0.330.0 ~x86 +amd64 dev-perl/Signal-Mask-0.8.0 ~x86 +amd64 dev-perl/Sort-Key-1.330.0-r1 ~x86 ~arm +amd64 ~arm64 dev-perl/Starlet-0.310.0 ~x86 +amd64 dev-perl/Test-File-1.443.0 ~x86 +amd64 dev-perl/Test-Mock-LWP-Dispatch-0.80.0 ~x86 +amd64 dev-perl/Test-utf8-1.10.0 ~x86 +amd64 dev-perl/Text-CSV-Simple-1.0.0-r1 ~x86 ~ppc-macos ~amd64-linux ~x86-macos ~ppc-aix ~arm ~sparc-solaris ~mips +amd64 ~x86-linux dev-perl/Text-Markdown-1.0.31-r1 ~x86 +amd64 dev-perl/Text-Password-Pronounceable-0.300.0-r1 ~x86 +amd64 dev-perl/Text-Roman-3.500.0 ~x86 ~arm +amd64 ~arm64 dev-perl/Text-vFile-asData-0.80.0 ~x86 +amd64 dev-perl/Thread-SigMask-0.4.0 ~x86 +amd64 dev-perl/Tie-Cycle-1.225.0 ~x86 ~arm +amd64 ~arm64 dev-perl/URI-Fetch-0.130.0 ~x86 +amd64 dev-perl/XML-Feed-0.530.0-r1 ~x86 +amd64 dev-perl/XML-LibXML-Simple-0.990.0 ~x86 ~arm +amd64 ~arm64 dev-perl/XML-RSS-LibXML-0.310.500 ~x86 +amd64 dev-perl/aliased-0.340.0-r1 ~x86 ~ppc-macos ~amd64-linux ~sparc64-solaris ~x86-macos ~ppc-aix ~x64-solaris ~arm ~m68k-mint ~sparc-solaris +amd64 ~x86-linux ~x86-solaris ~ppc ~x64-macos dev-perl/asa-1.30.0-r1 ~x86 ~ppc64 ~alpha ~ia64 ~sparc +amd64 ~ppc
It not longer segv's for me, however, it now does nothing other than print help output. :/ None of the following do anything at all, other than produce the help message: qcache -a qcache -n qcache -a -v qcache -n -v Specifying any arch on the command line manages to skip the help output, but instead, produces nothing at all. strace is really not enlightening :/ Shall this be re-opened, or should I file a new bug?
Hm, stepping through with gdb gives very little help: gdb --args qcache -a (gdb) break main Breakpoint 1 at 0x7910: file main.c, line 784. (gdb) run Starting program: /usr/bin/qcache -a Breakpoint 1, main (argc=2, argv=0x7fffffffcf58) at main.c:784 784 { (gdb) n 786 warnout = stderr; (gdb) n 788 argv0 = argv[0]; (gdb) n 794 if (fstat(fileno(stdout), &st) != -1) (gdb) n 795 if (!isatty(fileno(stdout))) (gdb) n 797 if ((getenv("TERM") == NULL) || (strcmp(getenv("TERM"), "dumb") == 0)) (gdb) n 800 initialize_portage_env(); (gdb) n 801 optind = 0; (gdb) n 802 return q_main(argc, argv); (gdb) n Any non-null pointers to reference me to? :)
qcache_main (argc=2, argv=0x7fffffffcf58) at qcache.c:732 732 { (gdb) n 739 while ((i = GETOPT_LONG(QCACHE, qcache, "")) != -1) { (gdb) n 740 switch (i) { (gdb) n 749 if (action) (gdb) n 752 action = i; (gdb) n 753 break; (gdb) n 759 data.arch = NULL; (gdb) n 760 if (optind < argc) (gdb) n 763 if ((data.arch == NULL && action != 's') || optind + 1 < argc) (gdb) n 764 qcache_usage(EXIT_FAILURE); (gdb) n Hmmm....
Created attachment 574410 [details] gdb trace session (xz compressed) I've made a somewhat exhaustive trace with gdb of what it seems to do running: qcache -c dev-perl -a amd64 It does a lot of rim spinning but produces no output, and its not clear why. Hopefully there's something in this you can make sense of.
my suspicion is that current git head should get you going further
A little better: 512b29304c7ddb1ea979631937f8d93bb4a2392a w/ CFLAGS -O0 -ggdb Starting program: /usr/bin/qcache -c dev-perl -v -n amd64 cache: Failed to read keywords for dev-perl/Lab-Measurement-9999 cache: Failed to read keywords for dev-perl/Lab-Zhinst-9999 dev-perl/Mac-Pasteboard-0.3.0 ~ppc-macos ~x86-macos ~x64-macos dev-perl/Net-DNS-1.40.0 ~x86-fbsd ~m68k ~sh Program received signal SIGSEGV, Segmentation fault. 0x000055555558b4b9 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 254 if (cat_ctx->ctx->do_sort) { (gdb) thread apply all bt full Thread 1 (process 24602): #0 0x000055555558b4b9 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 pkg_ctx = 0x0 #1 0x00005555555856f9 in cache_next_pkg (cat_ctx=0x5555555d9c10) at cache.c:160 p = 0x5555555bef20 "\003" ctx = 0x5555555bef20 ret = 0x0 #2 0x0000555555587449 in cache_foreach_pkg_int (sroot=0x5555555b6260 "/", portdir=0x5555555becb0 "/usr/local/portage", callback=0x555555568dba <qcache_results_cb>, priv=0x7fffffffb6a0, filter=0x0, sort=true, catsortfunc=0x0, pkgsortfunc=0x555555567bf2 <qcache_vercmp>) at cache.c:521 ctx = 0x5555555bef20 cat_ctx = 0x5555555d9c10 pkg_ctx = 0x0 ret = 1 #3 0x000055555558751d in cache_foreach_pkg_sorted (sroot=0x5555555b6260 "/", portdir=0x5555555becb0 "/usr/local/portage", callback=0x555555568dba <qcache_results_cb>, priv=0x7fffffffb6a0, catsortfunc=0x0, pkgsortfunc=0x555555567bf2 <qcache_vercmp>) at cache.c:545 No locals. #4 0x0000555555569413 in qcache_traverse (func=0x555555567d19 <qcache_not>, priv=0x7fffffffb6a0) at qcache.c:725 ret = 1 n = 14 overlay = 0x5555555becb0 "/usr/local/portage" data = 0x7fffffffb6a0 #5 0x0000555555569994 in qcache_main (argc=6, argv=0x7fffffffd908) at qcache.c:800 i = -1 action = 110 'n' data = {qatom = 0x5555555bed80, lastatom = 0x5555555d46e0, keywordsbuf = 0x5555555b9410, keywordsbuflen = 33, arch = 0x7fffffffdd09 "amd64", runfunc = 0x555555567d19 <qcache_not>} pkg = 0x0 cat = 0x7fffffffdcfa "dev-perl" __func__ = "qcache_main" #6 0x000055555556678a in q_main (argc=6, argv=0x7fffffffd908) at q.c:92 i = 0 install = 0 p = 0x7fffffffdcf0 "qcache" func = 0x555555569454 <qcache_main> reinitialize_metacache = 0 reinitialize = 0 __func__ = "q_main" #7 0x000055555556648b in main (argc=6, argv=0x7fffffffd908) at main.c:802 st = {st_dev = 21, st_ino = 13, st_nlink = 1, st_mode = 8592, st_uid = 1000, st_gid = 5, __pad0 = 0, st_rdev = 34826, st_size = 0, st_blksize = 1024, st_blocks = 0, st_atim = {tv_sec = 1556559008, tv_nsec = 616911709}, st_mtim = {tv_sec = 1556559008, tv_nsec = 616911709}, st_ctim = {tv_sec = 1555321065, tv_nsec = 616911709}, __glibc_reserved = {0, 0, 0}}
Is this a git tree or something you're running on?
/etc/portage/repos.conf/local.conf [local-portage] priority = 100 location = /usr/local/portage sync-type = auto-sync = no Its a "work tree", I symlink stuff I need to have installed immediately from a git checkout of ::gentoo, that way I don't need the full git ::gentoo in portage, and can use rsync as the baseline (eg: I'll use git checkout to resurrect removed ebuilds, and then symlink the dir into place). Though currently, there's no symlinks, just regular stuff that is hand maintained stuff. There are incidentally a *lot* of empty categories in there due to how my symlink/reap symlink approach works, and that might be partially to blame. Poking deep also discovers that for some reason I have a *file* "sys-apps/typescript", and I don't know where it came from: /usr/local/portage # cat sys-apps/typescript Script started on Fri 06 Jan 2017 22:15:46 NZDT /dev/pts/37 But I can't experimentally see tweaking any of these facts resolves any issue. find /usr/local/portage/ -type d -empty | grep -v '.git' | wc -l 163 find /usr/local/portage/ -not "-(" -type d -empty "-)" | grep -v '.git' | wc -l 31 find /usr/local/portage/ -not "-(" -type d -empty "-)" -and -type f | grep -v '.git' /usr/local/portage/app-emulation/packer-bin/Manifest /usr/local/portage/app-emulation/packer-bin/packer-bin-0.7.5.ebuild /usr/local/portage/app-emulation/vagrant-bin/vagrant-bin-1.7.2.ebuild /usr/local/portage/app-emulation/vagrant-bin/Manifest /usr/local/portage/sys-apps/typescript /usr/local/portage/dev-libs/udis86/udis86-1.7-r2.ebuild /usr/local/portage/dev-libs/udis86/files/udis86-1.7-yasm.patch /usr/local/portage/dev-libs/udis86/Manifest /usr/local/portage/dev-libs/udis86/udis86-1.7.2.ebuild /usr/local/portage/test/sandbox-path/files/mkscript.pl /usr/local/portage/test/sandbox-path/Manifest /usr/local/portage/test/sandbox-path/sandbox-path-0.01.ebuild /usr/local/portage/profiles/categories /usr/local/portage/profiles/eapi /usr/local/portage/profiles/repo_name /usr/local/portage/metadata/layout.conf for i in $( find /usr/local/portage/ -not "-(" -type d -empty "-)" -and -type f | grep -v '.git' ); do /usr/bin/stat $i; done File: /usr/local/portage/app-emulation/packer-bin/Manifest Size: 1160 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1054529 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2018-09-15 20:43:47.717448353 +1200 Modify: 2015-04-18 17:55:50.147750518 +1200 Change: 2016-03-02 22:58:14.559243129 +1300 Birth: 2015-04-18 17:55:50.147750518 +1200 File: /usr/local/portage/app-emulation/packer-bin/packer-bin-0.7.5.ebuild Size: 701 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1054531 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-27 08:34:06.733319074 +1200 Modify: 2015-04-18 17:55:47.456769812 +1200 Change: 2016-03-02 22:58:14.559243129 +1300 Birth: 2015-04-18 17:55:47.455769819 +1200 File: /usr/local/portage/app-emulation/vagrant-bin/vagrant-bin-1.7.2.ebuild Size: 594 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1054522 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-27 08:34:06.844319218 +1200 Modify: 2015-04-18 15:06:59.068800607 +1200 Change: 2016-03-02 22:58:14.559243129 +1300 Birth: 2015-04-18 15:06:59.068800607 +1200 File: /usr/local/portage/app-emulation/vagrant-bin/Manifest Size: 1153 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1051462 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2018-11-20 16:29:41.127246079 +1300 Modify: 2015-04-18 16:36:48.263247961 +1200 Change: 2016-03-02 22:58:14.571243009 +1300 Birth: 2015-04-18 16:36:48.263247961 +1200 File: /usr/local/portage/sys-apps/typescript Size: 61 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1048805 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2017-01-06 22:15:39.310569277 +1300 Modify: 2017-01-06 22:15:46.484550814 +1300 Change: 2017-01-06 22:15:46.484550814 +1300 Birth: 2017-01-06 22:15:39.310569277 +1300 File: /usr/local/portage/dev-libs/udis86/udis86-1.7-r2.ebuild Size: 960 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1050012 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-27 08:34:07.007319428 +1200 Modify: 2014-01-29 16:30:59.275448867 +1300 Change: 2016-03-02 22:58:16.583222883 +1300 Birth: 2014-01-29 16:30:59.249448971 +1300 File: /usr/local/portage/dev-libs/udis86/files/udis86-1.7-yasm.patch Size: 907 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1053855 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2016-05-01 05:15:23.874571104 +1200 Modify: 2014-01-29 16:31:07.836414736 +1300 Change: 2016-03-02 22:58:16.597222743 +1300 Birth: 2014-01-29 16:31:07.808414848 +1300 File: /usr/local/portage/dev-libs/udis86/Manifest Size: 1890 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1054027 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2016-05-01 05:15:23.855571149 +1200 Modify: 2014-01-29 16:32:17.234139392 +1300 Change: 2016-03-02 22:58:16.597222743 +1300 Birth: 2014-01-29 16:32:17.234139392 +1300 File: /usr/local/portage/dev-libs/udis86/udis86-1.7.2.ebuild Size: 928 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1054128 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-27 08:34:07.014319437 +1200 Modify: 2014-01-29 16:32:01.286202459 +1300 Change: 2016-03-02 22:58:16.598222733 +1300 Birth: 2014-01-29 16:32:01.286202459 +1300 File: /usr/local/portage/test/sandbox-path/files/mkscript.pl Size: 526 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1055011 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2015-05-01 01:09:20.394090555 +1200 Modify: 2015-05-01 01:09:20.333090537 +1200 Change: 2016-03-02 22:58:16.633222383 +1300 Birth: 2015-05-01 01:09:20.333090537 +1200 File: /usr/local/portage/test/sandbox-path/Manifest Size: 383 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1055008 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2015-09-09 17:53:24.916064282 +1200 Modify: 2015-05-01 00:40:29.051683847 +1200 Change: 2016-03-02 22:58:16.633222383 +1300 Birth: 2015-05-01 00:40:29.051683847 +1200 File: /usr/local/portage/test/sandbox-path/sandbox-path-0.01.ebuild Size: 623 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1055010 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-27 08:34:07.073319513 +1200 Modify: 2015-05-01 01:07:20.610055447 +1200 Change: 2016-03-02 22:58:16.633222383 +1300 Birth: 2015-05-01 01:07:20.610055447 +1200 File: /usr/local/portage/profiles/categories Size: 34 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1054197 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-29 22:30:23.279138232 +1200 Modify: 2017-02-20 21:30:09.480972799 +1300 Change: 2017-02-20 21:30:09.480972799 +1300 Birth: 2015-05-01 00:20:25.832907996 +1200 File: /usr/local/portage/profiles/eapi Size: 2 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1053674 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-29 22:30:16.338140895 +1200 Modify: 2016-05-09 09:37:46.840894011 +1200 Change: 2016-05-09 09:37:46.840894011 +1200 Birth: 2016-05-09 09:37:46.840894011 +1200 File: /usr/local/portage/profiles/repo_name Size: 14 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1085811 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-29 22:30:16.320140902 +1200 Modify: 2014-01-05 23:13:21.736281139 +1300 Change: 2016-03-02 22:58:16.633222383 +1300 Birth: 2014-01-05 23:13:21.736281139 +1300 File: /usr/local/portage/metadata/layout.conf Size: 465 Blocks: 8 IO Block: 4096 regular file Device: 806h/2054d Inode: 1085812 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/ kent) Gid: ( 1000/ kent) Access: 2019-04-29 22:30:16.368140882 +1200 Modify: 2015-05-01 00:44:36.843738930 +1200 Change: 2016-03-02 22:58:16.659222122 +1300 Birth: 2014-01-05 23:12:42.720325328 +1300
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=737cbc7da8473b7f2cd98b18201bf438c7e6c8c7 commit 737cbc7da8473b7f2cd98b18201bf438c7e6c8c7 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-04-30 07:52:27 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-04-30 07:52:27 +0000 libq/cache: fix some issues (likely bugs #684252 and #684476) cache_next_pkg: fix package traversal and cat_ctx shoveling cache_read_file_ebuild: fix key parsing (not to loop forever) cache_pkg_read: open correct file for ebuild case Bug: https://bugs.gentoo.org/684252 Bug: https://bugs.gentoo.org/684476 Signed-off-by: Fabian Groffen <grobian@gentoo.org> libq/cache.c | 37 ++++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=262ec2d23a8819c36b6983971dfead24bdd6cbee commit 262ec2d23a8819c36b6983971dfead24bdd6cbee Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-04-30 08:00:59 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-04-30 08:00:59 +0000 qcache: suppress keywordless ebuilds in -n mode Only return matches for ebuilds that actually have *any* keywords. Bug: https://bugs.gentoo.org/684252 Signed-off-by: Fabian Groffen <grobian@gentoo.org> qcache.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
I switched development to Linux now, what a funny platform (crash was due to running out of filedescriptors, guess I should've closed them) ;) This is on a git tree: % ./q cache -c dev-perl -v -n amd64 dev-perl/Cisco-IPPhone-0.50.0-r2 ~x86 dev-perl/Data-Serializer-0.600.0-r1 ~x86 dev-perl/HTML-StripScripts-1.60.0 ~x86 dev-perl/HTML-StripScripts-Parser-1.30.0 ~x86 cache: Failed to read keywords for dev-perl/Lab-Measurement-3.631 cache: Failed to read keywords for dev-perl/Lab-Measurement-3.661 cache: Failed to read keywords for dev-perl/Lab-Measurement-9999 cache: Failed to read keywords for dev-perl/Lab-Zhinst-0.50.0 cache: Failed to read keywords for dev-perl/Lab-Zhinst-1.20.0 cache: Failed to read keywords for dev-perl/Lab-Zhinst-0.30.0 cache: Failed to read keywords for dev-perl/Lab-Zhinst-9999 dev-perl/Mac-Pasteboard-0.3.0 ~ppc-macos ~x64-macos dev-perl/Math-NumberCruncher-5.0.0-r1 ~x86 dev-perl/Net-DNS-1.40.0 ~m68k ~sh dev-perl/Time-Format-1.120.0-r1 ~x86 (drop the -v to omit the fugly failure warnings) Hope this is more like what you expect.
previous paste included a bug, now that's fixed it looks like ./q cache -c dev-perl -n amd64 dev-perl/Mac-Pasteboard-0.3.0 ~ppc-macos ~x86-macos ~x64-macos dev-perl/Net-DNS-1.40.0 ~x86-fbsd ~m68k ~sh (which makes git tree match rsync tree where md5-cache is used)
Odd. Now strace gets as far as this: openat(6, "Mixin-ExtraFields-Param-0.21.0.ebuild", O_RDONLY|O_CLOEXEC) = 7 fcntl(7, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE) fstat(7, {st_mode=S_IFREG|0644, st_size=489, ...}) = 0 fstat(7, {st_mode=S_IFREG|0644, st_size=489, ...}) = 0 read(7, "# Copyright 1999-2015 Gentoo Fou"..., 4096) = 489 And then strace output halts indefinitely. Looks like it gets stuck forever in the loop: while (*p != '\0' && *p != *q) at cache.c:424 (gdb) run Starting program: /usr/bin/qcache -c dev-perl -v -n amd64 cache: Failed to read keywords for dev-perl/Lab-Measurement-9999 cache: Failed to read keywords for dev-perl/Lab-Zhinst-9999 ^C Program received signal SIGINT, Interrupt. 0x00005555555870cb in cache_read_file_ebuild (pkg_ctx=0x5555558d5b60) at cache.c:430 430 if (r != q && (p - 1 - r) % 2 == 1) (gdb) bt full #0 0x00005555555870cb in cache_read_file_ebuild (pkg_ctx=0x5555558d5b60) at cache.c:430 f = 0x5555555b68b0 s = {st_dev = 2054, st_ino = 32507346, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0, st_rdev = 0, st_size = 489, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1556736687, tv_nsec = 112377649}, st_mtim = {tv_sec = 1505615556, tv_nsec = 92349842}, st_ctim = {tv_sec = 1505615556, tv_nsec = 92349842}, __glibc_reserved = {0, 0, 0}} ret = 0x5555558d84a0 len = 666 p = 0x5555558d8631 "\"param\\\" method\"\nSLOT=\"0\"\nKEYWORDS=\"~amd64 ~x86\"\nIUSE=\"test\"\nRDEPEND=\"\n\tvirtual/perl-Carp\n\t>=dev-perl/Mixin-ExtraFields-0.2.0\n\tvirtual/perl-parent\n\"\nDEPEND=\"\n\t${RDEPEND}\n\t>=virtual/perl-ExtUtils-MakeM"... q = 0x5555558d860c "\"make your class provide a familiar \\\"param\\\" method\"\nSLOT=\"0\"\nKEYWORDS=\"~amd64 ~x86\"\nIUSE=\"test\"\nRDEPEND=\"\n\tvirtual/perl-Carp\n\t>=dev-perl/Mixin-ExtraFields-0.2.0\n\tvirtual/perl-parent\n\"\nDEPEND=\"\n\t${RD"... r = 0x5555558d862f " \\\"param\\\" method\"\nSLOT=\"0\"\nKEYWORDS=\"~amd64 ~x86\"\nIUSE=\"test\"\nRDEPEND=\"\n\tvirtual/perl-Carp\n\t>=dev-perl/Mixin-ExtraFields-0.2.0\n\tvirtual/perl-parent\n\"\nDEPEND=\"\n\t${RDEPEND}\n\t>=virtual/perl-ExtUtils-Mak"... key = 0x5555558d84e0 findnl = true #1 0x0000555555587364 in cache_pkg_read (pkg_ctx=0x5555558d5b60) at cache.c:491 ctx = 0x5555555bd880 #2 0x0000555555568f37 in qcache_results_cb (pkg_ctx=0x5555558d5b60, priv=0x7ffffffface0) at qcache.c:630 keywords = 0x5555555bd650 data = 0x7ffffffface0 buf = "dev-perl/Mixin-ExtraFields-Param-0.21.0\000\000\061\000\000\000\000\000ML-0.60.0\000\062.110\000ve-0.1.0-r1\000\377\377\177\000\000\000\000\000\000\000\000\000\000\253\261YU", '\000' <repeats 20 times>, "\377\377\377\377\377\377\377\377\000\227Zdž\307\027\220\006\000\000\000\000\000\000\000\340\241\377\377\377\177\000\000С\377\377\377\177\000\000\t\000\000\000\000\000\000\000 \243\377\377\377\177\000\000\261\261YUUU\000\000 \244\326\367\377\177\000\000B\277\300\367\377\177\000\000\000\000\000\000\000\000\000\000\260\235"... patom = 0x5555555d47d0 meta = 0x5555558d8180 ret = 1 #3 0x000055555558755e in cache_foreach_pkg_int (sroot=0x5555555b6260 "/", portdir=0x5555555bda20 "/usr/local/perl-overlay", callback=0x555555568dd9 <qcache_results_cb>, priv=0x7ffffffface0, filter=0x0, sort=true, catsortfunc=0x0, pkgsortfunc=0x555555567bac <qcache_vercmp>) at cache.c:541 ctx = 0x5555555bd920 cat_ctx = 0x5555555e8d50 pkg_ctx = 0x5555558d5b60 ret = 0 #4 0x000055555558764d in cache_foreach_pkg_sorted (sroot=0x5555555b6260 "/", portdir=0x5555555bda20 "/usr/local/perl-overlay", callback=0x555555568dd9 <qcache_results_cb>, priv=0x7ffffffface0, catsortfunc=0x0, pkgsortfunc=0x555555567bac <qcache_vercmp>) at cache.c:564 No locals. #5 0x0000555555569432 in qcache_traverse (func=0x555555567cd3 <qcache_not>, priv=0x7ffffffface0) at qcache.c:734 ret = 1 n = 9 overlay = 0x5555555bda20 "/usr/local/perl-overlay" data = 0x7ffffffface0 #6 0x00005555555699b3 in qcache_main (argc=6, argv=0x7fffffffcf38) at qcache.c:809 i = -1 action = 110 'n' data = {qatom = 0x5555555bed80, lastatom = 0x0, keywordsbuf = 0x5555555bd650, keywordsbuflen = 3, arch = 0x7fffffffd46f "amd64", runfunc = 0x555555567cd3 <qcache_not>} pkg = 0x0 cat = 0x7fffffffd460 "dev-perl" __func__ = "qcache_main" #7 0x0000555555566776 in q_main (argc=6, argv=0x7fffffffcf38) at q.c:90 i = 0 install = 0 p = 0x7fffffffd456 "qcache" func = 0x555555569473 <qcache_main> __func__ = "q_main" #8 0x000055555556648b in main (argc=6, argv=0x7fffffffcf38) at main.c:802 st = {st_dev = 21, st_ino = 17, st_nlink = 1, st_mode = 8592, st_uid = 1000, st_gid = 5, __pad0 = 0, st_rdev = 34830, st_size = 0, st_blksize = 1024, st_blocks = 0, st_atim = {tv_sec = 1556737033, tv_nsec = 827127620}, st_mtim = {tv_sec = 1556737033, tv_nsec = 827127620}, st_ctim = {tv_sec = 1556674479, tv_nsec = 827127620}, __glibc_reserved = {0, 0, 0}} The offending ebuild is quite simple: # Copyright 1999-2015 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 EAPI=5 MODULE_AUTHOR=RJBS MODULE_VERSION=0.021 inherit perl-module DESCRIPTION="make your class provide a familiar \"param\" method" SLOT="0" KEYWORDS="~amd64 ~x86" IUSE="test" RDEPEND=" virtual/perl-Carp >=dev-perl/Mixin-ExtraFields-0.2.0 virtual/perl-parent " DEPEND=" ${RDEPEND} >=virtual/perl-ExtUtils-MakeMaker-6.300.0 test? ( >=virtual/perl-Test-Simple-0.960.0 ) " So ... I'd guess there's a bug in your string parsing :)
> So ... I'd guess there's a bug in your string parsing :) I'm quite certain it's a bug in my escape processing :D (thanks for continuing to persue this)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=b6866cb7cbe9172ad2ac46e5e72b71901ec44841 commit b6866cb7cbe9172ad2ac46e5e72b71901ec44841 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-05-02 15:16:16 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-05-02 15:16:16 +0000 libq/cache: fix handling of (escaped) quotes in cache_read_file_ebuild Bug: https://bugs.gentoo.org/684252 Signed-off-by: Fabian Groffen <grobian@gentoo.org> libq/cache.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Starting program: /usr/bin/qkeyword -c dev-perl -s Program received signal SIGSEGV, Segmentation fault. 0x000055555558c731 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 254 if (cat_ctx->ctx->do_sort) { (gdb) bt full #0 0x000055555558c731 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 pkg_ctx = 0x0 #1 0x0000555555586844 in cache_next_pkg (cat_ctx=0x5555555fd950) at cache.c:165 p = 0x5555555be920 "\003" ctx = 0x5555555be920 ret = 0x0 #2 0x00005555555886c1 in cache_foreach_pkg_int (sroot=0x5555555b7260 "/", portdir=0x5555555bfcb0 "/usr/local/portage", callback=0x55555556f210 <qkeyword_results_cb>, priv=0x7fffffffacf0, filter=0x0, sort=true, catsortfunc=0x0, pkgsortfunc=0x55555556dfe3 <qkeyword_vercmp>) at cache.c:559 ctx = 0x5555555be920 cat_ctx = 0x5555555fd950 pkg_ctx = 0x0 ret = 1 #3 0x0000555555588795 in cache_foreach_pkg_sorted (sroot=0x5555555b7260 "/", portdir=0x5555555bfcb0 "/usr/local/portage", callback=0x55555556f210 <qkeyword_results_cb>, priv=0x7fffffffacf0, catsortfunc=0x0, pkgsortfunc=0x55555556dfe3 <qkeyword_vercmp>) at cache.c:583 No locals. #4 0x000055555556f869 in qkeyword_traverse (func=0x55555556e71f <qkeyword_stats>, priv=0x7fffffffacf0) at qkeyword.c:734 ret = 1 n = 15 overlay = 0x5555555bfcb0 "/usr/local/portage" data = 0x7fffffffacf0 #5 0x000055555556fd99 in qkeyword_main (argc=4, argv=0x7fffffffcf48) at qkeyword.c:806 i = -1 action = 115 's' data = {qatom = 0x5555555bfd80, lastatom = 0x0, keywordsbuf = 0x5555555be650, keywordsbuflen = 3, arch = 0x555555595c5c "amd64", runfunc = 0x55555556e71f <qkeyword_stats>} pkg = 0x0 cat = 0x7fffffffd459 "dev-perl" __func__ = "qkeyword_main" #6 0x0000555555566786 in q_main (argc=4, argv=0x7fffffffcf48) at q.c:90 i = 0 install = 0 p = 0x7fffffffd44d "qkeyword" func = 0x55555556f8aa <qkeyword_main> __func__ = "q_main" #7 0x000055555556649b in main (argc=4, argv=0x7fffffffcf48) at main.c:802 st = {st_dev = 21, st_ino = 14, st_nlink = 1, st_mode = 8592, st_uid = 1000, st_gid = 5, __pad0 = 0, 4827, st_size = 0, st_blksize = 1024, st_blocks = 0, st_atim = {tv_sec = 1556874344, tv_nsec = 223845745}, st_mtim = { tv_sec = 1556874344, tv_nsec = 223845745}, st_ctim = {tv_sec = 1555527841, tv_nsec = 223845745}, __glibc_reserved = {0, 0, 0}} Any specific context that would help you make sense of this one? Fwiw, strace's last IO indicates it hitting my earlier mentioned "possibly a problem" case with a random file inside a category: openat(4, "sec-policy", O_RDONLY|O_CLOEXEC) = 5 fstat(5, {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0 fcntl(5, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE) fcntl(5, F_SETFD, FD_CLOEXEC) = 0 openat(5, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6 fstat(6, {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0 getdents64(6, /* 2 entries */, 32768) = 48 getdents64(6, /* 0 entries */, 32768) = 0 close(6) = 0 close(5) = 0 openat(4, "sys-apps", O_RDONLY|O_CLOEXEC) = 5 fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 fcntl(5, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE) fcntl(5, F_SETFD, FD_CLOEXEC) = 0 openat(5, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6 fstat(6, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 getdents64(6, /* 3 entries */, 32768) = 80 getdents64(6, /* 0 entries */, 32768) = 0 close(6) = 0 openat(5, "typescript", O_RDONLY|O_CLOEXEC) = 6 fstat(6, {st_mode=S_IFREG|0644, st_size=61, ...}) = 0 close(6) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x18} --- +++ killed by SIGSEGV +++ Segmentation fault I'm not *entirely* sure why iteration is traversing sys-apps when passed `-c dev-perl`, but if `-s` is supposed to return context relative to `dev-perl`, then that seems like an inefficiency that could do with some optimization. ( I wasted a *LOT* of time attempting to get ltrace context for this due to all the events related to this seemingly unrelated activity ... actually, its sill running, I waited to post this comment for an hour while it went, its still going, I'll comment with anything relevant when I wake up in the morning, I'm going to bed xD )
(In reply to Kent Fredric (IRC: kent\n) from comment #18) > Starting program: /usr/bin/qkeyword -c dev-perl -s > > Program received signal SIGSEGV, Segmentation fault. > 0x000055555558c731 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 > 254 if (cat_ctx->ctx->do_sort) { > Any specific context that would help you make sense of this one? Will have a look first. Important for me though, what commit rev are you on? > I'm not *entirely* sure why iteration is traversing sys-apps when passed `-c > dev-perl`, but if `-s` is supposed to return context relative to `dev-perl`, > then that seems like an inefficiency that could do with some optimization. Indeed, there is a TODO item for this. In many cases we do filtering on an atom, and for that we shouldn't traverse categories that itself don't match. The (original) code however doesn't care about this. I guess this might have been the reason for introducing a flat cache, because a "grep" through a file is in that case more efficient than walking all directories. > ( I wasted a *LOT* of time attempting to get ltrace context for this due to > all the events related to this seemingly unrelated activity ... actually, > its sill running, I waited to post this comment for an hour while it went, > its still going, I'll comment with anything relevant when I wake up in the > morning, I'm going to bed xD ) I expect to plug in this optimisation soon, it's just some work because all callers need to start making use of it, when they can, and there may be multiple queries which ideally are handled in a single tree-walk (as is now).
(In reply to Kent Fredric (IRC: kent\n) from comment #18) > Fwiw, strace's last IO indicates it hitting my earlier mentioned "possibly a > problem" case with a random file inside a category: Yes it is, thanks. One cannot open a file as if it were a directory, apparently. I'll add a check for that.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=ab51b5e60db69f6ee6f2513fef7ce12ebc01af92 commit ab51b5e60db69f6ee6f2513fef7ce12ebc01af92 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-05-03 11:44:40 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-05-03 11:44:40 +0000 libq/cache: fix crash when a category contains a non-directory Bug: https://bugs.gentoo.org/684252 Signed-off-by: Fabian Groffen <grobian@gentoo.org> libq/cache.c | 5 +++++ 1 file changed, 5 insertions(+)
updating from commit: f95495b2a574c291caa6ce3564d32bb265ccd32c to commit: ab51b5e60db69f6ee6f2513fef7ce12ebc01af92 No change in the segfault: Starting program: /usr/bin/qkeyword -c dev-perl -s Program received signal SIGSEGV, Segmentation fault. 0x000055555558c739 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 254 if (cat_ctx->ctx->do_sort) { (gdb) bt full #0 0x000055555558c739 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 pkg_ctx = 0x0 #1 0x000055555558684c in cache_next_pkg (cat_ctx=0x5555555fd950) at cache.c:170 p = 0x5555555be920 "\003" ctx = 0x5555555be920 ret = 0x0 #2 0x00005555555886c9 in cache_foreach_pkg_int (sroot=0x5555555b7260 "/", portdir=0x5555555bfcb0 "/usr/local/portage", callback=0x55555556f210 <qkeyword_results_cb>, priv=0x7fffffffacf0, filter=0x0, sort=true, catsortfunc=0x0, pkgsortfunc=0x55555556dfe3 <qkeyword_vercmp>) at cache.c:564 ctx = 0x5555555be920 cat_ctx = 0x5555555fd950 pkg_ctx = 0x0 ret = 1 #3 0x000055555558879d in cache_foreach_pkg_sorted (sroot=0x5555555b7260 "/", portdir=0x5555555bfcb0 "/usr/local/portage", callback=0x55555556f210 <qkeyword_results_cb>, priv=0x7fffffffacf0, catsortfunc=0x0, pkgsortfunc=0x55555556dfe3 <qkeyword_vercmp>) at cache.c:588 No locals. #4 0x000055555556f869 in qkeyword_traverse (func=0x55555556e71f <qkeyword_stats>, priv=0x7fffffffacf0) at qkeyword.c:734 ret = 1 n = 15 overlay = 0x5555555bfcb0 "/usr/local/portage" data = 0x7fffffffacf0 #5 0x000055555556fd99 in qkeyword_main (argc=4, argv=0x7fffffffcf48) at qkeyword.c:806 i = -1 action = 115 's' data = {qatom = 0x5555555bfd80, lastatom = 0x0, keywordsbuf = 0x5555555be650, keywordsbuflen = 3, arch = 0x555555595c5c "amd64", runfunc = 0x55555556e71f <qkeyword_stats>} pkg = 0x0 cat = 0x7fffffffd459 "dev-perl" __func__ = "qkeyword_main" #6 0x0000555555566786 in q_main (argc=4, argv=0x7fffffffcf48) at q.c:90 i = 0 install = 0 p = 0x7fffffffd44d "qkeyword" func = 0x55555556f8aa <qkeyword_main> __func__ = "q_main" #7 0x000055555556649b in main (argc=4, argv=0x7fffffffcf48) at main.c:802 st = {st_dev = 21, st_ino = 14, st_nlink = 1, st_mode = 8592, st_uid = 1000, st_gid = 5, __pad0 = 0, st_rdev = 34827, st_size = 0, st_blksize = 1024, st_blocks = 0, st_atim = {tv_sec = 1556903752, tv_nsec = 223845745}, st_mtim = {tv_sec = 1556903752, tv_nsec = 223845745}, st_ctim = {tv_sec = 1555527841, tv_nsec = 223845745}, __glibc_reserved = {0, 0, 0}} However, strace gets further: openat(4, "sec-policy", O_RDONLY|O_CLOEXEC) = 5 fstat(5, {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0 fcntl(5, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE) fcntl(5, F_SETFD, FD_CLOEXEC) = 0 openat(5, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6 fstat(6, {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0 getdents64(6, /* 2 entries */, 32768) = 48 getdents64(6, /* 0 entries */, 32768) = 0 close(6) = 0 close(5) = 0 openat(4, "sys-apps", O_RDONLY|O_CLOEXEC) = 5 fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 fcntl(5, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE) fcntl(5, F_SETFD, FD_CLOEXEC) = 0 openat(5, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 6 fstat(6, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 getdents64(6, /* 3 entries */, 32768) = 80 getdents64(6, /* 0 entries */, 32768) = 0 close(6) = 0 openat(5, "typescript", O_RDONLY|O_CLOEXEC) = 6 fstat(6, {st_mode=S_IFREG|0644, st_size=61, ...}) = 0 close(6) = 0 close(5) = 0 openat(4, "sys-auth", O_RDONLY|O_CLOEXEC) = 5 fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 fcntl(5, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE) fcntl(5, F_SETFD, FD_CLOEXEC) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x18} --- +++ killed by SIGSEGV +++ Segmentation fault sys-auth is an empty directory if that helps. ltrace I left running last night crashed when it generated 1.7Gb of trace file (filling the disk), while still failing to progress as far as the problem.
This is daft, I can't reproduce this. Tried several trees, using empty cat. No way I can get it to crash on Linux (nor Solaris).
it seems impossible for cat_ctx to be NULL, are you sure you recompiled/installed?
(In reply to Fabian Groffen from comment #24) > it seems impossible for cat_ctx to be NULL, are you sure you > recompiled/installed? I tried again, and the `git checkout` step didn't need to update anything. *shrug* I'll need to find a good breakpoint to step through.
Program received signal SIGSEGV, Segmentation fault. 0x000055555558c739 in q_vdb_next_pkg (cat_ctx=0x0) at vdb.c:254 254 if (cat_ctx->ctx->do_sort) { (gdb) frame 1 #1 0x000055555558684c in cache_next_pkg (cat_ctx=0x5555555ca270) at cache.c:170 170 ret = q_vdb_next_pkg(ctx->ebuilddir_cat_ctx); (gdb) print cat_ctx $1 = (q_vdb_cat_ctx *) 0x5555555ca270 (gdb) print *cat_ctx $2 = {name = 0x5555555cadf3 "sys-auth", fd = 5, dir = 0x5555555c0e40, ctx = 0x5555555c0150, pkg_de = 0x0, pkg_cnt = 1, pkg_cur = 1} (gdb) print *cat_ctx->ctx $4 = {portroot_fd = 3, vdb_fd = 4, dir = 0x5555555f87b0, cat_de = 0x5555555f51c0, cat_cnt = 164, cat_cur = 132, catsortfunc = 0x7ffff7c6ec40 <alphasort64>, pkgsortfunc = 0x55555556dfe3 <qkeyword_vercmp>, do_sort = true, cachetype = CACHE_EBUILD, ebuilddir_pkg_ctx = 0x5555555ca360, ebuilddir_cat_ctx = 0x0, ebuilddir_ctx = 0x5555555d5ec0, repo = 0x5555555d9f30 "local-portage"} (gdb) print **cat_ctx->ctx->cat_de $11 = {d_ino = 1147311, d_off = 2129016523846326385, d_reclen = 40, d_type = 4 '\004', d_name = "app-accessibility\000\000\000\000A\000\000\000\000\000\000\000\063\371[UUU\000\000\000\000\000\000\000\000\000\000\060\237]UUU\000\000\300^]UUU\000\000\300\235[UUU\000\000\377\377\377\377\000\000\000\000\260\065]UUU\000\000A\000\000\000\000\000\000\000Sl]UUU\000\000\000\000\000\000\000\000\000\000\060\237]UUU\000\000\300^]UUU\000\000\300\235[UUU\000\000\377\377\377\377\000\000\000\000\260\065]UUU\000\000A\000\000\000\000\000\000\000s\251[UUU\000\000\000\000\000\000\000\000\000\000\060\237]UUU\000\000\300^]UUU\000\000\300\235[UUU\000\000\377\377\377\377"...} (gdb) print *cat_ctx->ctx->ebuilddir_pkg_ctx $12 = {name = 0x5555555ca133 "typescript", slot = 0x0, repo = 0x5555555d9f30 "local-portage", slot_len = 93824992763584, repo_len = 93824992676816, fd = -1, cat_ctx = 0x5555555ca270} (gdb) print *cat_ctx->ctx->ebuilddir_ctx $13 = {portroot_fd = -1, vdb_fd = 5, dir = 0x0, cat_de = 0x0, cat_cnt = 0, cat_cur = 0, catsortfunc = 0x7ffff7c6ec40 <alphasort64>, pkgsortfunc = 0x55555556dfe3 <qkeyword_vercmp>, do_sort = true, cachetype = CACHE_EBUILD, ebuilddir_pkg_ctx = 0x0, ebuilddir_cat_ctx = 0x0, ebuilddir_ctx = 0x0, repo = 0x5555555d9f30 "local-portage"} (gdb) Does this help? Its clear that in cache_next_pkg, cat_ctx -> ctx -> ebuilddir_cat_ctx is indeed 0x0
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=55a3b37772e4e4f70b3eaf76916f2b1450d26a6e commit 55a3b37772e4e4f70b3eaf76916f2b1450d26a6e Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-05-04 11:51:31 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-05-04 11:51:31 +0000 libq/cache: probably fix crash from bug #684252#c22 Ensure we renew the local ebuild dir context after we found a problem with the previous directory Bug: https://bugs.gentoo.org/684252 Signed-off-by: Fabian Groffen <grobian@gentoo.org> libq/cache.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
/usr/bin/qkeyword -c dev-perl -s +-------------------------+ | general statistics | +-------------------------+ | architectures | 3 | | categories | 3 | | packages | 2329 | | ebuilds | 2875 | +-------------------------+ +---------------------------------------------------+ | keyword distribution | +---------------------------------------------------+ | architecture | stable | ~arch | total | total/#pkgs | | | | only | | | +---------------------------------------------------+ | x86 | 1045 | 1238 | 2283 | 98.02% | | arm | 273 | 226 | 499 | 21.43% | | amd64 | 1107 | 1221 | 2328 | 99.96% | +---------------------------------------------------+ Completed in 0 seconds \o/
Is there a good reason why only arches x86, arm and amd64 do anything? Or is that a bug :) ( FTR: I'm particularly interested in minor arches, prefix, etc )
(In reply to Kent Fredric (IRC: kent\n) from comment #29) > Is there a good reason why only arches x86, arm and amd64 do anything? Or is > that a bug :) ( FTR: I'm particularly interested in minor arches, prefix, > etc ) at least it was FAST wasn't it? :) I think there's a problem with it handling overlays, do you have an overlay that by chance only has those keywords?
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=a6ca96154d2429bdc840454bdfa9fcb42c0e49a8 commit a6ca96154d2429bdc840454bdfa9fcb42c0e49a8 Author: Fabian Groffen <grobian@gentoo.org> AuthorDate: 2019-05-04 17:23:12 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2019-05-04 17:23:12 +0000 qkeyword: use combined set of arches found on all overlays Bug: https://bugs.gentoo.org/684252 Signed-off-by: Fabian Groffen <grobian@gentoo.org> qkeyword.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
qkeyword -c dev-perl -s +-------------------------+ | general statistics | +-------------------------+ | architectures | 33 | | categories | 3 | | packages | 2329 | | ebuilds | 2875 | +-------------------------+ +-------------------------------------------------------------+ | keyword distribution | +-------------------------------------------------------------+ | architecture | stable | ~arch | total | total/#pkgs | | | | only | | | +-------------------------------------------------------------+ | x86 | 1045 | 1238 | 2283 | 98.02% | | ppc-macos | 0 | 452 | 452 | 19.41% | | amd64-linux | 0 | 484 | 484 | 20.78% | | sparc64-solaris | 0 | 320 | 320 | 13.74% | | x86-fbsd | 0 | 258 | 258 | 11.08% | | x86-macos | 0 | 443 | 443 | 19.02% | | ppc64 | 499 | 99 | 598 | 25.68% | | arm64-linux | 0 | 0 | 0 | 0.00% | | ppc-aix | 0 | 365 | 365 | 15.67% | | m68k | 41 | 152 | 193 | 8.29% | | hppa | 454 | 57 | 511 | 21.94% | | alpha | 506 | 70 | 576 | 24.73% | | arm-linux | 0 | 0 | 0 | 0.00% | | x64-solaris | 0 | 333 | 333 | 14.30% | | x64-cygwin | 0 | 35 | 35 | 1.50% | | arm | 273 | 226 | 499 | 21.43% | | ppc64-linux | 0 | 0 | 0 | 0.00% | | amd64-fbsd | 0 | 160 | 160 | 6.87% | | m68k-mint | 0 | 292 | 292 | 12.54% | | sparc-solaris | 0 | 400 | 400 | 17.17% | | x86-winnt | 0 | 0 | 0 | 0.00% | | ia64 | 526 | 63 | 589 | 25.29% | | mips | 0 | 331 | 331 | 14.21% | | sparc | 563 | 74 | 637 | 27.35% | | amd64 | 1107 | 1221 | 2328 | 99.96% | | x86-linux | 0 | 486 | 486 | 20.87% | | sh | 65 | 208 | 273 | 11.72% | | x86-solaris | 0 | 457 | 457 | 19.62% | | x86-cygwin | 0 | 0 | 0 | 0.00% | | ppc | 684 | 208 | 892 | 38.30% | | arm64 | 88 | 244 | 332 | 14.26% | | s390 | 174 | 121 | 295 | 12.67% | | x64-macos | 0 | 357 | 357 | 15.33% | +-------------------------------------------------------------+ Completed in 6 seconds \o/
(In reply to Kent Fredric (IRC: kent\n) from comment #32) > qkeyword -c dev-perl -s ... > > Completed in 6 seconds Out of curiosity, I get ~6 seconds for 2 repos over NFS (one with metadata, the other without), what kind of repos are these?
Local, but on a nearly full spinning rust, a crap, thermally throttled CPU and never enough ram, always fighting with a web browser.
