eix shows wrong slot information for sci-libs/plasma-2.6.0 The ebuild uses inherit versionator ... SOVER=$(get_version_component_range 1) SLOT="0/${SOVER}" eix shows "Available versions: ~2.5.2 ~2.6.0(0/(get_version_component_range)" Reproducibility: always! Steps to reproduce: 1.) layman -a science 2.) eix-update 3.) eix sci-libs/plasma
Out of curiosity, if you do "egencache --repo=science --update" then redo eix-update (which will then be based on generated metadata cache), does eix output look better?
Interestingly, yes, this works. Running 1.) egencache --repo=science --update 2.) eix-update 3.) eix sci-libs/plasma now shows "Available versions: ~2.5.2 ~2.6.0(0/2) {doc examples fortran static-libs test}" egencache --help just shows "--update update metadata/md5-cache/ (generate as necessary)". I am not sure I understand what egencache is doing and how this is related to this bug.
When possible, eix bases its index on $repo/metadata/md5-cache which already has the ebuild logic all parsed out into the final vars (so the $(get_version_component_range) part is evaluated already). On an overlay w/o this cache, it has to read from the ebuilds directly, it's not advanced enough to parse complex eclass-generated vars, I guess.
Ah okay, thanks for your explanation, that makes sense.
By the way, it does not matter if I use the layman version or use the Github mirror (via /etc/portage/repos.conf/). And before I completely forget: emerge --info Portage 2.3.3 (python 3.4.5-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-4.9.4, glibc-2.23-r3, 4.9.6-gentoo-r1 x86_64) ================================================================= System uname: Linux-4.9.6-gentoo-r1-x86_64-Intel-R-_Core-TM-_i7-2670QM_CPU_@_2.20GHz-with-gentoo-2.3 KiB Mem: 8162684 total, 2519680 free KiB Swap: 9713660 total, 9713660 free Timestamp of repository gentoo: Tue, 07 Mar 2017 22:15:01 +0000 sh bash 4.3_p48-r1 ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1 app-shells/bash: 4.3_p48-r1::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.22.3_rc4::gentoo dev-lang/python: 2.7.12::gentoo, 3.4.5::gentoo dev-util/cmake: 3.7.2::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.3::gentoo sys-apps/openrc: 0.23.2::gentoo sys-apps/sandbox: 2.10-r3::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69::gentoo sys-devel/automake: 1.11.6-r1::gentoo, 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.25.1-r1::gentoo sys-devel/gcc: 4.9.4::gentoo sys-devel/gcc-config: 1.7.3::gentoo sys-devel/libtool: 2.4.6-r3::gentoo sys-devel/make: 4.2.1::gentoo sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers) sys-libs/glibc: 2.23-r3::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.at.gentoo.org/gentoo-portage priority: -1000 libreoffice-online location: /usr/local/libreoffice-online sync-type: git sync-uri: git://github.com/timeraider4u/libreoffice-online.git masters: gentoo maven-overlay location: /usr/local/maven-overlay sync-type: git sync-uri: git://github.com/timeraider4u/maven-overlay.git masters: gentoo myoverlay location: /usr/local/myoverlay sync-type: git sync-uri: git://github.com/timeraider4u/myoverlay.git masters: gentoo eclipse location: /var/lib/layman/eclipse sync-type: laymansync sync-uri: https://github.com/gentoo/eclipse-overlay.git masters: gentoo priority: 50 foo-overlay location: /var/lib/layman/foo-overlay sync-type: laymansync sync-uri: git://github.com/slashbeast/foo-overlay.git masters: gentoo priority: 50 fritteli location: /var/lib/layman/fritteli sync-type: laymansync sync-uri: https://gittr.ch/linux/gentoo-overlay.git masters: gentoo priority: 50 java location: /var/lib/layman/java sync-type: laymansync sync-uri: git://anongit.gentoo.org/proj/java.git masters: gentoo priority: 50 science location: /var/lib/layman/science sync-type: laymansync sync-uri: git://anongit.gentoo.org/proj/sci.git masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.3/conf" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=core2 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://gd.tuwien.ac.at/opsys/linux/gentoo/ http://gentoo.wheel.sk/" LANG="de_AT.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages/core2" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="S3TC X a52 aac aacplus acl acpi additions alsa amd64 android ant archive autotools avcodec avformat bash-completion berkdb bluetooth branding bzip2 cairo cdda cdparanoia cdr cdrdao chm cli cmake colordiff commonslogging commonsnet consolekit contrast contrib cracklib crash-reporter crashreporter crypt cryptsetup cups cxx dbus declarative dri dts dvd dvdr dvi dvipdfm eap-sim eap-tls ebook emboss encode epspdf epub exceptions exif expat ext4 extensions extra faac faad facebook fam fat ffmpeg firefox flac foomatic foomaticdb fortran g-sorcery gcj gdbm geolocation gif gimp git glamor gpm gpx-extensions graphics groupwise grub gstreamer gtk gui guidexml gzip hpijs icons iconv icu id3tag imap importers inotify ipc iso jpeg kate kde kipi kscreen kwallet lame lcms libnotify libtiger libxml2 lightning location log4j mad map marble matroska midi mime mmx mng modules mp3 mp4 mp4v2 mpeg mtp multilib musicbrainz mysql mysqli ncurses networkmanager nls normalize nptl nsplugin ntfs ntp odk ogg ogm okteta okular openclipart opengl openmp openrc openstreetmap opus osmesa pam pango pcre pdf pdfannotextractor pdftk pdo phonon pidgin pixmaps plasma playlist plugins pm-utils png policykit positioning postgis postscript ppds preview-latex printsupport prison privacy privacylists ps psd pstricks publishers qml qrcode qt3support qt4 qt5 quodlibet raw rdesktop readline regex regexp replaygain resolver rss rtc rtf s3tc scanner science sdl seccomp sendto servletapi session showdesktop skype slideshow smtp spatialite spell sqlite srt sse sse2 ssl startup-notification statistics subtitles subversion svg system-cairo system-crontab system-ffmpeg system-icu system-jpeg system-libs system-libvpx system-libyaml system-sqlite taglib tcpd teamarena templates terminal testutil themes theora thesaurus thumbnail thumbnails tiff timezone tinfo tomcat tray truetype udev udisks ukit uncommon-eap-types unicode upower urlpicpreview usb v4l valgrind vba vcard vcd vcdx vhosts video visio vkontakte vlc vnc vorbis vpx wallpapers wav waveform weather webdav webgl webm webpresence webstart widgets wifi wma wxwidgets x264 x265 xattr xcb xcomposite xetex xindy xinerama xiph xml xmlreader xscreensaver xslt xterm xv xvid zip zlib" ABI_X86="32 64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="synaptics evdev" KERNEL="linux" L10N="de en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" QEMU_SOFTMMU_TARGETS="x86_64 arm" QEMU_USER_TARGETS="x86_64 arm" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="nvidia nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
The bug is invalid: The cache method parse is unreliable, in principle. It is only meant as a secure fallback in the lack of other data or proper configuration. See section "SPEEDUP" on the manpage.
@Martin: Sorry, but your reply is just strange! In fact it seems just ridiculous to me. This bug is neither invalid nor non-existing. It is ALWAYS reproducible and I have provided detailed instructions on how to reproduce it. I have absolutely NO idea which parsing method your are talking about and as I am no developer - just an ordinary user - of eix I also do not care about how the parsing is done. In fact, I am just following the instructions at the wiki-page under https://wiki.gentoo.org/wiki/Eix Sorry for the rant, but I have taken the time to report an unexpected behaviour and, therefore, expect that somebody takes enough time to deal with my bug report in a proper way!
(In reply to Martin Väth from comment #6) > The bug is invalid: The cache method parse is unreliable, in principle. > It is only meant as a secure fallback in the lack of other data or proper > configuration. See section "SPEEDUP" on the manpage. If that method is unreliable, would it make sense to perhaps remove it? Doesn't seem to make sense to me to allow it to be used when it's unreliable, and instead just error when other data is lacking or configuration needs fixing...?
I think it's just a known limitation of eix that it cannot be expected to properly index uncached repos, and there are no plans to fix that. There is complex internal portage logic used to parse out the full ebuild into evaluated variables, fix this in eix would mean re-implementing all that. Or just tell overlay maintainers they should provide an md5-cache. Maybe a warning of some sort would help, to let users know that indexing an uncached overla produces unreliable results?
> This bug is neither invalid nor non-existing Previously, there existed the category "NOTABUG" which is what I mean by "INVALID": It is not a bug, because it is documented behaviour (although admittedly the eix documentation is hard to understand): The cache methods "parse" and "parse*" are a heuristic scanning which is sufficient for many purposes. At least, one gets package name and version number reliable, and usually also most other data (EAPI, keywords, description, license, homepage, use, dependencies) are correct unless they are set through "tricky" shell functions or eclasses. > If that method is unreliable, would it make sense to perhaps remove it? It certainly has its use cases, because it is very fast and needs no preprocessing (e.g. to get a quick overview over packages in some new overlay or for overlays for which it "works"). Moreover, it can be combined with other methods (see below). Also to enable it by default makes sense: The alternative would be that the packages are not shown at all (or at least without any data) which would be even more confusing than occassionally some wrong data when it is set in a "tricky" way. > fix this in eix would mean re-implementing all that. This cache method is implemented in eix in two similar flavors: ebuild and ebuild* However, it is not chosen as the default in eix for 3 reasons. 1. Security: Ebuilds are executed, and thus a malevolent overlay (or MITM if the overlay is not synced by a method which exclude this) can do practically any damage to your system which he can do with the reduced permissions (which is quite a lot, including "practically" an installation of a rootkit - I will not go into detail here) 2. It is extremely slow. 3. Security ;) It is certainly not a good idea to choose this as a default fallback, because this can give users much worse surprises than the displaying of obviously wrong data. Personally, I use for my local overlay the method parse|ebuild*: This needs no preprocessing for most ebuilds, and for those ebuilds for which parse has "doubts" about the correctness of the data, the slower method "ebuild" is chosen: In this case, this is a not a security issue, because I trust of course the ebuilds in my local overlay. In general, the only good recommendation which can be given to the user is to get the metadata in some other way: either directly from the overlay maintainer or through a more dedicated tool like egencache (though currently, the latter is not more secure than the ebuild cache method of eix, but at least parallel and faster). Hoewver, it is outside of the scope of eix to do this
Ah okay thank you Martin for the clarification, now I understand your argument and agree with you that the behaviour of eix is documented (at least in the man-page. A hint for the unreliable cache may also be a good idea for the wiki-page). Anyway, I am still not fully convinced about marking the bug as resolved yet, as the issue still remains. Any ideas on how to continue to get the issue solved? Should I open a separate bug which would suggest that the people from $overlay add the cache-updating somehow to their work-flow? Any other plans?
(In reply to Harald Weiner from comment #11) > Ah okay thank you Martin for the clarification, now I understand your > argument and agree with you that the behaviour of eix is documented (at > least in the man-page. A hint for the unreliable cache may also be a good > idea for the wiki-page). Anyway, I am still not fully convinced about > marking the bug as resolved yet, as the issue still remains. Any ideas on > how to continue to get the issue solved? Should I open a separate bug which > would suggest that the people from $overlay add the cache-updating somehow > to their work-flow? Any other plans? Well the "bug" is resolved, we're now into the realm of new-feature-request. That said, I don't think overlays providing cache is going to be feasible, but what would likely help a lot would be to make layman generate the cache by default when it adds or updates an overlay. This doesn't solve manually-installed overlays of course but there's only so much that can be done systematically.
Wouldn't it make more sense to integrate the cache-generation functionality into eix-update instead? That way not only overlays added through layman would benefit from this feature.
First of all, note that having a proper cache is not only necessary for eix but actually for every portage (or other package manager) command. If you do not have the proper cache, portage will generate it (partially or completely) in /var/cache/edb: This can take a lot of time and also disk space.... Therefore, it is important to have a "correct" cache not only for eix. > overlays providing cache The problem with this is that the cache becomes invalid in the moment when some used eclass in the gentoo repository changes (even in a trivial way), becaues its md5 is stored in the cache. eix does not care about the md5 sum for eclasses (only for the ebuild itself), but this is actually not PMS conformant. Moreover, portage will regenerate the cache with wrong checksums in /var/cache/edb, using duplicate disk space for the corresponding packages. It seems that really the only good way is to generate is locally. I would recommend overlay maintainers to not provide a cache but instead a .gitignore containing the cache directory, so that the user can use egencache locally without causing any problems with syncing the overlay. The problem here is with other VCS than git: AFAIK there is not always some analogue for .gitignore. > would likely help a lot would be to make layman generate the cache by > default when it adds or updates an overlay The problem with this is exactly with overlays not providing a corresponding .gitignore or being based on other VCS: Just locally changing the repository will cause problems with the next sync. Moreover, it depends on the overlay whether additional egencache options like --update-use-local-desc --update-pkg-desc-index --update-changelogs (or --changelog-reversed) should be used when generating the cache: Some overlays provide these data and some don't and rely on local generation instead. > integrate the cache-generation functionality into eix-update eix-update is much too overloaded already: close to unusable and unmaintainable. Adding even more hooks is probably not a good idea. Moreover, there is a natural and official place to add such hooks: /etc/portage/repo.postsync.d/ There is an example file provided which describes how it is done. If you use app-portage/portage-postsyncd-mv from the mv overlay, it will add some files to that directory so that egencache is called automatically, and you can conveniently specify for which overlay which egencache options to use.
