| Summary: | sys-apps/portage-2.2.14 fails to clean outdated versions with sqlite backend | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Sergey S. Starikoff <Ikonta> |
| Component: | Unclassified | Assignee: | Martin Väth <martin> |
| Status: | RESOLVED CANTFIX | ||
| Severity: | normal | CC: | axs, dev-portage, proxy-maint, xmw |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
Also, man eix section, describing sqlite cache contains referencies to dead urls: sqlite This is an extremely fast cache method if you are using portage with the sqlite backend, see http://en.gentoo-wiki.com/wiki/Portage_SQLite_Cache (originally this was described in http://gentoo-wiki.com/TIP_speed_up_portage_with_sqlite which might be still accessible at http://gentoo-wiki.info/TIP_speed_up_portage_with_sqlite). Note that in contrast to the default metadata cache method you must use emerge --metadata before you call eix-update with this method. For now backup of this atricle is available at http://gentoo-en.vfose.ru/wiki/Portage_SQLite_Cache What happens if after the "repairing" with another cache method you switch back to sqlite: Are the obsolete versions still gone, or do they re-appear? (In reply to Martin Väth from comment #2) > What happens if after the "repairing" with another cache method you switch > back to sqlite: Are the obsolete versions still gone, or do they re-appear? After switching back to sqlite cache obsolete versions appeared again. I.e.: # grep PORTDIR /etc/eixrc/00-eixrc PORTDIR_CACHE_METHOD='parse' # eix-update … # eix -Ie firefox [I] www-client/firefox Available versions: *10.0.11 24.3.0 24.8.0 31.3.0 ~31.4.0 ~35.0 # vim /etc/eixrc/00-eixrc … # grep PORTDIR /etc/eixrc/00-eixrc PORTDIR_CACHE_METHOD='sqlite' # eix-update … # eix -Ie firefox [I] www-client/firefox Available versions: *10.0.11 17.0.9 24.3.0 24.5.0 24.6.0 24.7.0 24.8.0 ~29.0.1 ~30.0 ~31.0 ~31.1.0 ~31.1.0-r1 ~31.2.0 ~31.2.0-r1 31.3.0 ~31.4.0 ~32.0 ~33.0 ~33.0-r1 ~34.0.5-r1 ~35.0 CC'ing portage team: The test confirms that it is portage's sqlite database which contains the outdated versions. Is it possible to keep in the portage database only the current versions, i.e. to clean up the database in a sense? It appears very artificial (and would take a long time) if eix would need to check that an ebuild actually exists when portage's sqlite database reports it as existent. When you are at it, it would also be nice if /var/cache/edb/dep would receive such a cleaning, since stray data in there might similarly be a problem. Maybe it is possible to introduce a corresponding "emaint" command which eix-sync might call which cleans up the sqlite database and /var/cache/edb/dep? (Such a cleanup might be desirable independent of eix) @Sergey: As a workaround, you can put "-rM" into /etc/eix-sync.conf which causes eix-sync to clean /var/cache/edb/dep and to call emerge --metada to regenerate the sqlite database. This will als solve your bug 538918. Unfortunately, you loose then all of the speed gain of the sqlite backend for eix. (In reply to Martin Väth from comment #4) > CC'ing portage team: > > The test confirms that it is portage's sqlite database which contains the > outdated versions. > > Is it possible to keep in the portage database only the current versions, > i.e. to clean up the database in a sense? > > It appears very artificial (and would take a long time) if eix would need to > check that an ebuild actually exists when portage's sqlite database reports > it as existent. Portage is supposed to delete the outdated versions when you run emerge --metadata or emerge --regen. (In reply to Zac Medico from comment #5) > Portage is supposed to delete the outdated versions when you run emerge > --metadata or emerge --regen. Thanks for the information; I was not aware of this. So - assuming that this really works in portage - I guess the problem is that Sergey is using sqlite cache without emerge --metadata. In this case, eix cannot guarantee that the data is correct: Even if it would check ebuilds for existence, it might happen that not all new ebuilds are contained in the cache. eix would need to do everything portage does and this is too complex for eix. So I close this bug as CANTFIX. @Sergey: Either put -M into eix-sync.conf or FEATURES=metadata-transfer into your portage configuration. For the same reason, I close bug 538918 as CANTFIX. |
# eix-update Чтение настроек Portage .. Построение базы данных (/var/cache/eix/portage.eix) .. [0] 'gentoo' /usr/portage/gentoo/ (кэш: sqlite) Чтение Пакетов .. Готово [1] 'local_hdd' /usr/portage/local (кэш: parse|ebuild*#metadata-md5#metadata-assign#assign) Чтение категории 164|164 (100%) Готово Применение масок .. Расчёт хеш-таблиц .. Запись файла базы данных /var/cache/eix/portage.eix .. База данных содержит 15552 пакетов в 164 категориях. $ eix -Ie firefox [I] www-client/firefox Available versions: *10.0.11 17.0.9 24.3.0 24.5.0 24.6.0 24.7.0 24.8.0 ~29.0.1 ~30.0 ~31.0 ~31.1.0 ~31.1.0-r1 ~31.2.0 ~31.2.0-r1 31.3.0 ~31.4.0 ~32.0 ~33.0 ~33.0-r1 ~34.0.5-r1 ~35.0 … 17.0.9 was already removed from tree, but was used at its time and now still presents in eix list, together with all ever installed 24.?.0 versions: $ ls /usr/portage/gentoo/www-client/firefox ChangeLog files/ firefox-24.3.0.ebuild firefox-31.3.0.ebuild firefox-35.0.ebuild metadata.xml ChangeLog-2009 firefox-10.0.11.ebuild firefox-24.8.0.ebuild firefox-31.4.0.ebuild Manifest On my current system www-client/firefox presents only in distro (gentoo) overlay. Switching to parse cache method and running eix-update fixes issue: $ eix -Ie firefox [I] www-client/firefox Available versions: *10.0.11 24.3.0 24.8.0 31.3.0 ~31.4.0 ~35.0 $ einfo Portage 2.2.14 (python 3.3.5-final-0, default/linux/amd64/13.0, gcc-4.8.3, glibc-2.19-r1, 3.16.7-aufs x86_64) ================================================================= System uname: Linux-3.16.7-aufs-x86_64-AMD_Athlon-tm-_II_X2_250_Processor-with-gentoo-2.2 KiB Mem: 1794424 total, 65616 free KiB Swap: 8000364 total, 7605888 free Timestamp of tree: Wed, 04 Feb 2015 06:15:01 +0000 ld GNU ld (Gentoo 2.24 p1.4) 2.24 app-shells/bash: 4.2_p53 dev-java/java-config: 2.2.0 dev-lang/perl: 5.18.2-r2 dev-lang/python: 2.7.9-r1, 3.3.5-r1 dev-util/cmake: 2.8.12.2-r1 dev-util/pkgconfig: 0.28-r1 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.13.8 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6-r1, 1.13.4 sys-devel/binutils: 2.24-r3 sys-devel/gcc: 4.8.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.4 sys-devel/make: 4.0-r1 sys-kernel/linux-headers: 3.16 (virtual/os-headers) sys-libs/glibc: 2.19-r1 Repositories: gentoo local_hdd ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /etc/env.d /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.4/ext-active/ /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--ask --verbose --autounmask=n" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildsyspkg config-protect-if-modified distlocks downgrade-backup ebuild-locks fixlafiles merge-sync metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.yandex.ru/gentoo-distfiles/ ftp://mirror.yandex.ru/gentoo-distfiles/ http://ftp.corbina.net/pub/Linux/gentoo/ ftp://ftp.corbina.net/pub/Linux/gentoo/" LANG="ru_RU.utf8" LC_ALL="" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage/gentoo" PORTDIR_OVERLAY="/usr/portage/local" USE="X a52 ac3 acl alsa amd64 avi berkdb bold bzip2 cdr cli consolekit cracklib crypt cups cxx dbus djvu dri dvd flac fortran gdbm gif gtk iconv inotify jpeg jpeg2k libav lock mmx modules mp3 multilib ncurses nls nptl ogg openmp pam pcre pdf png policykit qt3support readline session sse sse2 ssl tcpd thunar tiff udev udisks unicode utf8 vorbis xcb xulrunner zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="ru ru_RU" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-4" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby20" USERLAND="GNU" VIDEO_CARDS="radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, SYNC, USE_PYTHON