With shared-mime-info-1.1, this command takes only 1 second to complete: /usr/bin/update-mime-database /usr/share/mime With shared-mime-info-1.2 it takes over two minutes. There's no CPU load, only disk I/O. I noticed this while doing a KDE upgrade. I had left the PC on its own, but when I came back only very few KDE packages had been emerged. The ebuilds call /usr/bin/update-mime-database several times, so the overhead this added for each ebuild was something like 10 minutes. Reproducible: Always Portage 2.2.7 (default/linux/amd64/13.0/desktop/kde, gcc-4.8.1, glibc-2.17, 3.10.15-gentoo x86_64) ================================================================= System uname: Linux-3.10.15-gentoo-x86_64-Intel-R-_Core-TM-_i5-2500K_CPU_@_3.30GHz-with-gentoo-2.2 KiB Mem: 16386932 total, 10952652 free KiB Swap: 25166844 total, 25166844 free Timestamp of tree: Thu, 10 Oct 2013 09:15:01 +0000 ld GNU ld (GNU Binutils) 2.23.2 app-shells/bash: 4.2_p45 dev-java/java-config: 2.2.0 dev-lang/python: 2.7.5-r2, 3.2.5-r2, 3.3.2-r2 dev-util/cmake: 2.8.11.2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.2 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.12.6 sys-devel/binutils: 2.23.2 sys-devel/gcc: 4.6.4, 4.7.3-r1, 4.8.1-r1 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.10 (virtual/os-headers) sys-libs/glibc: 2.17 Repositories: gentoo vmware interactive-fiction steam-overlay Local ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-pipe -mtune=native -march=native -O2 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-pipe -mtune=native -march=native -O2 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block" DISTDIR="/mnt/Data/portage/distfiles" EMERGE_DEFAULT_OPTS="--quiet-build=n" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.ntua.gr/pub/linux/gentoo http://mirror.ovh.net/gentoo-distfiles http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common" MAKEOPTS="-j4" PKGDIR="/mnt/Data/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/vmware /var/lib/layman/interactive-fiction /var/lib/layman/steam /usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X a52 aac acpi aes-ni alsa amd64 audiofile avx bash-completion branding bzip2 cairo cdr cli consolekit cracklib crypt cxx dbus declarative dri dts dvd dvdr emboss encode exif fam fbcondecor flac gdbm gif gpm gtk iconv ipv6 jpeg kde kipi lcdfilter lcms libnotify libsamplerate lm_sensors mad mmx mmxext mng modules mp3 mp4 mpeg mudflap multilib ncurses nptl nvidia offensive ogg opengl openmp pango pcre pdf pgo phonon plasma png policykit ppds qt4 readline semantic-desktop session spell sse sse2 sse3 sse4 sse4_1 ssl ssse3 startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vdpau vorbis wxwidgets x264 xcb xcomposite xml xv xvid xvmc zlib" ABI_X86="32 64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_US" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby19 ruby18" SANE_BACKENDS="nothing" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
I am updating KDE now and this is making the process unbelievable slow. I was thinking "what the hell happened with my PC, that now it compiles KDE all day instead of a few hours" and then I noticed HDD bzzzzzzzzz and ages at stage Updating shared mime info database ...
(In reply to David Kredba from comment #1) > I am updating KDE now and this is making the process unbelievable slow. I > was thinking "what the hell happened with my PC, that now it compiles KDE > all day instead of a few hours" and then I noticed HDD bzzzzzzzzz and ages > at stage Updating shared mime info database ... As a workaround, simply abort the emerge, downgrade to x11-misc/shared-mime-info-1.1 and then do a @world upgrade again.
Added workaround to Portage: + 11 Oct 2013; Samuli Suominen <ssuominen@gentoo.org> + shared-mime-info-1.2.ebuild: + Disable use of fdatasync() due to severe time regression wrt #487504 by Nikos + Chantziaras And reported it to upstream: https://bugs.freedesktop.org/show_bug.cgi?id=70366
(In reply to Samuli Suominen from comment #3) > Added workaround to Portage: > > + 11 Oct 2013; Samuli Suominen <ssuominen@gentoo.org> > + shared-mime-info-1.2.ebuild: > + Disable use of fdatasync() due to severe time regression wrt #487504 by > Nikos > + Chantziaras Perhaps it would be useful to do a revision bump to ensure everyone gets the workaround? This is causing major issues for KDE users, since update-mime-database is called for every ebuild by the eclass.
(In reply to Michael Palimaka (kensington) from comment #4) > (In reply to Samuli Suominen from comment #3) > > Added workaround to Portage: > > > > + 11 Oct 2013; Samuli Suominen <ssuominen@gentoo.org> > > + shared-mime-info-1.2.ebuild: > > + Disable use of fdatasync() due to severe time regression wrt #487504 by > > Nikos > > + Chantziaras > > Perhaps it would be useful to do a revision bump to ensure everyone gets the > workaround? > > This is causing major issues for KDE users, since update-mime-database is > called for every ebuild by the eclass. Revision bump is required here!
I see no bump here? :)
... > Revision bump is required here! Yes, please please make a Revision bump. A sleek kdebase-meta minor update needed here over two hours only for update-mime-database :(
Please make a revision bump. I had to google this issue first just to figure out it could have been easily avoided if you bothered to revbump the package. Thanks!
+*shared-mime-info-1.2-r1 (06 Nov 2013) + + 06 Nov 2013; Pacho Ramos <pacho@gentoo.org> +shared-mime-info-1.2-r1.ebuild, + -shared-mime-info-1.2.ebuild: + Ensure people get fix for bug #487504 +
CCing dev-portage@ Can we integrate fdo-mime.eclass functionality to the PM and make it run the database updates only once, at the end of emerge, just like eg. ldconfig is ran now As upstream suggests at https://bugs.freedesktop.org/show_bug.cgi?id=70366#c5 Because I consider disabling the fdatasync() a workaround. There is indeed no point in running it so many times. Portage could run /usr/bin/update-desktop-database only when files get installed to /usr/share/applications and /usr/bin/update-mime-database only when files get installed to /usr/share/mime
Same applies for gtk icon cache, immodules cache, pango cache, and a few others.
Also would be useful for systemd as would allow to , for example, run command for creating new files when tmpfiles.d are installed, running it only one time at the end of emerge (like other PMs like yum do)
(In reply to Pacho Ramos from comment #12) > Also would be useful for systemd as would allow to , for example, run > command for creating new files when tmpfiles.d are installed, running it > only one time at the end of emerge (like other PMs like yum do) This probably should be reassigned to portage team (and tagged as enhancement as we don't have patches or similar :( )
This is fixed in Portage by disabling upstream's use of fdatasync() and this will be fixed in shared-mime-info-1.3 by the new -n parameter which I intend to add to fdo-mime.eclass after the version bump (after it's released) http://cgit.freedesktop.org/xdg/shared-mime-info/commit/?id=29a04be6c9cbaf0865c8b57428b7b7c37fbda4c3 http://cgit.freedesktop.org/xdg/shared-mime-info/commit/?id=4b3f9f774da8859d4f1f7e991b12832d6c09b63e So, fix summary away from shared-mime-info if you intend to reuse this bug to fix something else? We have no problem with shared-mime-info, whatsoever, not now, not in future.
(In reply to Samuli Suominen from comment #10) > CCing dev-portage@ > > Can we integrate fdo-mime.eclass functionality to the PM and make it run the > database updates only once, at the end of emerge, just like eg. ldconfig is > ran now as in, this is no longer required... upstream fixed this in the application itself for shared-mime-info if you still want to do it, i suppose you still can, but nothing is broken meanwhile
Is it OK to close this bug now?
(In reply to Samuli Suominen from comment #14) > So, fix summary away from shared-mime-info if you intend to reuse this bug > to fix something else? We have no problem with shared-mime-info, whatsoever, > not now, not in future. We should open a new bug against PMS for this. (In reply to Nikos Chantziaras from comment #16) > Is it OK to close this bug now? Done.