Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 487504

Summary: x11-misc/shared-mime-info-1.2 updates database very slowly
Product: Portage Development Reporter: Nikos Chantziaras <realnc>
Component: CoreAssignee: Freedesktop bugs <freedesktop-bugs>
Status: RESOLVED FIXED    
Severity: enhancement CC: adaptee, david, doug.hunley, gmurray, josef64, marduk, nheghathivhistha, pacho, phantom4, systemd
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: All   
See Also: https://bugs.freedesktop.org/show_bug.cgi?id=70366
https://bugs.gentoo.org/show_bug.cgi?id=487626
https://bugs.gentoo.org/show_bug.cgi?id=462118
https://bugs.gentoo.org/show_bug.cgi?id=480616
Whiteboard:
Package list:
Runtime testing required: ---

Description Nikos Chantziaras 2013-10-10 10:37:18 UTC
With shared-mime-info-1.1, this command takes only 1 second to complete:

  /usr/bin/update-mime-database /usr/share/mime

With shared-mime-info-1.2 it takes over two minutes. There's no CPU load, only disk I/O.

I noticed this while doing a KDE upgrade. I had left the PC on its own, but when I came back only very few KDE packages had been emerged. The ebuilds call /usr/bin/update-mime-database several times, so the overhead this added for each ebuild was something like 10 minutes.

Reproducible: Always




Portage 2.2.7 (default/linux/amd64/13.0/desktop/kde, gcc-4.8.1, glibc-2.17, 3.10.15-gentoo x86_64)
=================================================================
System uname: Linux-3.10.15-gentoo-x86_64-Intel-R-_Core-TM-_i5-2500K_CPU_@_3.30GHz-with-gentoo-2.2
KiB Mem:    16386932 total,  10952652 free
KiB Swap:   25166844 total,  25166844 free
Timestamp of tree: Thu, 10 Oct 2013 09:15:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.5-r2, 3.2.5-r2, 3.3.2-r2
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.2
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.6.4, 4.7.3-r1, 4.8.1-r1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.10 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo vmware interactive-fiction steam-overlay Local
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-pipe -mtune=native -march=native -O2 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-pipe -mtune=native -march=native -O2 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block"
DISTDIR="/mnt/Data/portage/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.ntua.gr/pub/linux/gentoo http://mirror.ovh.net/gentoo-distfiles http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common"
MAKEOPTS="-j4"
PKGDIR="/mnt/Data/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/vmware /var/lib/layman/interactive-fiction /var/lib/layman/steam /usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac acpi aes-ni alsa amd64 audiofile avx bash-completion branding bzip2 cairo cdr cli consolekit cracklib crypt cxx dbus declarative dri dts dvd dvdr emboss encode exif fam fbcondecor flac gdbm gif gpm gtk iconv ipv6 jpeg kde kipi lcdfilter lcms libnotify libsamplerate lm_sensors mad mmx mmxext mng modules mp3 mp4 mpeg mudflap multilib ncurses nptl nvidia offensive ogg opengl openmp pango pcre pdf pgo phonon plasma png policykit ppds qt4 readline semantic-desktop session spell sse sse2 sse3 sse4 sse4_1 ssl ssse3 startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vdpau vorbis wxwidgets x264 xcb xcomposite xml xv xvid xvmc zlib" ABI_X86="32 64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_US" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby19 ruby18" SANE_BACKENDS="nothing" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 David Kredba 2013-10-10 17:53:02 UTC
I am updating KDE now and this is making the process unbelievable slow. I was thinking "what the hell happened with my PC, that now it compiles KDE all day instead of a few hours" and then I noticed HDD bzzzzzzzzz and ages at stage Updating shared mime info database ...
Comment 2 Nikos Chantziaras 2013-10-10 17:59:19 UTC
(In reply to David Kredba from comment #1)
> I am updating KDE now and this is making the process unbelievable slow. I
> was thinking "what the hell happened with my PC, that now it compiles KDE
> all day instead of a few hours" and then I noticed HDD bzzzzzzzzz and ages
> at stage Updating shared mime info database ...

As a workaround, simply abort the emerge, downgrade to x11-misc/shared-mime-info-1.1 and then do a @world upgrade again.
Comment 3 Samuli Suominen (RETIRED) gentoo-dev 2013-10-11 07:53:48 UTC
Added workaround to Portage:

+  11 Oct 2013; Samuli Suominen <ssuominen@gentoo.org>
+  shared-mime-info-1.2.ebuild:
+  Disable use of fdatasync() due to severe time regression wrt #487504 by Nikos
+  Chantziaras

And reported it to upstream:

https://bugs.freedesktop.org/show_bug.cgi?id=70366
Comment 4 Michael Palimaka (kensington) gentoo-dev 2013-10-13 12:48:38 UTC
(In reply to Samuli Suominen from comment #3)
> Added workaround to Portage:
> 
> +  11 Oct 2013; Samuli Suominen <ssuominen@gentoo.org>
> +  shared-mime-info-1.2.ebuild:
> +  Disable use of fdatasync() due to severe time regression wrt #487504 by
> Nikos
> +  Chantziaras

Perhaps it would be useful to do a revision bump to ensure everyone gets the workaround?

This is causing major issues for KDE users, since update-mime-database is called for every ebuild by the eclass.
Comment 5 Johannes Huber gentoo-dev 2013-10-13 13:25:37 UTC
(In reply to Michael Palimaka (kensington) from comment #4)
> (In reply to Samuli Suominen from comment #3)
> > Added workaround to Portage:
> > 
> > +  11 Oct 2013; Samuli Suominen <ssuominen@gentoo.org>
> > +  shared-mime-info-1.2.ebuild:
> > +  Disable use of fdatasync() due to severe time regression wrt #487504 by
> > Nikos
> > +  Chantziaras
> 
> Perhaps it would be useful to do a revision bump to ensure everyone gets the
> workaround?
> 
> This is causing major issues for KDE users, since update-mime-database is
> called for every ebuild by the eclass.

Revision bump is required here!
Comment 6 Lukáš Závodný 2013-10-30 17:31:55 UTC
I see no bump here? :)
Comment 7 josef.95 2013-11-06 11:04:56 UTC
... 
> Revision bump is required here!

Yes, please please make a Revision bump.
A sleek kdebase-meta minor update needed here over two hours only for update-mime-database :(
Comment 8 David Watzke 2013-11-06 13:08:30 UTC
Please make a revision bump. I had to google this issue first just to figure out it could have been easily avoided if you bothered to revbump the package. Thanks!
Comment 9 Pacho Ramos gentoo-dev 2013-11-06 20:04:20 UTC
+*shared-mime-info-1.2-r1 (06 Nov 2013)
+
+  06 Nov 2013; Pacho Ramos <pacho@gentoo.org> +shared-mime-info-1.2-r1.ebuild,
+  -shared-mime-info-1.2.ebuild:
+  Ensure people get fix for bug #487504
+
Comment 10 Samuli Suominen (RETIRED) gentoo-dev 2013-12-03 18:32:49 UTC
CCing dev-portage@

Can we integrate fdo-mime.eclass functionality to the PM and make it run the database updates only once, at the end of emerge, just like eg. ldconfig is ran now

As upstream suggests at https://bugs.freedesktop.org/show_bug.cgi?id=70366#c5

Because I consider disabling the fdatasync() a workaround. There is indeed no point in running it so many times.

Portage could run /usr/bin/update-desktop-database only when files get installed to /usr/share/applications and /usr/bin/update-mime-database only when files get installed to /usr/share/mime
Comment 11 Gilles Dartiguelongue gentoo-dev 2013-12-05 00:54:54 UTC
Same applies for gtk icon cache, immodules cache, pango cache, and a few others.
Comment 12 Pacho Ramos gentoo-dev 2014-01-01 14:28:19 UTC
Also would be useful for systemd as would allow to , for example, run command for creating new files when tmpfiles.d are installed, running it only one time at the end of emerge (like other PMs like yum do)
Comment 13 Pacho Ramos gentoo-dev 2014-07-15 11:42:18 UTC
(In reply to Pacho Ramos from comment #12)
> Also would be useful for systemd as would allow to , for example, run
> command for creating new files when tmpfiles.d are installed, running it
> only one time at the end of emerge (like other PMs like yum do)

This probably should be reassigned to portage team (and tagged as enhancement as we don't have patches or similar :( )
Comment 14 Samuli Suominen (RETIRED) gentoo-dev 2014-07-15 11:58:54 UTC
This is fixed in Portage by disabling upstream's use of fdatasync() and this will be fixed in shared-mime-info-1.3 by the new -n parameter which I intend to add to fdo-mime.eclass after the version bump (after it's released)

http://cgit.freedesktop.org/xdg/shared-mime-info/commit/?id=29a04be6c9cbaf0865c8b57428b7b7c37fbda4c3
http://cgit.freedesktop.org/xdg/shared-mime-info/commit/?id=4b3f9f774da8859d4f1f7e991b12832d6c09b63e

So, fix summary away from shared-mime-info if you intend to reuse this bug to fix something else? We have no problem with shared-mime-info, whatsoever, not now, not in future.
Comment 15 Samuli Suominen (RETIRED) gentoo-dev 2014-07-15 12:04:45 UTC
(In reply to Samuli Suominen from comment #10)
> CCing dev-portage@
> 
> Can we integrate fdo-mime.eclass functionality to the PM and make it run the
> database updates only once, at the end of emerge, just like eg. ldconfig is
> ran now

as in, this is no longer required... upstream fixed this in the application itself for shared-mime-info

if you still want to do it, i suppose you still can, but nothing is broken meanwhile
Comment 16 Nikos Chantziaras 2016-10-23 21:49:43 UTC
Is it OK to close this bug now?
Comment 17 Zac Medico gentoo-dev 2016-10-23 23:39:17 UTC
(In reply to Samuli Suominen from comment #14)
> So, fix summary away from shared-mime-info if you intend to reuse this bug
> to fix something else? We have no problem with shared-mime-info, whatsoever,
> not now, not in future.

We should open a new bug against PMS for this.

(In reply to Nikos Chantziaras from comment #16)
> Is it OK to close this bug now?

Done.