Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 407325 - x11-plugins/wmmon fails to report I/O statistics
Summary: x11-plugins/wmmon fails to report I/O statistics
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Dockapp Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-03-07 20:19 UTC by wbk
Modified: 2012-03-29 22:25 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
patch for disk I/O stats from /proc/diskstats instead of /proc/stat. (wmmon.wbk.gentoo.patch,4.98 KB, patch)
2012-03-09 06:56 UTC, wbk
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description wbk 2012-03-07 20:19:50 UTC
wmmon is a simple load monitor graph. In "I/O" mode, it should report disk I/O. Currently, it does not. Testing under tuxonice-3.0.17-r1 kernel. 

This has apparently been broken since 2.6 and later 2.4 kernels, when disk_rio and disk_wio stats were removed from /proc/stat. The kernel developers' intent was apparently to move this information under the sysfs heirarchy; as far as I can tell it wound up back in /proc, as part of /proc/diskstats. The new stats are formatted differently and need to be parsed differently.

I already have a fix coded, and am testing locally to determine the most effective stats to graph. I will post my patch once it's tested (probably tonight). 

Historically, the I/O meter in wmmon has produced very sparse graphs, probably due to system buffering, read/write merges, and the like. The newer /proc/diskstats interface gives us more flexibility in choosing a metric for I/O activity, so I am experimenting with the metrics in order to make the graph a bit more sensitive.

It seems wmmon has very recently been handed to a new upstream maintainer, whom I will try to contact tonight. Will post details once confirmed. They have a new release out (1.1 I think), but it still contains this bug.

This fix is very linux-specific. Fixing this for BSD would probably involve iostat() calls, since procfs is deprecated on FreeBSD.
Comment 1 wbk 2012-03-07 20:21:31 UTC
case@rlyeh ~ $ emerge --info
/usr/lib64/portage/pym/portage/package/ebuild/config.py:353: UserWarning: 'cache.metadata_overlay.database' is deprecated: /etc/portage/modules
  (user_auxdbmodule, modules_file))
Portage 2.1.10.44 (default/linux/amd64/10.0, gcc-4.5.3, glibc-2.13-r4, 3.0.17-tuxonice-r1-rlyeh-v10-tuxonice x86_64)
=================================================================
System uname: Linux-3.0.17-tuxonice-r1-rlyeh-v10-tuxonice-x86_64-AMD_A8-3510MX_APU_with_Radeon-tm-_HD_Graphics-with-gentoo-2.0.3
Timestamp of tree: Tue, 06 Mar 2012 11:45:01 +0000
app-shells/bash:          4.1_p9
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.2-r3, 3.1.4-r3
dev-util/cmake:           2.8.6-r4
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.0.3
sys-apps/openrc:          0.9.8.4
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.11.1
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.5.3-r1
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r1
sys-kernel/linux-headers: 3.1 (virtual/os-headers)
sys-libs/glibc:           2.13-r4
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="rsync://rsync.gtlib.gatech.edu/gentoo ftp://ftp.gtlib.gatech.edu/pub/gentoo ftp://lug.mtu.edu/gentoo/ rsync://mirrors.rit.edu/gentoo/"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"
USE="3dnow X a52 a53 aac aalib acl acpi alsa amd64 apng bash-completion berkdb bluetooth bzip2 cdparanoia cdr cli cracklib crypt cups cxx dbus dri ffmpeg fglrx firefox flac fortran gdbm gnome gpm gstreamer gtk gzip iconv ipv6 jpeg mmx modules mp3 mp4 mpeg mplayer mudflap multilib nautilus ncurses nls nptl nptlonly offensive ogg opengl openmp pam pcre png posix pppd python readline sdl session sse sse2 ssl sysfs tcpd threads tiff udev unicode v4l vorbis x264 xcomposite xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fglrx" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 2 wbk 2012-03-07 20:31:47 UTC
It should be noted, my fix will probably break I/O monitoring in wmmon on ancient kernels (probably the earlier 2.4 kernels and back). I don't think Gentoo supports these old kernels, but please let me know if I need to account for this. I could add some failsafe code to fall back to /proc/stat if /proc/diskstats doesn't exist. But it would look cluttered, since we'd have to have an alternative parsing loop for the older stats' format :).
Comment 3 wbk 2012-03-08 22:32:16 UTC
My testing has been delayed a bit. I'm pretty sure there are other bugs here. The I/O graph seems to quit working or work inconsistently after the app has been running for a while. I suspect this is an old issue: from what I can remember of using this app circa 1999-2001, it had the same issue back then. I suspect it's something to do with fopen()/freopen()/fseek() return codes not being checked. My patch follows the same conventions as the existing code, but I'm adding some error logging to stderr.

Have heard from new upstream maintainer, and will post his contact info as soon as I get permission. Will post patch very soon (testing against new upstream version).
Comment 4 wbk 2012-03-09 06:56:01 UTC
Created attachment 304701 [details, diff]
patch for disk I/O stats from /proc/diskstats instead of /proc/stat.

In addition to using /proc/diskstats, merged in a change from upstream to buffer input by line instead of a fixed length buffer, resulting in more accurate reading of stats and better graphing.
Comment 5 wbk 2012-03-09 06:56:11 UTC
OK, here is what I've got (unified diff patch attached). In addition to my fix of using /dev/diskstats, I also incorporated one of the changes from upstream to buffer the /proc/ files by lines, instead of a fixed-length buffer. The fixed buffer seemed like it was interfering with strstr() matching, meaning we'd sometimes miss reading stats. I'm not sure exactly how this impacted graphing, but graphs look much nicer after this change.

The new upstream repository is available as follows: 

git://repo.or.cz/dockapps.git

The new upstream guys have good bugfixes and seem to be actively developing SMP features, so I'd urge the Gentoo maintainer to make sure we pull in the newest version if we haven't already. This app in particular seemed to be getting split between divergent Gentoo and Debian forks. The Debian patches are now merged into the new upstream repository, so maybe we can get the Gentoo patches incorporated as well and everyone can play nicely together :).

Let me know if more is needed, or if there are any problems with the patch (I'm still learning this collaboration stuff). Thanks!
-wbk
Comment 6 wbk 2012-03-12 04:48:47 UTC
I just realised, I forgot to include steps to reproduce (I must have misremembered; I thought there was a separate for input for this and omitted it from the initial bug  description). It's simple:

1. emerge x11-plugins/wmmon
2, run "wmmon"
3. right-click until you see "I/O". Do something to generate disk I/O (I like to emerge libreoffice when testing load monitors in general, but use whatever method you like).
4. I/O read and write sectors should be graphed in the dockapp window. Instead, the graph will be blank.

As a side note, I'm tracking down the inconsistent graphing behaviour I've described above, and I'm working on some code tweaks to fix it. It's mostly an issue with the graph scaling - a large read or write bumps the scale up to a high value, but it never decreases due to being a static variable in a local function namespace which is not initialised on function entry. I'm going to file a separate bug for this in the near future. No point now, since this isn't even testable until the stat reading is patched (with my attached fix).
Comment 7 Bernard Cafarelli gentoo-dev 2012-03-26 09:20:17 UTC
Again, thanks for your work on dockapps! Sorry for the delay here (a bit busy)

By coincidence, did you get upstream commit access? I see the "wmmon I/O monitor bug fixes" commit that just got pushed at dockapps.git

I'll make a clean wmmon-1.2b1 ebuild of it if possible, and update HOMEPAGE to point to the repo, as I am not sure where to find the 1.1 Debian uses
Comment 8 wbk 2012-03-26 20:11:15 UTC
No worries! Yes, 1.2b1 is my work. It turns out that repository is maintained by Carlos at the Windowmaker project. The repository I posted is the same upstream that Debian now uses.

You may want to wait a moment and pull again. I just sent in a patch to change an accidental -O3 flag to -O2 in the Makefile.

I intentionally bumped the version number up to 1.2, in order to avoid conflict or confusion with any of the local distro- or OS- specific patchkits. I don't think 1.1 was actually announced or released, but the Debian patches were applied upstream, so I guess we can consider that to be 1.1. Some of the Debian patches are very good and quite necessary for reading accurate stats, so my fixes depend on them to work optimally (this is why I haven't posted a patch here yet; it'll be better to pull from upstream and get the Debian patches).

I intend 1.2 to be primarily bugfixes. For 1.3 and beyond, I'm going to look into merging in patches from the BSD ports trees, if it can be done cleanly. When this is done, I have some ideas for new color scheme or skinning features. This was on the original authors' TODO list, but it never got done. 

Thanks!
-wbk
Comment 9 wbk 2012-03-26 20:44:47 UTC
Oops, I guess I DID post a patch here.

I'd recommend just pulling from upstream. The patch I posted here is an improvement, but it doesn't address some more subtle graph scaling bugs that cropped up during testing.
Comment 10 Bernard Cafarelli gentoo-dev 2012-03-29 11:52:32 UTC
Here you go!
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/x11-plugins/wmmon/

I have made a tarball to keep the ebuild readable (the URLs on git repo tarballs are a nightmare)

Also can you take a look at this Makefile patch? It allows me to use "make CC=bla CFLAGS=bla LDFLAGS=bla" and respect these variables when compiling:
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/x11-plugins/wmmon/files/wmmon-1.2_beta1-Makefile.patch?view=log

Thanks again anyway, I will keep track of the git repo (and wmaker mailing lists), but don't hesitate to open bugs for version bumps or other problems :)
Comment 11 wbk 2012-03-29 22:25:46 UTC
Thank you!

The Makefile looks good as far as I can tell. I have very little experience editing Makefiles, so thanks for the help :). I will ask Carlos to apply it upstream once I test it.

FYI, it's on my TODO list to add the Gentoo patches from 1.0-beta2-r3 to upstream (such as the -b flag to include kernel buffers in Memory mode). I actually use the -b flag and find it useful. It's missing from the current version in the repository at the moment. I plan to patch it back in the near future as part of 1.3. So in case anyone misses that feature in 1.2b1, it is definitely coming back. I would have already added it in, but the project was moving a bit slower than I intended, so I wanted to push out the bugfixes before worrying about feature enhancements to the upstream version.

Thanks again for your help. This is all very new to me and I am learning as I go.