Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 410093 - x11-plugins/wmtop-0.9.0 memory usage monitor broken by /proc/meminfo changes and improper use of int type
Summary: x11-plugins/wmtop-0.9.0 memory usage monitor broken by /proc/meminfo changes ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Dockapp Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-03-29 04:44 UTC by wbk
Modified: 2012-04-18 09:32 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
patch to x11-plugins/wmtop-0.9.0 to use current /proc/meminfo fields and store system statistics in larger data types (proc_meminfo_MemTotal_long_types.patch,5.66 KB, patch)
2012-03-30 04:16 UTC, wbk
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description wbk 2012-03-29 04:44:46 UTC
There are two separate issues coming into play; see below.

wmtop displays the top three load-producing processes on the system, a la "top." It has two modes: CPU and memory. CPU seems to work fine. Memory mode is broken. Currently, it misidentifies the three largest resident processes. Additionally, the "load bar" or meter for memory usage for all three processes is always at 100%.

Always reproducible. Step are below:

1. Start wmtop. Either use the -m flag to start in memory mode, or left-click the dockapp once it is running.

2. For reference, in a terminal window, start "top" and hit "M" to sort by memory usage.

3. Compare the top three processes in "top" with the display in wmtop. The process names and percentages should match. They will not.

Two separate issues are coming into play here. First, at some point in Linux 2.6.x, some redundant values were removed from /proc/meminfo. The old /proc/meminfo had a line beginning with "Mem:" which contained several values, including total system memory. wmtop uses this line to determine total memory. The same values, in a different format, are present elsewhere in /proc/meminfo. It's a trivial fix to make wmtop read the "MemTotal:" field instead of parsing the "Total" field out of the "Mem:" line. This should be a low-risk fix, since the newer values have been present since at least the 2.2.x branch; possibly even 2.0.x or earlier. I can find references to the newer fields from the mid-to-late 1990's.

After fixing the /proc/meminfo issue, another bug will emerge on systems where total system RAM in bytes exceeds INT_MAX. The correct top three processes will be identified, but their load bars will have the wrong value. This is caused by a value overflow due to storing total system memory in an "int" type variable. We will end up thinking ((Total RAM) modulo (INT_MAX)) is our total system RAM, so our percentages will be inflated when drawing the load bar. This fix will require a bit more care to follow the flow of data and ensure this value isn't being cast to "int" along the way. Function return types will need to be changed.

There are at least three or four other potential issues where we parse values from /proc into smaller data types than those specified in the proc(5) manpage. I'll fix these while I'm at it. The FreeBSD code should also be looked at, but the only FreeBSD system I have regular access to has very low system specs, and won't be a very good test case for the value overflow issue.

My patch is in the works; will post here when finished.
Comment 1 wbk 2012-03-29 04:49:45 UTC
Portage 2.1.10.49 (default/linux/amd64/10.0, gcc-4.5.3, glibc-2.13-r4, 3.0.17-tuxonice-r1-rlyeh-v11-consolekit x86_64)
=================================================================
System uname: Linux-3.0.17-tuxonice-r1-rlyeh-v11-consolekit-x86_64-AMD_A8-3510MX_APU_with_Radeon-tm-_HD_Graphics-with-gentoo-2.0.3
Timestamp of tree: Thu, 22 Mar 2012 07:15:01 +0000
app-shells/bash:          4.2_p20
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.2-r3, 3.1.4-r3, 3.2.2
dev-util/cmake:           2.8.6-r4
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.0.3
sys-apps/openrc:          0.9.8.4
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.11.1
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.5.3-r2
sys-devel/gcc-config:     1.5-r2
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r1
sys-kernel/linux-headers: 3.1 (virtual/os-headers)
sys-libs/glibc:           2.13-r4
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="rsync://rsync.gtlib.gatech.edu/gentoo ftp://ftp.gtlib.gatech.edu/pub/gentoo ftp://lug.mtu.edu/gentoo/ rsync://mirrors.rit.edu/gentoo/"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage"
USE="3dnow X a52 a53 aac aalib acl acpi alsa amd64 apng bash-completion berkdb bluetooth bzip2 cdparanoia cdr cli cracklib crypt cups cxx dbus dri ffmpeg fglrx firefox flac fortran gdbm gnome gpm gstreamer gtk gzip iconv ipv6 jpeg mmx modules mp3 mp4 mpeg mplayer mudflap multilib nautilus ncurses nls nptl nptlonly offensive ogg opengl openmp pam pcre png posix pppd python readline sdl session sse sse2 ssl sysfs tcpd threads tiff udev unicode v4l vorbis x264 xcomposite xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fglrx" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 2 wbk 2012-03-29 06:28:49 UTC
I have my fix coded and running on my local machine. The CPU monitoring functionality appeared to be potentially susceptible to similar value overflow bugs, so I made similar changes to the variable types used for CPU stats. I don't have a good feel for the range of values one might find on various architectures for CPU time stats, so I'm not sure how likely we are to hit the max value limits with CPU stats. But it seemed right to fix typing consistently throughout the app.

Since I've had to touch so many variables and mess with the internal function interfaces, I plan on testing this for a while locally. I'll probably post the patch in a couple days when I'm more confident that I haven't broken anything.

-wbk
Comment 3 wbk 2012-03-30 04:16:30 UTC
Created attachment 307161 [details, diff]
patch to x11-plugins/wmtop-0.9.0 to use current /proc/meminfo fields and store system statistics in larger data types

OK, I'm pretty confident this fixes the issue for Linux without introducing any new bugs. FreeBSD still needs to be looked at. I can research and code the fix if it ends up being warranted, but I don't have a FreeBSD system for testing with RAM > INT_MAX, which is a necessary condition for the integer overflow issue to occur.

I have also sent a copy upstream to Alan Swanson. Please let me know if there are any problems with the patch format.

Thanks!
-wbk
Comment 4 wbk 2012-04-14 04:30:26 UTC
I did finally look at this on FreeBSD (the mainline distro, not Gentoo). The ports tree version of wmtop completely disables the memory monitor feature (probably due to /proc reliance). I'll assume that more qualified eyes than mine have looked at this feature and found it problematic for FreeBSD. I may revisit this someday when I have more time to spend on a FreeBSD box, but for now I'm leaving it alone.

Please let me know if anything else is needed from me.
Comment 5 Bernard Cafarelli gentoo-dev 2012-04-18 09:32:11 UTC
In all the years I have been using wmtop, I never switched to memory mode apparently!

Thanks (as usual) for your work on this, I added your patch in 0.9.0-r1