Bug 398373 – sys-kernel/openvz-sources - Unable to enable disk write caching (2.6.32-openvz-feoktistov.1 #3 SMP)

• Product: Gentoo Linux
• Status: RESOLVED WORKSFORME

[Description Artem Kuznetsov 2012-01-10 12:31:06 UTC]

I've just reinstalled my homeserver, hence we have a fresh system.
The problem is that file copying is deadly slow (~10-20MB/s). 
atop says: 
MEM | tot 6.6G | free 5.7G | cache 66.7M | buff 44.1M | slab 84.3M | 
SWP | tot 0.0M | free 0.0M | | vmcom 0.0M | vmlim 3.3G | 
PAG | scan 25041 | stall 0 | | swin 0 | swout 0 | 
DSK | sda | busy 96% | read 169 | write 76 | avio 3 ms | 
DSK | sdd | busy 95% | read 296 | write 97 | avio 2 ms | 
DSK | sdb | busy 84% | read 152 | write 118 | avio 3 ms | 
DSK | sdc | busy 84% | read 311 | write 91 | avio 2 ms | 

Disks are loaded 100%, memory is completely free, it isn't used for caching disk writes.
FS is XFS (i've tried reiserfs too). 
I've checked all variables in /proc/sys/vm - no result.
There're nothnig in logs about it.
Sometimes (I can't understand why) speed is OK again (memory is using for disk cache).

Thanks.


Reproducible: Always




7GB RAM, mdadm RAID5 (4x1TB HDD) + LVM2 + cryptsetup for one LV (it isn't in use for now), OpenVZ (Linux storage 2.6.32-openvz-feoktistov.1 #3 SMP Thu Jan 5 13:43:56 MSK 2012 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 5000+ AuthenticAMD GNU/Linux).

emerge --info
Portage 2.1.10.41 (default/linux/amd64/10.0, gcc-4.5.3, glibc-2.13-r4, 2.6.32-openvz-feoktistov.1 x86_64)
=================================================================
System uname: Linux-2.6.32-openvz-feoktistov.1-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_5000+-with-gentoo-2.0.3
Timestamp of tree: Tue, 10 Jan 2012 02:30:01 +0000
app-shells/bash:          4.1_p9
dev-lang/python:          2.7.2-r3, 3.1.4-r3
dev-util/cmake:           2.8.6-r4
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.0.3
sys-apps/openrc:          0.9.4
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.68
sys-devel/automake:       1.11.1
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.5.3-r1
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r1
sys-kernel/linux-headers: 2.6.39 (virtual/os-headers)
sys-libs/glibc:           2.13-r4
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="ru_RU.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en ru"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="acl amd64 berkdb bzip2 cli cracklib crypt cups cxx dri fortran gdbm gpm iconv ipv6 mmx modules mudflap multilib ncurses nls nptl nptlonly openmp pam pcre pppd readline session sse sse2 ssl sysfs tcpd unicode xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ru" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 1 Peter Volkov (RETIRED) 2012-02-24 18:41:53 UTC]

Hm I've used this kernel without any problems, so at least in my case it worked. Also I think there is no way to disable caching globally and I don't see that you have empty caches so it looks like caching works... Please, show output of hdparm -i /dev/sda. Is dma enabled? Please, attach dmesg output.

Also although I don't think this is kernel, but configuration problem, I suggest you to unmask and move on newer kernel: 2.6.32.49.6. There is possibility that driver is broken/old in feoktistov kernel.

[Comment 2 Artem Kuznetsov 2012-02-24 19:44:55 UTC]

Hello, Peter.

Thanks  for reply.
I moved to regular kernel hence I couldn't fix it for a long time.
Yes, DMA was enabled, dmesg wasn't showing anything about this issue.
Task isn't actual for me any more.

[Comment 3 Peter Volkov (RETIRED) 2012-05-03 12:33:56 UTC]

I guess kernel is Ok, but statistics is broken here. Closing as worksforme since it works for me :) thanks for feedback.