Bug 362313 – WARNING: at net/core/dev.c when loading vmxnet3 driver

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Christian Seifert 2011-04-06 16:26:58 UTC]

I'm using Gentoo kernel 2.6.38 in an VMWare ESXi 4.1 VM. I wanted to use the vmxnet3 paravirtualized devices/drivers, but this seems not to work properly. I realized a bad network performance for the machine which acts as vpn gateway, so it has packet forwarding enabled.

On the VMWare website there are issues described with the LRO (Large receive offload) function which is enabled in the driver by default. A workaround states to use  'modprobe vmxnet3 disable_lro=1' . As far as I can see it this option is not existing in vmxnet3 V1.0.25.0-k, as modprobe complains ? Also I coudnt disable lro with ethtool. I'm not sure if I am missing something in my kernel, but also I get the following error trace in dmesg:

[ 6432.259052] vmxnet3 0000:03:00.0: PCI INT A disabled
[ 6437.184723] VMware vmxnet3 virtual NIC driver - version 1.0.25.0-k-NAPI
[ 6437.184900] # of Tx queues : 4, # of Rx queues : 4
[ 6437.185007] vmxnet3 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[ 6437.185091] vmxnet3 0000:03:00.0: setting latency timer to 64
[ 6437.276758] features: sg csum vlan jf tso tsoIPv6 lro highDMA
[ 6437.277349] vmxnet3 0000:03:00.0: irq 72 for MSI/MSI-X
[ 6437.277387] vmxnet3 0000:03:00.0: irq 73 for MSI/MSI-X
[ 6437.277424] vmxnet3 0000:03:00.0: irq 74 for MSI/MSI-X
[ 6437.277462] vmxnet3 0000:03:00.0: irq 75 for MSI/MSI-X
[ 6437.277497] vmxnet3 0000:03:00.0: irq 76 for MSI/MSI-X
[ 6437.277648] RSS is enabled.
[ 6437.277878] ------------[ cut here ]------------
[ 6437.277906] WARNING: at net/core/dev.c:1363 inetdev_init+0x9f/0xe8()
[ 6437.277916] Hardware name: VMware Virtual Platform
[ 6437.277918] Modules linked in: vmxnet3(+) vmblock vmci container ac processor button [last unloaded: vmxnet3]
[ 6437.277928] Pid: 2555, comm: modprobe Tainted: G        W   2.6.38-gentoo #5
[ 6437.277930] Call Trace:
[ 6437.277957]  [<ffffffff81033c3e>] ? warn_slowpath_common+0x78/0x8c
[ 6437.277960]  [<ffffffff813bdbd5>] ? inetdev_init+0x9f/0xe8
[ 6437.277964]  [<ffffffff813bdc72>] ? inetdev_event+0x54/0x3b2
[ 6437.277979]  [<ffffffff813796d7>] ? netdev_queue_update_kobjects+0x7b/0xcb
[ 6437.277991]  [<ffffffff8104ba62>] ? notifier_call_chain+0x2e/0x5b
[ 6437.278001]  [<ffffffff8136d347>] ? register_netdevice+0x1f6/0x2a3
[ 6437.278005]  [<ffffffff8136d42a>] ? register_netdev+0x36/0x49
[ 6437.278015]  [<ffffffffa00c4e25>] ? vmxnet3_probe_device+0x6c5/0x794 [vmxnet3]
[ 6437.278030]  [<ffffffff81256462>] ? __pm_runtime_set_status+0x15b/0x17d
[ 6437.278049]  [<ffffffff811e025c>] ? local_pci_probe+0x3c/0x6b
[ 6437.278052]  [<ffffffff811e0ef0>] ? pci_device_probe+0xbf/0xec
[ 6437.278059]  [<ffffffff8125078e>] ? driver_sysfs_add+0x66/0x8d
[ 6437.278062]  [<ffffffff812508cf>] ? driver_probe_device+0xa8/0x138
[ 6437.278066]  [<ffffffff812509a8>] ? __driver_attach+0x49/0x65
[ 6437.278069]  [<ffffffff8125095f>] ? __driver_attach+0x0/0x65
[ 6437.278072]  [<ffffffff8125095f>] ? __driver_attach+0x0/0x65
[ 6437.278074]  [<ffffffff8124ff53>] ? bus_for_each_dev+0x44/0x78
[ 6437.278078]  [<ffffffff812503aa>] ? bus_add_driver+0xa8/0x1f2
[ 6437.278081]  [<ffffffff81250c40>] ? driver_register+0x8f/0xf7
[ 6437.278084]  [<ffffffff811e1138>] ? __pci_register_driver+0x4e/0xbe
[ 6437.278089]  [<ffffffffa00ca000>] ? vmxnet3_init_module+0x0/0x3c [vmxnet3]
[ 6437.278096]  [<ffffffff81000209>] ? do_one_initcall+0x78/0x126
[ 6437.278103]  [<ffffffff8105a283>] ? sys_init_module+0x97/0x1d4
[ 6437.278106]  [<ffffffff8100283b>] ? system_call_fastpath+0x16/0x1b
[ 6437.278109] ---[ end trace c31907ed2e4c66c6 ]---
[ 6437.278473] eth3: NIC Link is Up 10000 Mbps


Reproducible: Always

Steps to Reproduce:
1.have /proc/sys/net/ipv4/ip_forward 1
2.have multiple vmxnet3 NICs in the VM
3.load vmxnet3 driver
3.look at dmesg

Actual Results:  
NIC works, but very slow
error message shown in description

Expected Results:  
no error message and good performance

I also read that there is an update for ESXi 4.1 which solves some problems with the virtual hardware for vmxnet3, maybe this is the solution ?
I changed to Intel e1000 hardware/drivers and this works very well.

Portage 2.1.9.45 (unavailable, gcc-4.4.3, glibc-2.12.1-r3, 2.6.38-gentoo x86_64)
=================================================================
System uname: Linux-2.6.38-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E5335_@_2.00GHz-with-gentoo-2.0.2
Timestamp of tree: Unknown
dev-lang/python:     2.6.6-r2::gentoo, 2.7.1-r1::gentoo, 3.1.3-r1::gentoo
sys-devel/autoconf:  2.68::gentoo
sys-devel/automake:  1.11.1::gentoo
sys-devel/binutils:  2.20.1-r1::gentoo
sys-devel/libtool:   2.4-r1::gentoo
virtual/os-headers:  2.6.38::gentoo (sys-kernel/linux-headers)
ACCEPT_KEYWORDS="~amd64"
ACCEPT_LICENSE="* -@EULA"
CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/openvpn/easy-rsa"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gentoo-release /etc/php/apache2-php5.2/ext-active/ /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.2/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.2/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="bash-completion berkdb logrotate ssl syslog unicode vim-syntax" 
Unset:  EMERGE_DEFAULT_OPTS, PORTAGE_BUNZIP2_COMMAND

[Comment 1 Stratos Psomadakis (RETIRED) 2011-05-17 11:22:45 UTC]

LRO issues seem to be fixed by upstream commit [1].

[1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=ebde6f8acba92abfc203585198a54f47e83e2cd0

[Comment 2 Mike Pagano 2011-05-24 17:04:45 UTC]

This patch release in gentoo-sources-2.6.38-r6 and vanilla-sources-2.6.38.7

[Comment 3 Mike Pagano 2011-06-03 18:00:03 UTC]

vanilla-sources 2.6.38.7 and gentoo-sources-2.6.38-r6 which contains 2.6.38.7 is now stable.