I'm using Gentoo kernel 2.6.38 in an VMWare ESXi 4.1 VM. I wanted to use the vmxnet3 paravirtualized devices/drivers, but this seems not to work properly. I realized a bad network performance for the machine which acts as vpn gateway, so it has packet forwarding enabled. On the VMWare website there are issues described with the LRO (Large receive offload) function which is enabled in the driver by default. A workaround states to use 'modprobe vmxnet3 disable_lro=1' . As far as I can see it this option is not existing in vmxnet3 V1.0.25.0-k, as modprobe complains ? Also I coudnt disable lro with ethtool. I'm not sure if I am missing something in my kernel, but also I get the following error trace in dmesg: [ 6432.259052] vmxnet3 0000:03:00.0: PCI INT A disabled [ 6437.184723] VMware vmxnet3 virtual NIC driver - version 1.0.25.0-k-NAPI [ 6437.184900] # of Tx queues : 4, # of Rx queues : 4 [ 6437.185007] vmxnet3 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 [ 6437.185091] vmxnet3 0000:03:00.0: setting latency timer to 64 [ 6437.276758] features: sg csum vlan jf tso tsoIPv6 lro highDMA [ 6437.277349] vmxnet3 0000:03:00.0: irq 72 for MSI/MSI-X [ 6437.277387] vmxnet3 0000:03:00.0: irq 73 for MSI/MSI-X [ 6437.277424] vmxnet3 0000:03:00.0: irq 74 for MSI/MSI-X [ 6437.277462] vmxnet3 0000:03:00.0: irq 75 for MSI/MSI-X [ 6437.277497] vmxnet3 0000:03:00.0: irq 76 for MSI/MSI-X [ 6437.277648] RSS is enabled. [ 6437.277878] ------------[ cut here ]------------ [ 6437.277906] WARNING: at net/core/dev.c:1363 inetdev_init+0x9f/0xe8() [ 6437.277916] Hardware name: VMware Virtual Platform [ 6437.277918] Modules linked in: vmxnet3(+) vmblock vmci container ac processor button [last unloaded: vmxnet3] [ 6437.277928] Pid: 2555, comm: modprobe Tainted: G W 2.6.38-gentoo #5 [ 6437.277930] Call Trace: [ 6437.277957] [<ffffffff81033c3e>] ? warn_slowpath_common+0x78/0x8c [ 6437.277960] [<ffffffff813bdbd5>] ? inetdev_init+0x9f/0xe8 [ 6437.277964] [<ffffffff813bdc72>] ? inetdev_event+0x54/0x3b2 [ 6437.277979] [<ffffffff813796d7>] ? netdev_queue_update_kobjects+0x7b/0xcb [ 6437.277991] [<ffffffff8104ba62>] ? notifier_call_chain+0x2e/0x5b [ 6437.278001] [<ffffffff8136d347>] ? register_netdevice+0x1f6/0x2a3 [ 6437.278005] [<ffffffff8136d42a>] ? register_netdev+0x36/0x49 [ 6437.278015] [<ffffffffa00c4e25>] ? vmxnet3_probe_device+0x6c5/0x794 [vmxnet3] [ 6437.278030] [<ffffffff81256462>] ? __pm_runtime_set_status+0x15b/0x17d [ 6437.278049] [<ffffffff811e025c>] ? local_pci_probe+0x3c/0x6b [ 6437.278052] [<ffffffff811e0ef0>] ? pci_device_probe+0xbf/0xec [ 6437.278059] [<ffffffff8125078e>] ? driver_sysfs_add+0x66/0x8d [ 6437.278062] [<ffffffff812508cf>] ? driver_probe_device+0xa8/0x138 [ 6437.278066] [<ffffffff812509a8>] ? __driver_attach+0x49/0x65 [ 6437.278069] [<ffffffff8125095f>] ? __driver_attach+0x0/0x65 [ 6437.278072] [<ffffffff8125095f>] ? __driver_attach+0x0/0x65 [ 6437.278074] [<ffffffff8124ff53>] ? bus_for_each_dev+0x44/0x78 [ 6437.278078] [<ffffffff812503aa>] ? bus_add_driver+0xa8/0x1f2 [ 6437.278081] [<ffffffff81250c40>] ? driver_register+0x8f/0xf7 [ 6437.278084] [<ffffffff811e1138>] ? __pci_register_driver+0x4e/0xbe [ 6437.278089] [<ffffffffa00ca000>] ? vmxnet3_init_module+0x0/0x3c [vmxnet3] [ 6437.278096] [<ffffffff81000209>] ? do_one_initcall+0x78/0x126 [ 6437.278103] [<ffffffff8105a283>] ? sys_init_module+0x97/0x1d4 [ 6437.278106] [<ffffffff8100283b>] ? system_call_fastpath+0x16/0x1b [ 6437.278109] ---[ end trace c31907ed2e4c66c6 ]--- [ 6437.278473] eth3: NIC Link is Up 10000 Mbps Reproducible: Always Steps to Reproduce: 1.have /proc/sys/net/ipv4/ip_forward 1 2.have multiple vmxnet3 NICs in the VM 3.load vmxnet3 driver 3.look at dmesg Actual Results: NIC works, but very slow error message shown in description Expected Results: no error message and good performance I also read that there is an update for ESXi 4.1 which solves some problems with the virtual hardware for vmxnet3, maybe this is the solution ? I changed to Intel e1000 hardware/drivers and this works very well. Portage 2.1.9.45 (unavailable, gcc-4.4.3, glibc-2.12.1-r3, 2.6.38-gentoo x86_64) ================================================================= System uname: Linux-2.6.38-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E5335_@_2.00GHz-with-gentoo-2.0.2 Timestamp of tree: Unknown dev-lang/python: 2.6.6-r2::gentoo, 2.7.1-r1::gentoo, 3.1.3-r1::gentoo sys-devel/autoconf: 2.68::gentoo sys-devel/automake: 1.11.1::gentoo sys-devel/binutils: 2.20.1-r1::gentoo sys-devel/libtool: 2.4-r1::gentoo virtual/os-headers: 2.6.38::gentoo (sys-kernel/linux-headers) ACCEPT_KEYWORDS="~amd64" ACCEPT_LICENSE="* -@EULA" CFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/openvpn/easy-rsa" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gentoo-release /etc/php/apache2-php5.2/ext-active/ /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.2/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.2/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=native -mtune=native -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs" GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="bash-completion berkdb logrotate ssl syslog unicode vim-syntax" Unset: EMERGE_DEFAULT_OPTS, PORTAGE_BUNZIP2_COMMAND
LRO issues seem to be fixed by upstream commit [1]. [1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=ebde6f8acba92abfc203585198a54f47e83e2cd0
This patch release in gentoo-sources-2.6.38-r6 and vanilla-sources-2.6.38.7
vanilla-sources 2.6.38.7 and gentoo-sources-2.6.38-r6 which contains 2.6.38.7 is now stable.
