Bug 195731 – regression(?): starting with 2.6.21 sending packets became broken

• Product: Gentoo Linux
• Status: RESOLVED UPSTREAM

[Description Peter Volkov (RETIRED) 2007-10-13 18:17:39 UTC]

Starting with 2.6.21 (or may be 2.6.20 as I have not tried it) kernel I have problem that most tcp based services freeze at some point of operation. I've noticed this first on ssh but then found out that at lease one other service became similarly. The problem sites somewhere in the kernel as I've compiled 2.6.19, 2.6.21, and 2.6.22 with the similar .config options (of course not exact, as some options does not exist in some kernels, but seems that enabled options are all the same) but I have this problem only with the 21 and 22. I've tried to debug the problem a bit, but not a lot as that is production box working as linux based firewall/router.

First I took tcpdump. Although ssh connection to the router is not
always possible as it often hangs before I get into router, after some
attempts ssh connection was established. On client computer I've started
tcpdump and worked a bit until hang. tcpdump output showed me that when
I press any keys the packets are sent to the server and proper ack are
received. Later I found that all commands I enter blindly are executed on router but I receive no reply packets with some data in them (pure ack). That's why nothing happens on the screen and it looks like hanging.

Now I've got to the router started ssh connection from router to some
other server. It hanged too. I attached strace and found that ssh
receive keyboard pressings (read() calls in the output) and writes them further to the kernel (write() calls) but tcpdump on the router shows no packets. So packets enter kernel and lost somewhere inside.

Now a information about my system. That's a pentium4 system with
hyper-threading enabled. cpuinfo and lspci output attached. kernel built
with "gcc version 4.1.2 (Gentoo 4.1.2 p1.0.2)" and binutils version
2.17. My .config file for all kernels I've mentioned is available here:

http://theor.ran.gpi.ru/linux-2.6.19-gentoo-r5-config (works)
http://theor.ran.gpi.ru/linux-2.6.21-gentoo-r4-config (not works)
http://theor.ran.gpi.ru/linux-2.6.22-gentoo-r8-config (not works)

Besides standard gentoo patchsets all kernels have IMQ and IPSET's
patches.

Does anybody have any idea what's going on with the latest kernels? How to debug it further?

[Comment 1 Peter Volkov (RETIRED) 2007-10-13 18:18:32 UTC]

Created attachment 133346 [details]
lspci output

[Comment 2 Peter Volkov (RETIRED) 2007-10-13 18:19:12 UTC]

Created attachment 133348 [details]
/proc/cpuinfo output

[Comment 3 Maarten Bressers (RETIRED) 2007-10-14 00:25:58 UTC]

Can you reproduce this on either vanilla- or gentoo-sources 2.6.23, not using any other (external) patches? If so, please post your .config and dmesg output.

[Comment 4 Daniel Drake (RETIRED) 2007-10-14 23:47:55 UTC]

Assuming you can reproduce it on an unpatched gentoo or vanilla kernel, try the following shortly after boot for the fun of it:

echo 0 > /proc/sys/net/ipv4/tcp_window_scaling

See if that makes the problem go away.. just a thought..

[Comment 5 Peter Volkov (RETIRED) 2007-10-23 18:37:46 UTC]

Thank you for advices, Maarten and Daniel. Seems that the problem was cause by IMQ patch (http://www.linuximq.net/patchs/linux-2.6.21-img2.diff). I do not know details but without patch 2.6.22-r8 works and with patch it does not. In IMQ mailing list I found patch for 2.6.23. Patch itself is not released, but I've tested it and seems that it works here. In any case this bug should be taken upstream. Thanks again.