Kernel freezes while downloading from network. It's connected to 1 Gbps network and uploads don't crash the kernel, but downloads do. NIC is integrated in Atom 330 box. with gentoo-sources-2.6.29-r5 (and older 2.6.29-rx versions) downloading is working, but uploads stop quite often in heavy usage, but it doesn't crash kernell. I think this bug is reported already (Bug#: 274765) $ uname -a: Linux atomi 2.6.29-gentoo-r5 #1 SMP Thu May 28 08:12:30 EEST 2009 x86_64 Intel(R) Atom(TM) CPU 330 @ 1.60GHz GenuineIntel GNU/Linux $ lspci: 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02) Reproducible: Always Steps to Reproduce: 1. download torrent with at least 10 MB/s speed. Actual Results: system freezes
Can you post the dmesg output, lspci -v output, and kernel config? There are also some commits in 2.6.31-rc1 regarding r8169...
Created attachment 199048 [details] gentoo-sources-2.6.29-r5 dmesg after 4 days of uptime
Created attachment 199050 [details] gentoo-sources-2.6.29-r5 config
Created attachment 199052 [details] gentoo-sources-2.6.30-r3 config
lspci -v output with 2.6.29-r5 kernel. It's quite hard to get any meaningfull results from 2.6.30-rx kernels, because they freeze the system so easilly. $ lspci -v 00:00.0 Host bridge: Intel Corporation 82945G/GZ/P/PL Memory Controller Hub (rev 02) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, fast devsel, latency 0 Capabilities: [e0] Vendor Specific Information <?> Kernel driver in use: agpgart-intel 00:02.0 VGA compatible controller: Intel Corporation 82945G/GZ Integrated Graphics Controller (rev 02) (prog-if 00 [VGA controller]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, fast devsel, latency 0, IRQ 16 Memory at fdf00000 (32-bit, non-prefetchable) [size=512K] I/O ports at ff00 [size=8] Memory at d0000000 (32-bit, prefetchable) [size=256M] Memory at fdf80000 (32-bit, non-prefetchable) [size=256K] Capabilities: [90] Message Signalled Interrupts: Mask- 64bit- Count=1/1 Enable- Capabilities: [d0] Power Management version 2 00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01) (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 0000c000-0000cfff Memory behind bridge: fdb00000-fdbfffff Prefetchable memory behind bridge: 00000000fdc00000-00000000fdcfffff Capabilities: [40] Express Root Port (Slot+), MSI 00 Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Count=1/1 Enable+ Capabilities: [90] Subsystem: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 Capabilities: [a0] Power Management version 2 Capabilities: [100] Virtual Channel <?> Capabilities: [180] Root Complex Link <?> Kernel driver in use: pcieport-driver 00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 01) (prog-if 00 [UHCI]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, medium devsel, latency 0, IRQ 23 I/O ports at fe00 [size=32] Kernel driver in use: uhci_hcd 00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 01) (prog-if 00 [UHCI]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, medium devsel, latency 0, IRQ 19 I/O ports at fd00 [size=32] Kernel driver in use: uhci_hcd 00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 01) (prog-if 00 [UHCI]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, medium devsel, latency 0, IRQ 18 I/O ports at fc00 [size=32] Kernel driver in use: uhci_hcd 00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 01) (prog-if 00 [UHCI]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, medium devsel, latency 0, IRQ 16 I/O ports at fb00 [size=32] Kernel driver in use: uhci_hcd 00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01) (prog-if 20 [EHCI]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, medium devsel, latency 0, IRQ 23 Memory at fdfff000 (32-bit, non-prefetchable) [size=1K] Capabilities: [50] Power Management version 2 Capabilities: [58] Debug port: BAR=1 offset=00a0 Kernel driver in use: ehci_hcd 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1) (prog-if 01 [Subtractive decode]) Flags: bus master, fast devsel, latency 0 Bus: primary=00, secondary=02, subordinate=02, sec-latency=32 I/O behind bridge: 0000d000-0000dfff Memory behind bridge: fde00000-fdefffff Prefetchable memory behind bridge: 00000000fdd00000-00000000fddfffff Capabilities: [50] Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, medium devsel, latency 0 Capabilities: [e0] Vendor Specific Information <?> 00:1f.2 IDE interface: Intel Corporation 82801GB/GR/GH (ICH7 Family) SATA IDE Controller (rev 01) (prog-if 8f [Master SecP SecO PriP PriO]) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 19 I/O ports at f700 [size=8] I/O ports at f600 [size=4] I/O ports at f500 [size=8] I/O ports at f400 [size=4] I/O ports at f300 [size=16] Capabilities: [70] Power Management version 2 Kernel driver in use: ata_piix 00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: medium devsel, IRQ 19 I/O ports at 0500 [size=32] Kernel driver in use: i801_smbus 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02) Subsystem: Holco Enterprise Co, Ltd/Shuttle Computer Device 3166 Flags: bus master, fast devsel, latency 0, IRQ 25 I/O ports at ce00 [size=256] Memory at fdbff000 (64-bit, non-prefetchable) [size=4K] Memory at fdcf0000 (64-bit, prefetchable) [size=64K] [virtual] Expansion ROM at fdc00000 [disabled] [size=128K] Capabilities: [40] Power Management version 3 Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Count=1/1 Enable+ Capabilities: [70] Express Endpoint, MSI 01 Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2 Capabilities: [d0] Vital Product Data <?> Capabilities: [100] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSVoil- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSVoil- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSVoil- CESta: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [140] Virtual Channel <?> Capabilities: [160] Device Serial Number 00-e0-4c-68-00-00-04-ab Kernel driver in use: r8169
have you tried with the last kernel
I think log/information regarding 2.6.30 kernel will be more useful. The uploads bug in 2.6.29-r5 is probably fixed("avoid losing MSI interrupts" patch). So, we want to see what happens with 2.6.30, and crashes the machine. FYI, 2.6.30 has 3 patches for the r8169 driver, the one, which is in gentoo-sources-2.6.29-r5 too(fix crash when large packets are received), the one which probably fixes the upload bug(mentioned above), and commit d78ad8cbfe73ad568de38814a75e9c92ad0a907c(reset IntrStatus after chip reset).
(In reply to comment #7) > I think log/information regarding 2.6.30 kernel will be more useful. > > The uploads bug in 2.6.29-r5 is probably fixed("avoid losing MSI interrupts" > patch). > So, we want to see what happens with 2.6.30, and crashes the machine. > > FYI, 2.6.30 has 3 patches for the r8169 driver, the one, which is in > gentoo-sources-2.6.29-r5 too(fix crash when large packets are received), the > one which probably fixes the upload bug(mentioned above), and commit > d78ad8cbfe73ad568de38814a75e9c92ad0a907c(reset IntrStatus after chip reset). I'm sorry. Pleace could you tell me what log files should I try to get with the 2.6.30 kernel? Logs from the crash are quite difficult to get, becaus the system freeze happens so fast, and it's verry common and fast when downloading something with it (at least with torrent).
the crash happens only when downloading torrents? if you rmmod r8169, or you don't use your nic at all, does the crash happen again? about dmesg, you can post the dmesg output after a clean boot, and/or the dmesg output when you rmmod r8169 && modprobe -v r8169
Created attachment 199235 [details] dmesg output after boot with 2.6.30-r3 kernel
I have been doing some testing with the 2.6.30-r3 kernel few hours, and now it took quite long to freeze the system. I had succesfull 20 GB FTP uploads and Downloads in 1Gbps LAN with no problems at all. Both directions can handle ~60 MB/s speeds verry well. It took about 20 minutes of heavy use of Ktorrent after I got the system to freeze, which was longer that I was expecting it to take. I got also some 'swapper: page allocation failure.' errors in dmesg while using the ktorrent when the system was alive. Here are few of those: swapper: page allocation failure. order:0, mode:0x20 Pid: 0, comm: swapper Not tainted 2.6.30-gentoo-r3 #1 Call Trace: <IRQ> [<ffffffff802ad21f>] __alloc_pages_internal+0x411/0x4ff [<ffffffff802d3d3e>] __slab_alloc+0x611/0x6c3 [<ffffffff806e4e48>] ? __alloc_skb+0x37/0x147 [<ffffffff806e4e48>] ? __alloc_skb+0x37/0x147 [<ffffffff802d3fa3>] kmem_cache_alloc+0xe6/0xee [<ffffffff806e4e48>] __alloc_skb+0x37/0x147 [<ffffffff806e5b9f>] __netdev_alloc_skb+0x1f/0x3e [<ffffffff805ec864>] rtl8169_rx_fill+0xbb/0x238 [<ffffffff805ecd42>] rtl8169_rx_interrupt+0x361/0x4e7 [<ffffffff805f0006>] rtl8169_poll+0x40/0x248 [<ffffffff806e9d1c>] net_rx_action+0xe9/0x196 [<ffffffff8026314e>] __do_softirq+0xaf/0x181 [<ffffffff8022cf3c>] call_softirq+0x1c/0x30 [<ffffffff8022e93d>] do_softirq+0x3d/0x78 [<ffffffff80262e55>] irq_exit+0x74/0x88 [<ffffffff8022e15d>] do_IRQ+0x7a/0xcc [<ffffffff8022c813>] ret_from_intr+0x0/0xa <EOI> [<ffffffff8023377c>] ? mwait_idle+0xfe/0x100 [<ffffffff8027723d>] ? atomic_notifier_call_chain+0x15/0x17 [<ffffffff8022b13c>] ? cpu_idle+0x5c/0xa2 [<ffffffff80836fb5>] ? rest_init+0x65/0x70 [<ffffffff80ae8ced>] ? start_kernel+0x35b/0x422 [<ffffffff80ae8289>] ? x86_64_start_reservations+0x99/0xb9 [<ffffffff80ae8389>] ? x86_64_start_kernel+0xe0/0xf2 Mem-Info: DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 CPU 1: hi: 0, btch: 1 usd: 0 CPU 2: hi: 0, btch: 1 usd: 0 CPU 3: hi: 0, btch: 1 usd: 0 DMA32 per-cpu: CPU 0: hi: 186, btch: 31 usd: 104 CPU 1: hi: 186, btch: 31 usd: 181 CPU 2: hi: 186, btch: 31 usd: 163 CPU 3: hi: 186, btch: 31 usd: 153 Active_anon:81457 active_file:146824 inactive_anon:28818 inactive_file:222500 unevictable:29 dirty:42225 writeback:1602 unstable:0 free:2070 slab:20032 mapped:9546 pagetables:3852 bounce:0 DMA free:6104kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:4920kB pages_scanned:0 all_unreclaimable? yes lowmem_reserve[]: 0 1995 1995 1995 DMA32 free:2176kB min:5708kB low:7132kB high:8560kB active_anon:325828kB inactive_anon:115272kB active_file:587296kB inactive_file:890000kB unevictable:116kB present:2043100kB pages_scanned:275 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 DMA: 2*4kB 4*8kB 3*16kB 6*32kB 5*64kB 1*128kB 3*256kB 1*512kB 2*1024kB 1*2048kB 0*4096kB = 6104kB DMA32: 228*4kB 48*8kB 15*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 2176kB 400489 total pagecache pages 1316 pages in swap cache Swap cache stats: add 63675, delete 62359, find 18647/21975 Free swap = 440316kB Total swap = 530136kB 521936 pages RAM 10638 pages reserved 411030 pages shared 117154 pages non-shared swapper: page allocation failure. order:0, mode:0x4020 Pid: 0, comm: swapper Not tainted 2.6.30-gentoo-r3 #1 Call Trace: <IRQ> [<ffffffff802ad21f>] __alloc_pages_internal+0x411/0x4ff [<ffffffff802d3d3e>] __slab_alloc+0x611/0x6c3 [<ffffffff806e5b9f>] ? __netdev_alloc_skb+0x1f/0x3e [<ffffffff802d46e0>] __kmalloc_track_caller+0xfe/0x193 [<ffffffff806e5b9f>] ? __netdev_alloc_skb+0x1f/0x3e [<ffffffff806e4e72>] __alloc_skb+0x61/0x147 [<ffffffff806e5b9f>] __netdev_alloc_skb+0x1f/0x3e [<ffffffff805ec864>] rtl8169_rx_fill+0xbb/0x238 [<ffffffff805ecd42>] rtl8169_rx_interrupt+0x361/0x4e7 [<ffffffff802d21b3>] ? __free_slab+0xa7/0x15a [<ffffffff805f0006>] rtl8169_poll+0x40/0x248 [<ffffffff806e9d1c>] net_rx_action+0xe9/0x196 [<ffffffff8026314e>] __do_softirq+0xaf/0x181 [<ffffffff8022cf3c>] call_softirq+0x1c/0x30 [<ffffffff8022e93d>] do_softirq+0x3d/0x78 [<ffffffff80262e55>] irq_exit+0x74/0x88 [<ffffffff8022e15d>] do_IRQ+0x7a/0xcc [<ffffffff8022c813>] ret_from_intr+0x0/0xa <EOI> [<ffffffff8023377c>] ? mwait_idle+0xfe/0x100 [<ffffffff8027723d>] ? atomic_notifier_call_chain+0x15/0x17 [<ffffffff8022b13c>] ? cpu_idle+0x5c/0xa2 [<ffffffff80836fb5>] ? rest_init+0x65/0x70 [<ffffffff80ae8ced>] ? start_kernel+0x35b/0x422 [<ffffffff80ae8289>] ? x86_64_start_reservations+0x99/0xb9 [<ffffffff80ae8389>] ? x86_64_start_kernel+0xe0/0xf2 Mem-Info: DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 CPU 1: hi: 0, btch: 1 usd: 0 CPU 2: hi: 0, btch: 1 usd: 0 CPU 3: hi: 0, btch: 1 usd: 0 DMA32 per-cpu: CPU 0: hi: 186, btch: 31 usd: 103 CPU 1: hi: 186, btch: 31 usd: 181 CPU 2: hi: 186, btch: 31 usd: 163 CPU 3: hi: 186, btch: 31 usd: 153 Active_anon:81457 active_file:146824 inactive_anon:28818 inactive_file:222500 unevictable:29 dirty:42225 writeback:1602 unstable:0 free:2038 slab:20064 mapped:9546 pagetables:3852 bounce:0 DMA free:6104kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:4920kB pages_scanned:0 all_unreclaimable? yes lowmem_reserve[]: 0 1995 1995 1995 DMA32 free:2048kB min:5708kB low:7132kB high:8560kB active_anon:325828kB inactive_anon:115272kB active_file:587296kB inactive_file:890000kB unevictable:116kB present:2043100kB pages_scanned:275 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 DMA: 2*4kB 4*8kB 3*16kB 6*32kB 5*64kB 1*128kB 3*256kB 1*512kB 2*1024kB 1*2048kB 0*4096kB = 6104kB DMA32: 228*4kB 48*8kB 15*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 2048kB 400489 total pagecache pages 1316 pages in swap cache Swap cache stats: add 63675, delete 62359, find 18647/21975 Free swap = 440316kB Total swap = 530136kB 521936 pages RAM 10638 pages reserved 411012 pages shared 117173 pages non-shared swapper: page allocation failure. order:0, mode:0x4020 Pid: 0, comm: swapper Not tainted 2.6.30-gentoo-r3 #1 Call Trace: <IRQ> [<ffffffff802ad21f>] __alloc_pages_internal+0x411/0x4ff [<ffffffff802d3d3e>] __slab_alloc+0x611/0x6c3 [<ffffffff806e5b9f>] ? __netdev_alloc_skb+0x1f/0x3e [<ffffffff802d46e0>] __kmalloc_track_caller+0xfe/0x193 [<ffffffff806e5b9f>] ? __netdev_alloc_skb+0x1f/0x3e [<ffffffff806e4e72>] __alloc_skb+0x61/0x147 [<ffffffff806e5b9f>] __netdev_alloc_skb+0x1f/0x3e [<ffffffff805ec864>] rtl8169_rx_fill+0xbb/0x238 [<ffffffff805ecd42>] rtl8169_rx_interrupt+0x361/0x4e7 [<ffffffff8022c813>] ? ret_from_intr+0x0/0xa [<ffffffff805f0006>] rtl8169_poll+0x40/0x248 [<ffffffff806e9d1c>] net_rx_action+0xe9/0x196 [<ffffffff8026314e>] __do_softirq+0xaf/0x181 [<ffffffff8022cf3c>] call_softirq+0x1c/0x30 [<ffffffff8022e93d>] do_softirq+0x3d/0x78 [<ffffffff80262e55>] irq_exit+0x74/0x88 [<ffffffff8022e15d>] do_IRQ+0x7a/0xcc [<ffffffff8022c813>] ret_from_intr+0x0/0xa <EOI> [<ffffffff8023377c>] ? mwait_idle+0xfe/0x100 [<ffffffff8027723d>] ? atomic_notifier_call_chain+0x15/0x17 [<ffffffff8022b13c>] ? cpu_idle+0x5c/0xa2 [<ffffffff80836fb5>] ? rest_init+0x65/0x70 [<ffffffff80ae8ced>] ? start_kernel+0x35b/0x422 [<ffffffff80ae8289>] ? x86_64_start_reservations+0x99/0xb9 [<ffffffff80ae8389>] ? x86_64_start_kernel+0xe0/0xf2 Mem-Info: DMA per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 CPU 1: hi: 0, btch: 1 usd: 0 CPU 2: hi: 0, btch: 1 usd: 0 CPU 3: hi: 0, btch: 1 usd: 0 DMA32 per-cpu: CPU 0: hi: 186, btch: 31 usd: 173 CPU 1: hi: 186, btch: 31 usd: 44 CPU 2: hi: 186, btch: 31 usd: 17 CPU 3: hi: 186, btch: 31 usd: 169 Active_anon:80236 active_file:149213 inactive_anon:28917 inactive_file:220910 unevictable:29 dirty:44895 writeback:690 unstable:0 free:2056 slab:20497 mapped:10155 pagetables:3852 bounce:0 DMA free:6104kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB present:4920kB pages_scanned:0 all_unreclaimable? yes lowmem_reserve[]: 0 1995 1995 1995 DMA32 free:2120kB min:5708kB low:7132kB high:8560kB active_anon:320944kB inactive_anon:115668kB active_file:596852kB inactive_file:883640kB unevictable:116kB present:2043100kB pages_scanned:94 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 DMA: 2*4kB 4*8kB 3*16kB 6*32kB 5*64kB 1*128kB 3*256kB 1*512kB 2*1024kB 1*2048kB 0*4096kB = 6104kB DMA32: 261*4kB 43*8kB 32*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2156kB 401173 total pagecache pages 1224 pages in swap cache Swap cache stats: add 65034, delete 63810, find 18647/21978 Free swap = 434980kB Total swap = 530136kB 521936 pages RAM 10638 pages reserved 414741 pages shared 114303 pages non-shared
Did this issue start occuring when you upgraded to the 2.6.29 kernel versions? Did 2.6.28 work fine?
I don't recall having any system freeze problems before 2.6.30 kernel series. This system is built in early this year (in February IIRC), but I don't remember what are the first kernel versions that I've used first. At least the gentoo-sources-2.6.29-r5 has been working without any system freeze problems for months now.
This looks similar to this: http://bugzilla.kernel.org/show_bug.cgi?id=13561 Looks like the same symptom where the DMA32 zone is below the minimum threshold.
upstream reports this as not a bug. See upstream bug comment #3 for details.
