After compile any sys-kernel/hardened-sources-2.6.32-rXX got many kernel information in dmesg, looks like kernel got problem with ocfs2. Reproducible: Always Steps to Reproduce: 1. Compile kernel with ocfs2 support 2. mount hard disc with ocfs2 3. Copy and delete files from ocfs2 disc Actual Results: INFO: task o2quot/0:11310 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. o2quot/0 D f9408e42 2724 11310 2 0x00000000 f74d0000 00000046 f94093de f9408e42 22222222 00000000 68461324 00000468 68463739 00000468 00000000 00007140 f74d0284 00007140 00002254 00000000 f74d0374 f74d0000 f74d0284 f74d0370 f74d0000 f74d0370 f6eda600 00000000 Call Trace: [<f94093de>] ? ocfs2_set_buffer_uptodate+0x10/0x6da [ocfs2] [<f9408e42>] ? ocfs2_buffer_cached+0x116/0x190 [ocfs2] [<c073d585>] ? rwsem_down_failed_common+0x13b/0x151 [<f93d3048>] ? ocfs2_read_inode_block_full+0x2f/0x44 [ocfs2] [<c073d5d4>] ? rwsem_down_read_failed+0x1a/0x26 [<c073d617>] ? call_rwsem_down_read_failed+0x7/0x10 [<c073cea2>] ? down_read+0xa/0xb [<f93ddea4>] ? ocfs2_start_trans+0x8c/0x18e [ocfs2] [<f94104fd>] ? ocfs2_sync_dquot_helper+0xe1/0x397 [ocfs2] [<c04dfe5d>] ? dquot_scan_active+0x6c/0xbb [<f941041c>] ? ocfs2_sync_dquot_helper+0x0/0x397 [ocfs2] [<f940f673>] ? qsync_work_fn+0x23/0x3b [ocfs2] [<c0443743>] ? worker_thread+0x174/0x1fd [<f940f650>] ? qsync_work_fn+0x0/0x3b [ocfs2] [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<c04435cf>] ? worker_thread+0x0/0x1fd [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task jbd2/dm-0-61:12697 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. jbd2/dm-0-61 D c04c90ce 2288 12697 2 0x00000000 f7584550 00000046 c04c90ce c04c90ce c19cd748 00000002 5ca4db64 00000469 5ca4e152 00000469 c19cd748 00007140 f75847d4 00007140 00002254 c272ee9c f75848c4 f7584550 f75847d4 f75848c0 00000001 c04c909b 00000002 c04c951a Call Trace: [<c04c90ce>] ? sync_buffer+0x0/0x33 [<c04c90ce>] ? sync_buffer+0x0/0x33 [<c04c909b>] ? __wait_on_buffer+0x16/0x18 [<c04c951a>] ? sync_dirty_buffer+0x63/0x94 [<fc19a152>] ? jbd2_journal_commit_transaction+0x1a8/0xf72 [jbd2] [<c040162c>] ? __switch_to+0x10/0x181 [<c073bdd0>] ? schedule+0xacb/0xafd [<c073d7ba>] ? _spin_lock_irqsave+0x11/0x2a [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<c073d92f>] ? _spin_unlock_irqrestore+0xe/0x20 [<c043cd66>] ? try_to_del_timer_sync+0x96/0x9d [<fc19fe14>] ? kjournald2+0x9e/0x1c7 [jbd2] [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<fc19fd76>] ? kjournald2+0x0/0x1c7 [jbd2] [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task ocfs2cmt:12698 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. ocfs2cmt D 00000000 2880 12698 2 0x00000000 f7574000 00000046 0000044d 00000000 00000000 00000002 28fbe63d 00000468 28fbea8a 00000468 c0420487 00007140 f7574284 00007140 00002254 c042d588 f7574374 00000000 f7574284 f7574370 00000002 00000002 c2025f9c c1fbd4d8 Call Trace: [<c0420487>] ? check_preempt_curr+0x1c/0x3c [<c042d588>] ? try_to_wake_up+0x3f2/0x40d [<c04469ef>] ? autoremove_wake_function+0xd/0x29 [<c073d585>] ? rwsem_down_failed_common+0x13b/0x151 [<c073d5b3>] ? rwsem_down_write_failed+0x18/0x1f [<c073d626>] ? call_rwsem_down_write_failed+0x6/0x10 [<c073ceb5>] ? down_write+0x12/0x13 [<f93dcba7>] ? ocfs2_commit_thread+0x12f/0x4ff [ocfs2] [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<f93dca78>] ? ocfs2_commit_thread+0x0/0x4ff [ocfs2] [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task rsync:13284 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rsync D e786ab00 1860 13284 13283 0x00000000 c2176000 00000082 c1526e40 e786ab00 00000006 00000003 28fbd816 00000468 28fbf0ca 00000468 e786ab00 00007140 c2176284 00007140 00002254 c2176370 c2176374 c2176000 c2176284 c2176370 00000282 00000000 c1526e7c 00000046 Call Trace: [<c073d585>] ? rwsem_down_failed_common+0x13b/0x151 [<c073d5d4>] ? rwsem_down_read_failed+0x1a/0x26 [<c073d617>] ? call_rwsem_down_read_failed+0x7/0x10 [<c073cea2>] ? down_read+0xa/0xb [<f93ddea4>] ? ocfs2_start_trans+0x8c/0x18e [ocfs2] [<f940a0f8>] ? ocfs2_modify_bh+0x24/0x300 [ocfs2] [<f9409dd8>] ? olq_alloc_dquot+0x0/0xe [ocfs2] [<f940c047>] ? ocfs2_local_read_dquot+0xc68/0xeb0 [ocfs2] [<c04dd144>] ? dquot_acquire+0x3c/0xaf [<f940fef3>] ? ocfs2_acquire_dquot+0xa5/0x124 [ocfs2] [<c04df26e>] ? dqget+0x257/0x28c [<c04dfbc7>] ? dquot_initialize+0x60/0x11b [<c04b3487>] ? vfs_create+0xaa/0x127 [<c04b54fd>] ? do_filp_open+0x310/0x949 [<c04bd8ca>] ? expand_files+0x27/0x220 [<c073d98f>] ? _spin_unlock+0xc/0x1e [<c04bdb7e>] ? alloc_fd+0xbb/0xc8 [<c04a8e58>] ? do_sys_open+0x4f/0xe3 [<c04a8f30>] ? sys_open+0x1e/0x23 [<c0402a0d>] ? sysenter_do_call+0x12/0x22 INFO: task kswapd0:364 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kswapd0 D 28c4fe9b 2572 364 2 0x00000000 f75c6000 00000046 00000468 28c4fe9b 00000468 00000001 28fbdc8c 00000468 28fbe001 00000468 00000000 00007140 f75c6284 00007140 00002254 f75c6000 f75c6374 f75c6000 f75c6284 f75c6370 f76d5e18 c2170414 c2170444 00112294 Call Trace: [<c073c647>] ? __mutex_lock_slowpath+0x11c/0x21e [<c073c751>] ? mutex_lock+0x8/0x13 [<c04dd0af>] ? dquot_release+0x2c/0x85 [<f940fdc4>] ? ocfs2_release_dquot+0x189/0x213 [ocfs2] [<c04dea0a>] ? dqput+0x177/0x1c2 [<c04df00a>] ? dquot_drop+0x4c/0x59 [<c04dcdd1>] ? vfs_dq_drop+0x35/0x36 [<c04bc25d>] ? clear_inode+0x4f/0xaf [<c04bc473>] ? dispose_list+0x36/0xc5 [<c04bc6b1>] ? shrink_icache_memory+0x1af/0x1d5 [<c048ab1e>] ? shrink_slab+0xe3/0x166 [<c048af8a>] ? kswapd+0x3e9/0x56d [<c0488fd6>] ? isolate_pages_global+0x0/0x1a5 [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<c04237db>] ? complete+0x28/0x36 [<c048aba1>] ? kswapd+0x0/0x56d [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task o2quot/0:11310 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. o2quot/0 D f9408e42 2724 11310 2 0x00000000 f74d0000 00000046 f94093de f9408e42 22222222 00000000 68461324 00000468 68463739 00000468 00000000 00007140 f74d0284 00007140 00002254 00000000 f74d0374 f74d0000 f74d0284 f74d0370 f74d0000 f74d0370 f6eda600 00000000 Call Trace: [<f94093de>] ? ocfs2_set_buffer_uptodate+0x10/0x6da [ocfs2] [<f9408e42>] ? ocfs2_buffer_cached+0x116/0x190 [ocfs2] [<c073d585>] ? rwsem_down_failed_common+0x13b/0x151 [<f93d3048>] ? ocfs2_read_inode_block_full+0x2f/0x44 [ocfs2] [<c073d5d4>] ? rwsem_down_read_failed+0x1a/0x26 [<c073d617>] ? call_rwsem_down_read_failed+0x7/0x10 [<c073cea2>] ? down_read+0xa/0xb [<f93ddea4>] ? ocfs2_start_trans+0x8c/0x18e [ocfs2] [<f94104fd>] ? ocfs2_sync_dquot_helper+0xe1/0x397 [ocfs2] [<c04dfe5d>] ? dquot_scan_active+0x6c/0xbb [<f941041c>] ? ocfs2_sync_dquot_helper+0x0/0x397 [ocfs2] [<f940f673>] ? qsync_work_fn+0x23/0x3b [ocfs2] [<c0443743>] ? worker_thread+0x174/0x1fd [<f940f650>] ? qsync_work_fn+0x0/0x3b [ocfs2] [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<c04435cf>] ? worker_thread+0x0/0x1fd [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task jbd2/dm-0-61:12697 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. jbd2/dm-0-61 D c04c90ce 2288 12697 2 0x00000000 f7584550 00000046 c04c90ce c04c90ce c19cd748 00000002 5ca4db64 00000469 5ca4e152 00000469 c19cd748 00007140 f75847d4 00007140 00002254 c272ee9c f75848c4 f7584550 f75847d4 f75848c0 00000001 c04c909b 00000002 c04c951a Call Trace: [<c04c90ce>] ? sync_buffer+0x0/0x33 [<c04c90ce>] ? sync_buffer+0x0/0x33 [<c04c909b>] ? __wait_on_buffer+0x16/0x18 [<c04c951a>] ? sync_dirty_buffer+0x63/0x94 [<fc19a152>] ? jbd2_journal_commit_transaction+0x1a8/0xf72 [jbd2] [<c040162c>] ? __switch_to+0x10/0x181 [<c073bdd0>] ? schedule+0xacb/0xafd [<c073d7ba>] ? _spin_lock_irqsave+0x11/0x2a [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<c073d92f>] ? _spin_unlock_irqrestore+0xe/0x20 [<c043cd66>] ? try_to_del_timer_sync+0x96/0x9d [<fc19fe14>] ? kjournald2+0x9e/0x1c7 [jbd2] [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<fc19fd76>] ? kjournald2+0x0/0x1c7 [jbd2] [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task ocfs2cmt:12698 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. ocfs2cmt D 00000000 2880 12698 2 0x00000000 f7574000 00000046 0000044d 00000000 00000000 00000002 28fbe63d 00000468 28fbea8a 00000468 c0420487 00007140 f7574284 00007140 00002254 c042d588 f7574374 00000000 f7574284 f7574370 00000002 00000002 c2025f9c c1fbd4d8 Call Trace: [<c0420487>] ? check_preempt_curr+0x1c/0x3c [<c042d588>] ? try_to_wake_up+0x3f2/0x40d [<c04469ef>] ? autoremove_wake_function+0xd/0x29 [<c073d585>] ? rwsem_down_failed_common+0x13b/0x151 [<c073d5b3>] ? rwsem_down_write_failed+0x18/0x1f [<c073d626>] ? call_rwsem_down_write_failed+0x6/0x10 [<c073ceb5>] ? down_write+0x12/0x13 [<f93dcba7>] ? ocfs2_commit_thread+0x12f/0x4ff [ocfs2] [<c04469e2>] ? autoremove_wake_function+0x0/0x29 [<f93dca78>] ? ocfs2_commit_thread+0x0/0x4ff [ocfs2] [<c044678c>] ? kthread+0x5f/0x64 [<c044672d>] ? kthread+0x0/0x64 [<c0403607>] ? kernel_thread_helper+0x7/0x10 INFO: task rsync:13284 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rsync D e786ab00 1860 13284 13283 0x00000000 c2176000 00000082 c1526e40 e786ab00 00000006 00000003 28fbd816 00000468 28fbf0ca 00000468 e786ab00 00007140 c2176284 00007140 00002254 c2176370 c2176374 c2176000 c2176284 c2176370 00000282 00000000 c1526e7c 00000046 Call Trace: [<c073d585>] ? rwsem_down_failed_common+0x13b/0x151 [<c073d5d4>] ? rwsem_down_read_failed+0x1a/0x26 [<c073d617>] ? call_rwsem_down_read_failed+0x7/0x10 [<c073cea2>] ? down_read+0xa/0xb [<f93ddea4>] ? ocfs2_start_trans+0x8c/0x18e [ocfs2] [<f940a0f8>] ? ocfs2_modify_bh+0x24/0x300 [ocfs2] [<f9409dd8>] ? olq_alloc_dquot+0x0/0xe [ocfs2] [<f940c047>] ? ocfs2_local_read_dquot+0xc68/0xeb0 [ocfs2] [<c04dd144>] ? dquot_acquire+0x3c/0xaf [<f940fef3>] ? ocfs2_acquire_dquot+0xa5/0x124 [ocfs2] [<c04df26e>] ? dqget+0x257/0x28c [<c04dfbc7>] ? dquot_initialize+0x60/0x11b [<c04b3487>] ? vfs_create+0xaa/0x127 [<c04b54fd>] ? do_filp_open+0x310/0x949 [<c04bd8ca>] ? expand_files+0x27/0x220 [<c073d98f>] ? _spin_unlock+0xc/0x1e [<c04bdb7e>] ? alloc_fd+0xbb/0xc8 [<c04a8e58>] ? do_sys_open+0x4f/0xe3 [<c04a8f30>] ? sys_open+0x1e/0x23 [<c0402a0d>] ? sysenter_do_call+0x12/0x22 Expected Results: Copy and delete files from ocfs2 filesystem with no problem.
Sorry just saw this bug. Can you give me your kernel config file and I'll try to reproduce here.
Created attachment 293607 [details] Kernel config Hello this is kernel config from my main server with ocfs2.
Thanks, everything looks sane in your config file. Unfortunately, it would be very hard for me to reproduced this. I'm going to pass this upstream.
can you enable all the lock debugging options and see if they detect anything? also trying 3.1.x may be interesting too.
Hey i check 3.X and working good.
(In reply to comment #5) > Hey i check 3.X and working good. Which one specifically.
This both: sys-kernel/vanilla-sources-3.1.1 ~x86 sys-kernel/hardened-sources-3.1.1-r1
(In reply to comment #7) > This both: > sys-kernel/vanilla-sources-3.1.1 ~x86 > sys-kernel/hardened-sources-3.1.1-r1 Was the problem happening with earlier vanilla kernels too?
This is problem with linux/fs/ocfs2 files in kernel. If i good understand some distro give a patch for this. I contact with OCFS2 developers and get this info.
(In reply to comment #9) > This is problem with linux/fs/ocfs2 files in kernel. > If i good understand some distro give a patch for this. > I contact with OCFS2 developers and get this info. Is this a problem also with the vanilla kernels? If so then this isn't a hardened issue.
ancient bug, reopen if this is still an issue.
Hello After 2 years fighting with OCFS2 developers and checking 100+ patches i stop using OCFS2 filesystem. This is the most unstable FS in kernel. Thanks
