Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 128870 - sys-kernel/hardened-sources-2.6.16 deadlock in conjunction with net-misc/ntp
Summary: sys-kernel/hardened-sources-2.6.16 deadlock in conjunction with net-misc/ntp
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Hardened (show other bugs)
Hardware: All Linux
: High normal
Assignee: The Gentoo Linux Hardened Team
URL:
Whiteboard:
Keywords:
: 129944 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-04-05 00:55 UTC by Jiri
Modified: 2007-04-29 14:07 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
straces and .config (2614deadlock.tar.gz,6.93 KB, application/octet-stream)
2006-04-05 12:20 UTC, Jiri
Details
straces and .config (2614deadlock.tar.gz,6.93 KB, application/octet-stream)
2006-04-05 12:22 UTC, Jiri
Details
gentoo-hardened 2.6.16 .config (.config,40.10 KB, text/plain)
2006-04-05 12:27 UTC, Jiri
Details
straces, dmesg, ... (ntpd-hardened2.6.14-lockout.tar,40.00 KB, application/octet-stream)
2006-04-06 11:22 UTC, Jiri
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri 2006-04-05 00:55:12 UTC
I have recently upgraded kernel hardened-sources from version 2.6.14 to 2.6.16 and  I am finding problem with net-misc/ntp-4.2.0.20040617-r3.
When I start ntpd, the process freezes and takes all CPU power, next some programs hangs too (for example ps -A -f).
It looks like kernel deadlock. With 2.6.14 everything works fine.
Comment 1 Jiri 2006-04-05 00:57:20 UTC
(In reply to comment #0)
> I have recently upgraded kernel hardened-sources from version 2.6.14 to 2.6.16
> and  I am finding problem with net-misc/ntp-4.2.0.20040617-r3.
> When I start ntpd, the process freezes and takes all CPU power, next some
> programs hangs too (for example ps -A -f).
> It looks like kernel deadlock. With 2.6.14 everything works fine.
> 

I am using only PAX, not GRSEC.
Comment 2 John Mylchreest (RETIRED) gentoo-dev 2006-04-05 05:49:23 UTC
just to try and isolate this, can you please try again with a vanilla 2.6.16 kernel?
Comment 3 PaX Team 2006-04-05 07:29:45 UTC
1. can you attach your .config?
2. per comment #2, please try vanilla as well
3. can you try the latest grsec test patch (from http://grsecurity.net/~spender/)?
4. are there any kernel messages/logs?
5. since it seems to be reproducible, can you run ntp through strace (repeatedly if necessary) until it hangs then attach the output (or the last few lines at least)?
Comment 4 John Mylchreest (RETIRED) gentoo-dev 2006-04-05 09:03:25 UTC
(In reply to comment #3)
> 3. can you try the latest grsec test patch (from
> http://grsecurity.net/~spender/)?

I am not aware of any further grsec changes which aren't already included. Am I missing something? :P

> 4. are there any kernel messages/logs?
> 5. since it seems to be reproducible, can you run ntp through strace
> (repeatedly if necessary) until it hangs then attach the output (or the last
> few lines at least)?
Comment 5 PaX Team 2006-04-05 10:00:48 UTC
(In reply to comment #4)
> I am not aware of any further grsec changes which aren't already included. Am I
> missing something? :P

$ tar tjvf /usr/portage/distfiles/hardened-patches-2.6.16-1.extras.tar.bz2|grep grsec-

says 2.6.16/4450_grsec-2.1.9-2.6.16-200603292139.patch whereas current is grsecurity-2.1.9-2.6.16.1-200604041154.patch 
Comment 6 Jiri 2006-04-05 12:20:46 UTC
Created attachment 84009 [details]
straces and .config
Comment 7 Jiri 2006-04-05 12:22:56 UTC
Created attachment 84011 [details]
straces and .config

The error is easily reproductible, I copied .config to my notebook and it works the same way.
There was nothing special in dmesg output but I am sory I had dmesg log rewriten so it is not attached.
So here is straces and .config, bud lockout of ps -A -f I was unable to trace into file.
Comment 8 Jiri 2006-04-05 12:27:30 UTC
Created attachment 84012 [details]
gentoo-hardened 2.6.16 .config

I have forgotten the .config
Comment 9 Jiri 2006-04-05 12:39:11 UTC
I have tried with suspend2-sources 2.6.16-suspend2-r1 without any problem, but with different .config...

(In reply to comment #2)
> just to try and isolate this, can you please try again with a vanilla 2.6.16
> kernel?
> 

Comment 10 PaX Team 2006-04-06 02:41:59 UTC
1. can you use strace -f on ntp please (it forks, and i guess the interesting things happen in the child)?

2. can you post the output of "paxctl -vQ /usr/sbin/ntpd" and "readelf -e /usr/sbin/ntpd"?

3. for kernel logs look at /var/log/messages.
Comment 11 Jiri 2006-04-06 11:22:49 UTC
Created attachment 84073 [details]
straces, dmesg, ...

straces, dmesg, ...
Comment 12 Jiri 2006-04-06 12:47:08 UTC
I have tried latest grsec patch with vanilla 2.6.16.1 with the same result.

(In reply to comment #3)
> 1. can you attach your .config?
> 2. per comment #2, please try vanilla as well
> 3. can you try the latest grsec test patch (from
> http://grsecurity.net/~spender/)?
> 4. are there any kernel messages/logs?
> 5. since it seems to be reproducible, can you run ntp through strace
> (repeatedly if necessary) until it hangs then attach the output (or the last
> few lines at least)?
> 

Comment 13 Daniel Kontsek 2006-04-13 07:31:49 UTC
same here:

net-misc/ntp-4.2.0.20050303-r1
sys-kernel/hardened-sources-2.6.16-r2

after running /etc/init.d/ntpd start the system freezes. With 2.6.14-hardened everything was ok. 
Comment 14 PaX Team 2006-04-13 09:42:12 UTC
(In reply to comment #11)
> Created an attachment (id=84073) [edit]
> straces, dmesg, ...

ok, a few comments, some not related to this bug though.

1. your dmesg shows that PaX killed your X server, that's because you're not using the new modular X server, but the old one with the elfloader, you should really upgrade.

2. your ntpd binary has PT_PAX_FLAGS but it lacks PT_GNU_RELRO, i thought the hardened toolchain would enforce that.

3. ntpd apparently hangs in a mmap() call, which at first sight seems completely innocuous, so i don't yet see what the underlying problem is. would be nice if you could check your syslogs for more PaX messages (or any kernel BUG reports, oopses) at that point.

4. for comment #13, can you post an strace as well please?
Comment 15 Jiri 2006-04-13 12:25:04 UTC
It were not traces from pax-enabled system, just from my notebook which is used to work with suspend2-sources...
Thanks anyway.

(In reply to comment #14)
> (In reply to comment #11)
> > Created an attachment (id=84073) [edit]
> > straces, dmesg, ...
> 
> ok, a few comments, some not related to this bug though.
> 
> 1. your dmesg shows that PaX killed your X server, that's because you're not
> using the new modular X server, but the old one with the elfloader, you should
> really upgrade.
> 
> 2. your ntpd binary has PT_PAX_FLAGS but it lacks PT_GNU_RELRO, i thought the
> hardened toolchain would enforce that.
> 
> 3. ntpd apparently hangs in a mmap() call, which at first sight seems
> completely innocuous, so i don't yet see what the underlying problem is. would
> be nice if you could check your syslogs for more PaX messages (or any kernel
> BUG reports, oopses) at that point.
> 
> 4. for comment #13, can you post an strace as well please?
> 

Comment 16 petre rodan (RETIRED) gentoo-dev 2006-04-15 01:03:44 UTC
*** Bug 129944 has been marked as a duplicate of this bug. ***
Comment 17 John Mylchreest (RETIRED) gentoo-dev 2006-04-15 11:01:11 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > I am not aware of any further grsec changes which aren't already included. Am I
> > missing something? :P
> 
> $ tar tjvf /usr/portage/distfiles/hardened-patches-2.6.16-1.extras.tar.bz2|grep
> grsec-
> 
> says 2.6.16/4450_grsec-2.1.9-2.6.16-200603292139.patch whereas current is
> grsecurity-2.1.9-2.6.16.1-200604041154.patch 
> 

very belated reply, but the upstream fix just backed out the i810 fix (which was in genpatches-base). The one I included already did that anyways.
Comment 18 David 2006-04-22 14:42:51 UTC
I am sure I remember having this problem before, so I switched to net-misc/openntpd which seems to work fine.
Comment 19 Jon Skanes 2006-04-23 13:32:28 UTC
(In reply to comment #18)
> I am sure I remember having this problem before, so I switched to
> net-misc/openntpd which seems to work fine.
> 

That's no solution here.  OpenNTPD doesn't support some of the advanced time services which are very important for me.  We're using a radio as a backup time source to keep our kerberos in check.  Also, I'd like the ability to use the IPsec features of shorewall which require some of the new functionality in 2.6.16.

Anything I can do to help?  I don't have much debugging experience but if someone were willing to guide me, I would be more then happy to allocate some brain cycles to this.

Thank,
Jon
Comment 20 solar (RETIRED) gentoo-dev 2006-04-23 14:15:06 UTC
Appears to be a problem in vanilla grsec and was reported upstream.
http://grsecurity.net/pipermail/grsecurity/2006-April/000726.html
Comment 21 PaX Team 2006-04-24 04:03:06 UTC
guys, yesterday i managed to reproduce the hang and am already debugging it. in the meantime you can disable SEGMEXEC on the affected executables and they should be fine (if you have PAGEEXEC enabled in the kernel .config, it will be used as a fall back automatically). the problem affects anything that uses mlockall and vma mirroring (SEGMEXEC), so if you run across similar issues, add them here in the meantime.
Comment 22 PaX Team 2006-04-24 05:55:36 UTC
i uploaded test6 to http://www.grsecurity.net/~paxguy1/ , please give it a try.
Comment 23 petre rodan (RETIRED) gentoo-dev 2006-05-24 05:12:20 UTC
good job, hardened-sources-2.6.16-r6 fixes the ntpd problem for me
Comment 24 Christian Heim (RETIRED) gentoo-dev 2007-04-29 14:07:34 UTC
If this is still an issue for you with any current version (say 2.6.20-r2), feel free to REOPEN this bug.