Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 265693 - sys-devel/gdb-6.8-r2 ignores breakpoints and provides no symbolic bt on amd64 hardened
Summary: sys-devel/gdb-6.8-r2 ignores breakpoints and provides no symbolic bt on amd64...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Hardened (show other bugs)
Hardware: AMD64 Linux
: High normal (vote)
Assignee: The Gentoo Linux Hardened Team
URL:
Whiteboard:
Keywords:
: 288531 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-04-10 19:39 UTC by Hugo Mildenberger
Modified: 2010-10-19 10:31 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info, i686 , gdb-6.8-r1, gcc-4.3.4-r1 from hardened overlay (emergeinfo,4.96 KB, text/plain)
2009-10-05 15:15 UTC, Dillon
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hugo Mildenberger 2009-04-10 19:39:36 UTC
On hardened amd64, all currently available gdb ebuilds (including gdb-6.8.50.20090302.8.11) always ignore previously set breakpoint and do not provide a symbolic stack trace. 

Reproducible: Always

Steps to Reproduce:
1.emerge =sys-devel/gdb-6.8-r2
2.use gcc (Gentoo 4.3.2-r2 p1.5, pie-10.1.5) 4.3.2 to compile & link a test program.
Comment 1 Hugo Mildenberger 2009-04-10 19:46:22 UTC
Just tried to send emerge --info along, but encountered a bugzilla error. 
Comment 2 solar (RETIRED) gentoo-dev 2009-04-10 19:57:36 UTC
gdb + pie is always a work in progress, but -r1 should of worked as far as I'm aware of. https://bugs.gentoo.org/show_bug.cgi?id=223533
Comment 3 Hugo Mildenberger 2009-04-10 20:02:19 UTC
(In reply to comment #2)
I think I tried -r1. And I know -r1 worked on x86, but this is amd64.

But true: this is pretty much the same behaviour which was fixed by version 6.8-r1 on x86. A workaround is: use -nopie for compile AND link (this will enable symbolic backtraces), and use paxctl -m on executable target for breakpoints becoming effective again. 
Comment 4 Dillon 2009-10-05 15:15:57 UTC
Created attachment 206137 [details]
emerge --info, i686 , gdb-6.8-r1, gcc-4.3.4-r1 from hardened overlay

sys-devel/gdb-6.8-r1 is broken on x86 PIE binaries for me, it was built with gcc4 from the hardened-development overlay. It should be noted that linking with -nopie seems unnecessary, for me at least,  with paxctl -mr being sufficient to get breakpoints for me. Can anyone else confirm this?
Comment 5 Thomas Sachau gentoo-dev 2009-10-11 20:03:31 UTC
*** Bug 288531 has been marked as a duplicate of this bug. ***
Comment 6 Hugo Mildenberger 2009-10-12 00:56:41 UTC
In response to comment #4:
on x86, now gcc-4.3.4,linux-2.6.31.1-grsec, gdb-6.8-r1 is again unable to set breakpoints. Playing around with paxctl flags I found that paxctl -mrs restores gdb's ability to interrupt the debugee. Maybe I need to disable the SEGMEXEC protection additionally because I had to select that pax config on a Pentium IV platform.
Comment 7 kavol 2009-10-12 09:39:58 UTC
(In reply to comment #5)
> *** Bug 288531 has been marked as a duplicate of this bug. ***

ahem, are you sure?

1. This bug talks about software from Hardened overlay, the newer version of gcc, while I use the Portage's "stable" version, i.e. gcc version 3.4.6 (Gentoo Hardened 3.4.6-r2 p1.6, ssp-3.4.6-1.0, pie-8.7.10)
(sorry for not mentioning that explicitly)

2. The documentation needs updating anyways, there is no longer "=sys-devel/gdb-6.3-r5"
Comment 8 Magnus Granberg gentoo-dev 2010-01-19 16:56:39 UTC
Xake have added a new gdb in the overlay for testing that have
upstream PIE support that will be in 7.1
PS. It is masked and missing keyword.


Comment 9 Magnus Granberg gentoo-dev 2010-03-20 15:39:49 UTC
Can some one test with the gdb 7.1 in the tree that have built-in PIE support?
And for PAX enable kernels you need to do paxctl -m on the bin that want to debug.
Comment 10 kavol 2010-04-01 09:01:46 UTC
(In reply to comment #9)
> Can some one test with the gdb 7.1 in the tree that have built-in PIE support?

I've decomissioned the machine that exhibited my problem, sorry

anyways, thanks for your work and wish you good luck in resolving the issues!
Comment 11 Hugo Mildenberger 2010-07-30 13:16:50 UTC
gdb-7.1 shows the same behaviour. paxctl -m enables breakpoints, and paxctrl -r brings the backtrace back on hardened amd64. But I wonder if gdb actually being unable to use hardware breakpoints is part of the problem:

(gdb) hbreak main
No hardware breakpoint support in the target.

This does not depend on if you are running a hardened kernel or not. Not having hardware breakpoints available also means that memory watch points are unusable.

For me, the only way to use gdb on hardened is with core dumps, but even then backtraces are often incomplete due to a bug elsewhere in context of split-debug / debugedit (for this see bug #273279)
Comment 12 SpanKY gentoo-dev 2010-07-31 01:08:55 UTC
no x86/x86_64 system has ever supported hardware breakpoints in userspace.  this is lack of support in the hardware and has nothing to do with PIE support.
Comment 13 Hugo Mildenberger 2010-07-31 17:26:17 UTC
(In reply to comment #12)
> no x86/x86_64 system has ever supported hardware breakpoints in userspace. 
> this is lack of support in the hardware and has nothing to do with 
> PIE support.

Happily, this is mostly untrue. For one, there seems to be a kernel interface:

   strings linux-2.6.34-hardened-r1/vmlinux|grep register_user_hw_breakpoint
   register_user_hw_breakpoint

Secondly, after applying an ugly hack to gdb/breakpoint.c, I'm now able to use a hardware breakpoint in userspace:

  $ paxctl -v test
  [...]
 
  - PaX flags: -------x---r [test]
           RANDEXEC is disabled
           RANDMMAP is disabled

   $ gdb ./test
   GNU gdb (Gentoo 7.1 p1) 7.1
   [...]
   Reading symbols from /home/hm/test...done.
   (gdb) hbreak main
   Hardware assisted breakpoint 1 at 0x7b4: file test.c, line 1.
   (gdb) run
   Starting program: /home/hm/test 
   warning: no loadable sections found in added symbol-file
   /usr/lib64/debug/lib64/ld-2.11.2.so.debug

   Breakpoint 1, main () at test.c:1
   1       int main() {
   (gdb) 


Here is the patch (which is only meant for testing):


--- a/gdb/breakpoint.c  2010-02-12 02:24:09.000000000 +0100
+++ b/gdb/breakpoint.c  2010-07-31 18:11:52.000000000 +0200
@@ -6250,9 +6250,13 @@
   if (type == bp_hardware_breakpoint)
     {
       int i = hw_breakpoint_used_count ();
+#if 0
       int target_resources_ok =
        target_can_use_hardware_watchpoint (bp_hardware_breakpoint,
                                            i + 1, 0);
+#else
+      int target_resources_ok = 1;
+#endif
       if (target_resources_ok == 0)
        error (_("No hardware breakpoint support in the target."));
       else if (target_resources_ok < 0)
@@ -9434,9 +9438,13 @@
     {
       int i;
       i = hw_breakpoint_used_count ();
+#if 0
       target_resources_ok =
        target_can_use_hardware_watchpoint (bp_hardware_breakpoint,
                                            i + 1, 0);
+#else
+      target_resources_ok = 1;
+#endif
       if (target_resources_ok == 0)
        error (_("No hardware breakpoint support in the target."));
       else if (target_resources_ok < 0)


This is most probably a hidden bug in the target managment code, as _initialize_amd64_linux_nat() calls i386_use_watchpoints (t), which in turn sets  

     t->to_can_use_hw_breakpoint = i386_can_use_hw_breakpoint;

the latter being a static function always returning 1;

I checked if _initialize_amd64_linux_nat() was actually called, and it is.

If someone who knows gdb from inside out please could investigate this. 
"t->to_can_use_hw_breakpoints" actually points to a function named return_zero (which does exactly that). This is probably set in update_current_target() within gdb/target.c, and then never correctly updated. 
Comment 14 Hugo Mildenberger 2010-08-01 14:48:04 UTC
Using an unmodified copy of sys-kernel/gdb-7.1 together with the following test program indicates that there is only a slight logic glitch present within gdb-7.1's implementation of hardware assisted breakpoints on (hardened?) the x86_64 platform:

#include <signal.h>
#include <stdio.h>
void function_1() {
	printf("entering %s\n",__PRETTY_FUNCTION__);
}

int main() {
	raise(SIGINT);
	function_1();
	printf("leaving %s\n",__PRETTY_FUNCTION__);
	return 0;
}

$ gcc -g -o test test.c
$ paxctl -r ./test # else no symbolic backtrace and maybe other problems too
- PaX flags: -------x-e-r [test]
        RANDEXEC is disabled
        EMUTRAMP is disabled
        RANDMMAP is disabled

Note that mprotect is still enabled: this is essentially the same situation as if one is debugging a ROM based program. You simply can't insert breakpoints by modifiying the machine code and therefore need to have hardware debug registers available:

$ gdb ./test
GNU gdb (Gentoo 7.1 p1) 7.1
[...]
Reading symbols from /home/hm/test...done.
(gdb) hbreak function_1
No hardware breakpoint support in the target.
(gdb) maintenance print target-stack
The current target stack is:
  - exec (Local exec file)
  - None (None)
(gdb) run
Starting program: /home/hm/test 
warning: no loadable sections found in added symbol-file /usr/lib64/debug/lib64/ld-2.11.2.so.debug

Program received signal SIGINT, Interrupt.
0x000003fff7ab6005 in ?? ()
(gdb) maintenance print target-stack
The current target stack is:
  - child (Unix child process)
  - exec (Local exec file)
  - None (None)
(gdb) hbreak function_1
Hardware assisted breakpoint 1 at 0x3fff7ffe854: file test.c, line 3.
(gdb) cont
Continuing.

Breakpoint 1, function_1 () at test.c:3
3       void function_1() {
(gdb) cont
Continuing.
entering function_1
leaving main

Program exited normally.
(gdb) q

Slightly extending the implementation of the gdb command "maintenance print target-stack" in such a way that the command also indicates if the target supports hw breakpoints resulted in the following output:

$ ./gdb.modified test
[...]
Reading symbols from /home/hm/test...done.
(gdb) maintenance print target-stack
The current target stack is:
  - exec (Local exec file) can use hw breakpoints: no 
  - None (None) can use hw breakpoints: no 
(gdb) run
Starting program: /home/hm/test 
warning: no loadable sections found in added symbol-file /usr/lib64/debug/lib64/ld-2.11.2.so.debug

Program received signal SIGINT, Interrupt.
0x000003fff7ab6005 in ?? ()
(gdb) maintenance print target-stack
The current target stack is:
  - child (Unix child process) can use hw breakpoints: yes 
  - exec (Local exec file) can use hw breakpoints: no 
  - None (None) can use hw breakpoints: no 
(gdb) q
 
Comment 15 Xake 2010-09-06 12:32:24 UTC
What is the current bug here anyway? Currently I can make nearly everything work with gdb by "paxctl -mr <filename>" with gdb-7.1 from portage. It feels like the bug from the beginning was about gdb not working at all?
Comment 16 Anthony Basile gentoo-dev 2010-09-06 14:53:22 UTC
(In reply to comment #15)
> What is the current bug here anyway? Currently I can make nearly everything
> work with gdb by "paxctl -mr <filename>" with gdb-7.1 from portage. It feels
> like the bug from the beginning was about gdb not working at all?
> 

This bug has changed focus.  The original problem is solved exactly as xake says.

I'm going to close this for now, even though gdb-7.1 is still ~arch in the tree.  If the reporter wants hw breakpoints and backtrace, feel free to open another bug.
Comment 17 Hugo Mildenberger 2010-10-19 10:31:37 UTC
(In reply to comment #16)

> I'm going to close this for now, even though gdb-7.1 is still ~arch in the
> tree.  If the reporter wants hw breakpoints and backtrace, feel free to open
> another bug.
> 

I can only wonder why you closed this bug as "fixed" while it isn't. Does an internal statistic looks better now? Even gdb-7.2-p1 shows the same behaviour, at least on hardened amd64, but also on hardened x86 if memory serves me. You can workaround the problem using paxctl, but the core problem -- which is no hardware breakpoints and also wrongly calculated image base addresses -- is in no way "fixed". This leads to a non-working drkonqi, for example.