Here is a patch to allow portage to build packages with the debug info stored in a separate file. Which gives the memory advantage of stripped executables and the debug information. I've added two FEATURES, keepdebug and keepdebugbin. keepdebug only keeps the information for libraries, the info is kept for executables only if keepdebugbin is set too. setting keepdebugbin but not keepdebug might break stuff. Its particularly useful if -g is in the CFLAGS, as that will produce huge debug files. For the implementation details, I've added keepdebug in most places were nostrip is, I hope I didn't forget anything. This requires a fairly recent binutils to work and gdb 6.0 to be useful. The debug information for a file x is kept in .debug/x.dbg is the same directory as the x file (the .debug directory name is where gdb will look by default).
Created attachment 27633 [details, diff] the patch.. version 1
That's a huge patch... I'm guessing your testing it a lot?
Well, I have to change many places, there might be a nicer way to do this (I noticed that many scripts (such as dobin/dosbin) are almost identical so it may be possible to have common code called by both. I havent tested it too much yet, but I build all of my libs with keepdebug now. I was hoping that others might be interested too and might want to help testing.
Created attachment 27738 [details, diff] fixed patch Oops, first patch did not include the keepdebugbin stuff and... After a little more research, I discovered that we can use normal strip instead of strip -g if binutils > ..8 which is ~arch on everything but s390. That would make the patch much smaller (dont have to switch between strip and strip -g and dont have to filter out install -s). And would give real stripped executables at the end.. I'm also not sure how keepdebug should interact with nostrip when both are given. The current version gives the user a separate debug info and a non-stripped file, so the debug information is duplicated. But I suppose that's giving the user more choice.
Created attachment 27739 [details, diff] newer version here is a version that will work with binutils >= ...8 It might work with ..7 but I need to do more testing.. btw, this patch is 80 lines shorter than the first iteration..
Created attachment 32328 [details, diff] new version.. against cvs New better version.. Uses eu-strip from elfutils.. I've introduced a another new FEATURES called "keepsources", it installs the sources for the preserved debug info into /usr/src/debug/${P}. keepsources only does something if nostrip, keepdebug or keepdebugbin are there.. Also, preplib.so should be removed, its used nowhere.. This patch is against the bin/ directory of the CVS.. I also have a version against 2.0.50, but I guess that has little interest. Also, it removes the stripping from any of the do* scripts, it doesnt belong there. That's necessary anyways if we want to do it at the end to keep the debug information.
Created attachment 32329 [details, diff] patch for the ebuild.. agsint 2.0.51_pre9
Created attachment 32330 [details] debugedit also, this belongs in src/ and is installed if the debug USE flag is used. Its necessary for keepsources to work.. If the packages is -debug, its not installed and keepsources just wont work, it should break nothing else..
Created attachment 35701 [details, diff] update of version against cvs I've updated the patch against cvs.. it would be nice if it went in before the current portage stabilises...
Ok. Looking at that patch in detail sometime soon.
i am a newwbe: 1)is the patche already merged in oficial portage ? or againt what should i apply it ? 2)what should i add to FEATURE in make.conf to have separate symbols & src 3) do u have a kind of ~/.gdbrc to autoconf gdb 5) the patche just look quite safe to include it in masked portage 6) this trhread is cool :)
7) and how do u compile? gcc debugedit.c -o debugedit /tmp/ccVvZyHD.o(.text+0x237): In function `read_abbrev': : undefined reference to `htab_try_create' /tmp/ccVvZyHD.o(.text+0x277): In function `read_abbrev': : undefined reference to `htab_delete' [other stuff ...] what are gcc options for compilation ?
To answers comments #11 and #12 1) its not yet.. and I havent updated it in a while.. so I'm not even sure it still applies 2) keepdebug (to keep it for libs) and keepdebugbin (for bins too) 3) no 4) Where is 4? 5) I totally agree! 7) emerge elfutils popt ; gcc -o debugedit debugedit.c hashtab.c -lelf -lpopt
> and I havent updated it in a while.. so I'm not even sure it still applies I tried the patch last week and it doesn't seem to apply anymore.
Created attachment 53254 [details, diff] Updated patch Here's an updated patch that applies cleanly to portage-2.0.51-r15 It seems to work fine, but I haven't tested it very much.
Putting a hold on feature requests for portage as they are drowning out the bugs. Most of these features should be available in the next major version of portage. But for the time being, they are just drowning out the major bugs and delaying the next version's progress. Any bugs that contain patches and any bugs for etc-update or dispatch-conf can be reopened. Sorry, I'm just not good enough with bugzilla. ;)
There is a working patch here (I maintain it and use it on my system)... Only an ebuild for debugedit is missing (or it could be included with portage, but it depends on libelf.so(elfutils) and libpopt).
Created attachment 69216 [details, diff] Updated patch Here's an updated patch that applies cleanly to portage-2.0.52-r1
*** Bug 112907 has been marked as a duplicate of this bug. ***
Please see bug 112907 for a cleaner method of doing this.
That's not enough.. You at least need to see why it's cleaner, what the pros are of using objcopy versus debugedit, etc. As far as I can see, you are both doing the same thing in a different way. Olivier's patch is a lot messier, but that's only because several FEATURES are added by it.
Few things that come to mind. 1. elfutils eu-strip is not very portable. (probably wont work on bsd/osx) for sure not on uclibc. 2. the method of stripping proposed on this bug is not cross-compiler friendly. 3. The FEATURE= is unneeded. this should be enabled per default for everything but the released media (and that we should save elsewhere). 4. using .gnu_debuglink via objcopy is the way the toolchain intends for this to be done. 5. /usr/lib/debug is one of the default search dirs for symbols. 6. 112907 is the same way other distros do it. (deb uses .dbg vs .debug as an extention. redhat and co use .debug) 7. debugedit is a new dep. It's not in the tree yet. It would depend on popt also. 8. libopts(),instinto(),execinto() from attachment #35701 [details, diff] where it prevents stripping at that stage is the right thing todo.
Sorry I ment attachment #69216 [details, diff]
Okay, part of binutils; that works. Do you know what version it started in? Also, there's a corner case that needs to be dealt with. xorg-x11 does it's own stripping. Perhaps a {dyn,src}_strip() is needed at the changeover?
The important thing that my patch does and your's doesnt it keeping the sources in /usr/src/debug which very useful for debugging into libraries. and that' where a large part of the mess is (that's what debugedit is used for). I agree with you that objcopy is much better.. I used eu-strip simply because when I first wrote it, the objcopy from stable binutils didnt have the separate debug stuff. Your version is not multilib friendly (that's unacceptable for amd64 at least). Not being able to disable debug info is not very small hard disk friendly..
(In reply to comment #24) > Okay, part of binutils; that works. Do you know what version it started in? > Also, there's a corner case that needs to be dealt with. xorg-x11 does it's > own stripping. Perhaps a {dyn,src}_strip() is needed at the changeover? fwiw, this is only necessary on non-dlloader (e.g., <=6.8.2) so will become a non-issue with the stabilization of 7.0 and removal of 6.8.2. It should be released around 7 December, according to http://xorg.freedesktop.org/wiki/X11R6970ReleasePlan.
(In reply to comment #24) > Okay, part of binutils; that works. Do you know what version it started in? Jason, Looks like July 2003 http://sourceware.org/ml/binutils/2003-06/msg00448.html Looking at an old 2.14.90.0.4 I see it there. (In reply to comment #25) > Your version is not multilib friendly (that's unacceptable for amd64 at least). How so? This as far as I know should play along with multilib perfectly well The executable you attempt to debug path is matched. > Not being able to disable debug info is not very small hard disk friendly.. INSTALL_MASK="/usr/lib/debug" or do you really want to tied it to a feature?
Oops, on the multilib thing, sorry, I didnt read properly, it seems fine (I hate bash string processing). The INSTALL_MASK trick doesnt allow keeping the debug info just for libraries and not executables... and it uses much more temporary space when building. But I guess if its a space problem, then INSTALL_MASK will do.. The only other thing then that your version (which I agree is much cleaner) doesnt do is the keepsources, you could add something like this to your save_elf_debuf() function. And I'll add debugedit to the tree as its own ebuild, I agree its only useful for developers, not "normal users" if [ -x /usr/bin/debugedit ] ; then debugedit -b ${WORKDIR} -d /usr/src/debug/${P} -l ${T}/debug.sources "$x" if [ -s ${T}/debug.sources ]; then [ -d ${D}/usr/src/debug/${P} ] || mkdir -p ${D}/usr/src/debug/${P} cat ${T}/debug.sources | (cd ${WORKDIR}; LANG=C sort -z -u | xargs -0 -- cp --parents -p --target-directory="${D}usr/src/debug/${P}" ) fi fi
Why you would want to save the sources. Pehrpas try -g3 -ggdb when building. I get. onyx gdb # gdb -q `which q` Using host libthread_db library "/lib/libthread_db.so.1". (gdb) b main Breakpoint 1 at 0x8057cec: file main.c, line 858. (gdb) quit The logic for keepdebugbin is 90% correct. It assumes that shared objects are not executables. About 10% of the userbase are hardened users. So users with that toolchain enabled create ET_DYN executables by default. Perhaps the simplest solution is to simply /usr/lib/debug/{s,usr/s,usr/,}bin Or if we want to shoot for that 100% mark.. (like catch mozilla and co) We could simply look for ET_DYN or ET_EXEC files containing the symbol main() The scanelf tool makes it quite easy. scanelf -F'%o %F#s' -s main -qyR ${D} | grep ^ET_'\(EXEC\|DYN\)' Anyway I'm quite pleased with the attached prepstrip on bug #112907 and would rather see that merged in it's basic form before we started adding additional functionality onto it. Would that work for everyone for the next release?
In reply to comment #29) > Why you would want to save the sources. It makes it much easier debug into libraries (which sometimes have bugs). That's the only reason. What do you not like about the implementation in comment #28 ? I dont care too much about keepdebugbin.. I guess that part can wait for a further release. The INSTALL_MASK way is probably good enough for now. If anyone cares enough, I guess they'll write something better for a later release.
(In reply to comment #30) > What do you not like about the implementation in comment #28 ? It's not that I dont like it. I've just never used debugedit. The checks that check that debugedit is executable which makes it optional. So thats ok for the arches/libcs that wont be able to compile it.
There is actually another point where debugedit comes to use. It would enable setting the source location to a logical location where one could easilly have the sources without them being in a temp directory. It would be easy to have ebuild install them on demand.
We we please add this debugedit ebuild to the tree so the rest of us can see how we can benefit from it's use. It's hard to justify features which depend on code from ebuilds not in the tree yet. My lastsync was on Sun Nov 20 02:36:43 UTC 2005 so if debugedit is already in the tree but not my $PORTDIR yet sorry for the noise.
debugedit is attached here.. I'll make an ebuild.
the reason I had not done a separate ebuild was that I was thinking of including it into portage and installing it into /usr/lib/portage/bin .. (redhat puts it in /usr/lib/rpm/debugedit).
Created attachment 73275 [details] debugedit from rpm-4.4.3 I'm attaching an updated version of debugedit taken from rpm-4.4.3 .. I'm a bit inconfortable with the idea of making it a separate package.. We could just use it from /usr/lib/rpm ... although I also dont like the idea of forcing ppl to install rpm to get the feature...
The attached debugedit will need to depend="!elibc_glibc? ( gnulib ) popt ( elfutils || libelf )" portage itself depending on rpm is not really an option. The masses would scream.
I'm not sure both elfutils and libelf work.. (we'd have to test).. I was saying that the other possibility was to only enable it when rpm is present (like we do ccache or distcc).. but I guess its not really nice. Or the other option is to make debugedit into its own package and only enable the sources thing when its present. Its not really nice either to make portage depend on popt and elfutils and pax-utils..
Ok so what do you want todo here? I started a thread on the portage-dev mailing list about making pax-utils a depend. I don't think there are any objections to it as it can be used to improve a number of the bash/elf tasks we already preformed. Are we in agreement that the prepstrip from the other bug should be merged for starters? I think yes with one concern if should it be enabled per default or not.
When adding this to portage, we also need to think to add "debug" to MULTILIB_STRICT_EXEMPT in the amd64 profiles, otherwise it breaks multilib-strict
(In reply to comment #40) > When adding this to portage, we also need to think to add "debug" to > MULTILIB_STRICT_EXEMPT in the amd64 profiles, otherwise it breaks multilib-strict debug added to the defaults. Are you going to put a debugedit ebuild in the tree?
Created attachment 76121 [details] ebuild for debugedit here is the ebuild for debugedit.. it will get it from my dev space. its the same that's attached here.
Created attachment 76124 [details, diff] patch to add keepsources since solar's patch has been committed for bug #112907 in SVN 2374. I'm obsoleting my old patch and just adding the patch to add support for debugedit.. which I'm adding to the tree under dev-util/debugedit
*** This bug has been marked as a duplicate of 112907 ***
