sys-devel/dwz can be used to optimize debugging information before compression which can improve disk space usage. We should see about integrating it into Portage. Especially when combined with zstd compressdebug support (bug 906367), this might lead to considerable disk space savings.
Is dwz multifile mode desired? I'm asking because I'm not sure what would be the best place & name to store the common object for the -m argument to dwz. If not, I did a quick patch, naming the feature FEATURE=dedupdebug, attached here (I haven't edited any of the man pages yet). This is a quick test result with =app-shells/bash-5.2_p26, FEATURES="splitdebug compressdebug", and the size of /usr/lib/debug/usr/bin/bash.debug: * -ggdb FEATURES=-dedupdebug: 1237808 * -ggdb FEATURES=dedupdebug: 1141408 (7.8% decrease) * -ggdb3 FEATURES=-dedupdebug: 1444872 * -ggdb3 FEATURES=dedupdebug: 1347472 (6.7% decrease) I made it run before the splitdebug phase in estrip because dwz seems unable to optimize a compressdebug-ed binary.
Created attachment 883016 [details, diff] dedupdebug POC
Did my periodic @world upgrade (last one was on the 20th), /usr/lib/debug/ went down from 4.6G to 4.2G, even though I changed global CFLAGS -ggdb to -ggdb3 (though I also have a lot of packages, such as nodejs, not have any -g* at all), so there's probably some significant space savings. I'll send a PR when I'm free.
this patch seems reasonable to me at a glance. re, multi-file mode: it sounds like it could be useful (esp. for c++ programs). what would that require and what effect would it have?
> what would that require and what effect would it have? I believe what I need mainly is a filename for the "combined debug info". i.e. > OPTIONS > -m FILE --multifile FILE > Multifile mode. After processing all named executables and > shared libraries, attempt to create ELF object FILE and put de‐ > bugging information duplicated in more than one object there, > afterwards optimize each named executable or shared library even > further if possible. What the file name for FILE should be. My initial thought is probably /usr/lib/debug/dwz/<atom with version>.debug But another issue comes with how it'll interact with splitdebug & compressdebug. The new files in /usr/lib/debug/dwz/ should not be split again, but it should be processed by compressdebug (I don't think dwz performs compression itself but I might be wrong). I think it'll take a bit of trial and error for me to do that correctly. As for the effect, I would believe that the final sum of the sizes of all the debug files will be even smaller.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=749d231d57fb64d3f2e909dfbba26ca7c1ab9d0e commit 749d231d57fb64d3f2e909dfbba26ca7c1ab9d0e Author: YiFei Zhu <zhuyifei1999@gmail.com> AuthorDate: 2024-01-29 08:50:28 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-02-21 02:13:04 +0000 estrip: Support debug info deduplication with sys-devel/dwz This this gates behind FEATURES=dedupdebug, and packages can opt-out with RESTRICT=dedupdebug, though I'm not sure why a package would need to opt this out, unless dwz could break something I'm not aware of... Bug: https://bugs.gentoo.org/906368 Signed-off-by: YiFei Zhu <zhuyifei1999@gmail.com> Closes: https://github.com/gentoo/portage/pull/1245 Signed-off-by: Sam James <sam@gentoo.org> bin/estrip | 35 +++++++++++++++++++++++++++++++++-- lib/_emerge/EbuildPhase.py | 1 + lib/portage/const.py | 1 + man/ebuild.5 | 3 +++ man/make.conf.5 | 8 +++++++- 5 files changed, 45 insertions(+), 3 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=77c44c46194922509bc4f2b5cfc099412a560a69 commit 77c44c46194922509bc4f2b5cfc099412a560a69 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-02-22 07:23:40 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-02-22 07:23:50 +0000 sys-apps/portage: add 3.0.62 Closes: https://bugs.gentoo.org/663324 Closes: https://bugs.gentoo.org/728046 Closes: https://bugs.gentoo.org/891137 Closes: https://bugs.gentoo.org/906368 Closes: https://bugs.gentoo.org/916566 Closes: https://bugs.gentoo.org/921170 Closes: https://bugs.gentoo.org/921208 Closes: https://bugs.gentoo.org/921400 Closes: https://bugs.gentoo.org/922038 Closes: https://bugs.gentoo.org/922142 Closes: https://bugs.gentoo.org/923368 Closes: https://bugs.gentoo.org/923750 Closes: https://bugs.gentoo.org/923841 Closes: https://bugs.gentoo.org/923852 Closes: https://bugs.gentoo.org/923854 Closes: https://bugs.gentoo.org/924192 Closes: https://bugs.gentoo.org/924273 Closes: https://bugs.gentoo.org/924585 Closes: https://bugs.gentoo.org/921380 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.62.ebuild | 246 +++++++++++++++++++++++++++++++++ 2 files changed, 247 insertions(+)
