Now that we support building binary packages with debug info and stripping on end users, I'm thinking of reaching the next level: splitting 'main data' and debuginfo into separate packages. This way people who want stripping done wouldn't have to download lots of extra data. I'm thinking of a few possible approaches. One approach is to force splitdebug. Then we could just move splitdebug directories into separate archives and have Portage conditionally download them. On the minus side, this will force people to actually use splitdebug and it won't necessarily work for all targets (the kernel?). Another approach is to create one package before stripping, and another after stripping. This way Portage can download the one it needs, at the cost of having some duplication on binpkg storage. Yet another approach is to create deltas before unstripped and stripped executables, and store them in a separate file. Then Portage would download the normal 'stripped' archive, and optionally deltas to unstrip files.
An approach for this is to leverage elfutils (and gdb, etc) support for debuginfod: https://sourceware.org/elfutils/Debuginfod.html, https://developers.redhat.com/blog/2019/10/14/introducing-debuginfod-the-elfutils-debuginfo-server. Idea would be something like: - Use PKG_INSTALL_MASK to strip debug info, installsources from produced binpkgs - Run debuginfod on binpkg host - Make sure elfutils on the binpkg consumers is compiled with the right debuginfo server (can you specify via config file)?
my binpkg builder runs debuginfod now and I've added support for it to the gnu toolchain in the repos. with debuginfod -F we can already serve debug info, but the problem with this is that only one build is available (so, changed use, outdated binaries in a slot, etc are lost), there can be file collisions due to multiple binaries having the same build id, and the (well-compressible) sources and debug info remain installed uncompressed on the disk. i think the best approach here is leveraging debuginfod. adding debug download support to portage would at this point be redundant and less seamless than a debuginfod based approach. to 'fully' implement debuginfod support in portage, we don't need to do much. the work boils down to: tar -cJf "${P}-${BUILD_ID}-debug.tar.xz" --xform="s@${D#/}@@" "${D}"/usr/{src,lib}/debug # untested, but probably right? ... and installing the produced file to some orphaned location. the user can specify the {,PKG_}INSTALL_MASK to be /usr/{src,lib}/debug/* to then exclude those files. i'll try hacking this together with a bashrc today.
okay, this works: # Debug info package generation generate_debuginfo() { [[ ${MERGE_TYPE} == source ]] \ || return local debugpath=/var/cache/debug addwrite /var/cache/debug install -d /var/cache/debug/"${CATEGORY}"{,/"${PN}"} \ || die "Failed to generate target debug directory" ( set -x cd /var/cache/debug/"${CATEGORY}"/"${PN}" || die tar -cJf "${P}-${BUILD_ID}-debug.tar.xz" \ --xform="s@${D#/}@@" \ "${D}"/usr/{src,lib}/debug \ || die "Failed to pack up debug info" ) adddeny /var/cache/debug/ } post_pkg_postinst() { generate_debuginfo } for a simple testcase (app-misc/hello) with debuginfo -Z .tar.xz /var/cache/debug
okay, that's not perfect (seems that, on occasion but not always, splitdebug files don't exist yet, and .build-id symlinks are missing), but it seems like a good PoC. I reckon we should implement this as FEATURES=packdebug in portage? the proposed idea being that packdebug simply generates the tarball of /usr/{src,lib}/debug and requires that installsources, splitdebug are set and lets the user decide whether they want to *INSTALL_MASK these these paths. it would then place this tarball in some location (/var/lib/debug/${CATEGORY}/${PN}/${P}-${BUILD_ID}.tar*? open to suggestions) in portage, we can probably avoid whatever caused this inconsistent ${D} problem that I'm seeing (note that the correct files *are* merged: bstg /var/tmp/portage/www-client/elinks-0.16.1.1-r2/image # file {"${D}",}/usr/lib/debug/usr/bin/elinks.debug /var/tmp/portage/www-client/elinks-0.16.1.1-r2/image/usr/lib/debug/usr/bin/elinks.debug: cannot open `/var/tmp/portage/www-client/elinks-0.16.1.1-r2/image/usr/lib/debug/usr/bin/elinks.debug' (No such file or directory) /usr/lib/debug/usr/bin/elinks.debug: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter *empty*, BuildID[sha1]=6e6ea13d35f9e4e1eb6bfdc11fca05d5a299d64b, for GNU/Linux 3.2.0, with debug_info, not stripped bstg /var/tmp/portage/www-client/elinks-0.16.1.1-r2/image # ... but they aren't always in ${D}). if anyone has an idea why this happens, I'd love to hear it CC'd portage team for opinions
That sounds good to me. No need to overcomplicate it on the Portage side, as debuginfod is a thing now.
