When running "equery check" on a package containing files which have been changed due to prelinking, checksum mismatch errors are reported. This renders "equery check" rather useless on a FEATURE=prelink Gentoo installation. Reproducible: Always Steps to Reproduce: 1.Install a package which installs a shared library 2.Let Gentoo's prelink support scripts prelink the system 3.run "equery check" on the previously installed package Actual Results: "equery check" will report checksum failures for the shared libraries Expected Results: No checksum error should be reported "equery check" should know about prelinking, because "prelink" is a valid Gentoo system FEATURE which can be set in "/etc/make.conf". All Gentoo-provided scripts, such as /etc/cron.daily/prelink, should honor the global settings in FEATURES if they are affected by those. What should be done? The prelink scripts should either directly update the database which holds the md5 checksums of installed packages. Or, better, it should maintain a separate overlay database with updated checksums of the object files after they have been modified by prelinking. "equery check" should know about that overlay database and consult it as well as the primary checksum files maintained for each ebuild (if the "prelink" FEATURE is enabled in "/etc/make.conf"). If "equery check" detects a checksum failure for a certain file according to the the primary checksum files which are kept in "/var/db/pkg/<cat>/<pkg>/CONTENTS", it should not immediately report the checksum mismatch. Instead, it should consult the overlay checksum file maintained by the prelink scripts, and see whether prelink added a checksum-override for that file to the overlay. If there is a matching overlay entry and its checksum matches that of the file in the live filesystem, no error should be reported. In its current form, "equery check" is mostly useless on a system using the "prelink" FEATURE, because most .so files will be reported to have mismatching checksums, and one never can tell whether this is due to the effect of prelinking, or due to an actual corruption of the files.
I guess equery needs to call perform_checksum(calc_prelink=1), since calc_prelink=0 is the default. Then it should work flawlessly (it will create a temp file with the prelink undone, and use the checksum of the temp file).
Thanks for reporting. Could you let us know what version of gentoolkit and portage you're having this problem with? From what I can see, every version of equery in the tree uses calc_prelink=1: http://sources.gentoo.org/viewcvs.py/gentoolkit/trunk/gentoolkit/pym/gentoolkit/equery/check.py?rev=659&view=markup
(In reply to comment #2) > Thanks for reporting. Could you let us know what version of gentoolkit and > portage you're having this problem with? Sure, it's app-portage/gentoolkit-0.2.4.2-r1 and sys-apps/portage-2.1.6.11. However, this issue is by no means a new one. I have been encountering it for a long time, but have always been too lazy to write a bug report so far. Generally, my system should be up to date as of last Saturday with respect to $ emerge -avuDN --with-bdeps=y world $ emerge -a --depclean $ revdep-rebuild -- -a $ eclean -d package $ eclean -d distfiles In other words: It should be quite recent; nothing should be out of date by more than about a week. (I'm using a stable system, however; so I might be missing the latest nightly builds of some masked packages, of course.) > From what I can see, every version of equery in the tree uses calc_prelink=1: > > http://sources.gentoo.org/viewcvs.py/gentoolkit/trunk/gentoolkit/pym/gentoolkit/equery/check.py?rev=659&view=markup Hmm, I can't even find that file in the output of "equery files" of gentoolkit! $ equery files gentoolkit | grep check /usr/bin/glsa-check /usr/share/man/man1/glsa-check.1.bz2 That's it.
Interestingly, I just discovered that not *all* packages containing prelinked files actually give checksum errors! I had to run a couple of attempts with the following command line until I encountered the following output: # equery check $(equery belongs -e $(strings /etc/prelink.cache | shuf -n 1) | cut -d" " -f1) [ Checking app-text/psutils-1.17 ] !!! /usr/bin/psbook has incorrect md5sum !!! /usr/bin/epsffit has incorrect md5sum !!! /usr/bin/psnup has incorrect md5sum !!! /usr/bin/psselect has incorrect md5sum !!! /usr/bin/pstops has incorrect md5sum !!! /usr/bin/psresize has incorrect md5sum * 46 out of 52 files good Next I ran: # prelink -u /usr/bin/psbook # equery check app-text/psutils [ Checking app-text/psutils-1.17 ] !!! /usr/bin/epsffit has incorrect md5sum !!! /usr/bin/psnup has incorrect md5sum !!! /usr/bin/psselect has incorrect md5sum !!! /usr/bin/pstops has incorrect md5sum !!! /usr/bin/psresize has incorrect md5sum * 47 out of 52 files good You see - one mismatch less. Hmmm... interestingly enough, I am also notice: The files which messed up "equery check" were not libraries but rather executables! Perhaps that's the explanation: Portage might not account for the possibility that executable files might also be prelinked! But in fact, any ELF-file can be prelinked - not just .so-files. Being a huge "prelink" fan, I am not only using prelink for shared libraries, but for normal executables also! Which is perfectly legal to do, and supported by "prelink", but obviously not a widespread practice. Which might explain why nobody else seems to have encountered the same issue so far!
In case someone might be wondering why I am prelinking normal executables also: Admittedly, that does not make much sense for plain normal executables. But it makes perfect sense for "fast-loader"-types of application such as "kdeinit". OK, perhaps not quite "kdeinit", as there is special prelinking-support from Portage for KDE. But there are other fast-loader applications, for instance the OOo fast loader. And I'm pretty sure other application frameworks use fastloaders also internally. What does a fastloader do? Essentially, it's a normal executable that opens all or a carefully selected subset of shared object libraries in order to keep them being loaded in memory. But more than that, if any relocations should be necessary for loading those shared libraries, they only need to be done at the time the fast loader starts up. How a fastloader works exactly depends on the implementation. But in order to make sure a newly launched application-framework component-application inherits the already-loaded (and possibly relocated) shared libraries held open by the fastloader, the fastloader cannot just fork and exec(): In that case, the dynamic loader would start with a completely new virtual address space, and there is no guarantee that any required shared objects would happen to be loaded exactly to the same base addresses. (All this of course applies only to a scenario where *not* all shared libraries are already prelinked.) But even in a scenario where all shared libraries are prelinked, one single problem remains: If the fastloader is located at the same address as the executable launched by the fastloader (which all executables are by default short of being prelinked), at least the component executable itself would have to be relocated then! If, on the other hand, the base addresses of the executable and of the fast loader are different (and the memory ranges do not overlap), neither the fastloader nor the component executable need to be relocated. The fastloader can just fork, load the component application using dlopen(), and jump to its entry point. (The same applies to in-process overlay code modules.) The dynamic loader will not have to load again (and possibly relocate) any of the .so files which have already be loaded by the fastloader; it can just re-use them. I don't know whether any fastloaders actually work that way; but that's the idea: Avoid relocation of component executables / overlay executables by assigning any executable in the system a different starting address. This is why I am prelinking executables as well as libraries. And just to recap: Why is relocation bad at all? It's not just about startup time. It's about memory efficiency! When the dynamic loader loads an executable or shared libraries, it first mmaps() the file into the virtual address space. If an ELF file can be mmapped to the same memory range it has been hardcoded ("linked") for, that is all which needs to be done: No relocations are required. But even better, such mmaps() can be shared among different processes! (That is: Read shared, but Copy-On-Write if anything is changed.) Which means, no matter into how many processes' virtual address spaces such a shared library is loaded ("projected"), it will be shared automatically by the kernel (assuming the MAP_SHARED flag has been used). A single instance of the code will thus be shared among all processes. But this is no longer true when an ELF file has to be relocated by the dynamic loader: Relocation means the mmapped() executable has to be modified. This will trigger Copy-On-Write for any modified page of the ELF file, and those Copy-On-Write created pages will not he shared with any other process (except processes directly forked from the current one, if the current process behaves like a fast loader as described above). Summing up: Although prelinking executables does not make much sense for typical executables, it does make sense if some of those executables are actually fastloaders or overlay code to be loaded as a module into the same address space. And even in the cases where it makes no sense, it does not hurt either. Provided the virtual address space is large enough (which is the case for 64 platforms), it makes sense to prelink all shared libraries as well as all executables, because then there is no need to differentiate between fastloaders, overlay code and shared libraries: If every ELF file in the system has a different starting address, relocation will simply never be necessary.
Maybe a bug in prelink is triggering it. The portage perform_checksum(calc_prelink=1) function relies on `prelink --verify <file>` reversing the prelink and restoring the file to its original state. If the result of that command differs from the orginal file in any way, then the checksum will not match.
Prelink support is being dropped upstream in glibc-2.35; sys-devel/prelink has been masked for removal.
