Based upon the following email exchange, I'm opening this as a meta-bug to track progress with updating revdep-rebuild Re: [gentoo-dev] Two orphans looking for a new daddy On Sat, 2004-08-21 at 18:26, Marius Mauch wrote: > Their names are ufed and revdep-rebuild .... > Applications for adoption should be sent to tools-portage or myself. On Wed, 2004-09-01 at 12:03, Paul Varner wrote: > Have you gotten any bites for revdep-rebuild? ... > So if you're interested in me helping let me know. If so, I'll start > looking at the bugs in bugzilla and start working on patches. On Thu, 2004-09-02 at 05:59, Marius Mauch wrote: > No one has volunteered for revdep-rebuild yet, so you're quite welcome. If > you need further information or help please contact karltk@gentoo.org as > I'm out of town for a while and won't be online much. Right now, my plan is to do the following: 1. Cleanup revdep-rebuild v1.5 from CVS to fix the various grammatical, spelling , and indentation errors. - I see this one as being able to be included in CVS right away as there will be no bug fixes per se and to be used as the baseline for the following bug fixes and enhancements. 2. Add support for the user to customize how revdep-rebuild works. This will take care of several of the current bug reports. 3. Walk through the remaining bug reports and classify as follows: fix, later, can't fix fix - A bug fix would be produced for the current bash based revdep-rebuild later - A fix will be produced when feasible based upon the rewrite of revdep-rebuild in Python and the use of the supporting API's based upon the Gentoolkit reorganization plan. can't fix - A fix isn't technically feasible. 4. Produce the fixes for the bugs classified as fix 5. Rewrite revdep-rebuild to take advantage of the portage API. This rewrite would be in Python instead of bash. The rewrite would include those fixes classified as later that can be supported through the use of the portage API. I see steps 1 - 4 as something that I can accomplish relatively quickly, since I'm an experienced shell scripter. Step 5 would take longer since it requires either someone else to rewrite or for me to get up to speed on python and the portage API. I plan on getting up to speed on both regardless of who does the rewrite, but it is a limiting factor.
Created attachment 38792 [details] revdep-rebuild Cleaned up script. Fixed spelling and grammatical errors. Fixed indentation errors. This is the script that I will base future bug fixes against.
Created attachment 38869 [details] revdep-rebuild First enhancement release Note: I'm uploading the full script since that patches were practically the same size as the script This release contains the following enhancements/fixes * Customizable searching controlled through environment variables. This removes the need for end users to directly modify the script LD_LIBRARY_MASK - Mask of specially evaluated libraries SEARCH_DIRS - List of directories to search for executables and libraries SEARCH_DIRS_MASK - List of directories to not search These variables can be prepended to either by setting the variable in your environment prior to execution, or by placing an entry in /etc/make.conf. An entry of "-*" means to clear the variable from that point forward. Example: env SEARCH_DIRS="/usr/bin -*" revdep-rebuild will set SEARCH_DIRS to contain only /usr/bin Bug(s): 32276, 38011, 59803 * The directories to search are no longer hard coded into the script. Revdep-rebuild now determines the directories to search based upon /etc/profile.env and /etc/ld.so.conf Bug(s): 32276, 38011 * --ignore option to ignore temporary files left from previous runs. Bug(s): 34052 * Automatically ignore temporary files older than 24 hours. Bug(s): 34052 * Always return an exit status based upon success or failure Bug(s): 38472 * --nocolor option to turn off colored output, the script also obeys the NOCOLOR setting from /etc/make.conf. * Removed dependency on qpkg. Script now uses equery. * Performance enhancement (Needs testing on architectures other than x86) After grabbing the list of files that have the executable bit set, I use the file command to find those files that are listed as "dynamically linked" This slows down the first stage of locating the files to check, but the slow down is more than made up for during the stage where the dynamic linking is checked for broken libraries. Other arches need to verify that the file command returns "dynamically linked" for dynamically linked executables. Testing on my 1.8GHz Pentium 4 showed the following results: revdep-rebuild --ignore --pretend Before: 115 seconds After: 86 seconds env SEARCH_DIRS="/usr/bin -*" revdep-rebuild --ignore --pretend Before: 22 seconds After: 25 seconds env SEARCH_DIRS="/ -*" SEARCH_DIRS_MASK="/home /mnt" revdep-rebuild --ignore --pretend Before: 359 seconds After: 243 seconds Testing on my 233MHz Pentium showed the following results: revdep-rebuild --ignore --pretend Before: 165 seconds After: 126 seconds env SEARCH_DIRS="/usr/bin -*" revdep-rebuild --ignore --pretend Before: 47 seconds After: 46 seconds env SEARCH_DIRS="/ -*" SEARCH_DIRS_MASK="/home /mnt" revdep-rebuild --ignore --pretend Before: 630 seconds After: 505 seconds
I have a problem with this. Here is what I get: Checking dynamic linking consistency... broken /usr/games/lib/torcs/accc-bin (requires libtgf.so libtxml.so) broken /usr/games/lib/torcs/nfsperf-bin (requires libtgf.so libtxml.so) broken /usr/games/lib/torcs/texmapper-bin (requires libtgf.so libtgfclient.so libtxml.so) broken /usr/games/lib/torcs/torcs-bin (requires libracescreens.so librobottools.so libclient.so libconfscreens.so libtgf.so libtgfclient.solibtxml.so libraceengine.so) broken /usr/games/lib/torcs/trackgen-bin (requires libtgf.so libtgfclient.so librobottools.so libtxml.so) done. (/root/.revdep-rebuild.3_rebuild) All of thehe library files it is looking for are found in /usr/games/lib/torcs/lib. I tried adding this to the PRELIMINARY_SEARCH_DIRS="$SEARCH_DIRS /bin /sbin /usr/bin /usr/sbin /lib /usr/lib" section of revdep-rebuild. It still didn't find them and wanted to re-emerge torcs.
Unfortunately, it works for me. I emerged torcs and then ran revdep-rebuild and it found everything just fine. I'm double checking how the LD_LIBRARY_PATH is built to try to narrow down the cause.
The first time I ran it there was an error on the emerge part of re-emerging torcs. I did emerge torcs after checking bugzilla for the reason it errored out. I then did emerge torcs. It emerged just fine but still was getting the revdep-rebuild errors that I reported to you. I wonder if I need to unmerge it and the re-emerge it to solve this problem.
I've looked through the code and I can't find a reason for it to report a file as broken unless it is broken. Please do the following: revdep-rebuild --ignore --pretend If it still wants to reemerge torcs, then do the following: cd /usr/games/lib/torcs LD_LIBRARY_PATH=$(cat /root/.revdep-rebuild.2_ldpath) ldd torcs-bin Attach the output from ldd and the /root/.revdep-rebuild.2_ldpath file so that I can see them.
Created attachment 39660 [details] Output of (cat /root/.revdep-rebuild.2th) ldd torcs-bin Here is the output of LD_LIBRARY_PATH=$(cat /root/.revdep-rebuild.2_ldpath) ldd torcs-bin that you requested.
Can you attach /root/.revdep-rebuild..2_ldpath as well please.
Created attachment 39686 [details] Contents /root/.revdep-rebuild..2_ldpath Here is what is contained in /root/.revdep-rebuild..2_ldpath
Created attachment 39734 [details] revdep-rebuild Updated script with bug fixes. * Script was not finding all shared objects due to the performance fix. I've updated the script to look for either dynamically linked or shared object when determining the files to be examined. * Script would generate and error if NOCOLOR variable was set to something other than true or false. Updated to accept yes/true for true and anything else as false.
Your update fixed the problem I was having with torcs. Now there is another problem. I let it finish once already. It re-emerged dev-java/sun-jdk-1.4.2.05. Java is working just fine. When I ran revdep-rebuild --ignore --pretend again, it still get the same message and doesn't even give a clue about what it thinks is missing: Checking reverse dependencies... Packages containing binaries and libraries broken by any package update will be recompiled. Collecting system binaries and libraries... done. (/root/.revdep-rebuild.1_files) Collecting complete LD_LIBRARY_PATH... done. (/root/.revdep-rebuild.2_ldpath) Checking dynamic linking consistency... broken /opt/sun-jdk-1.4.2.05/jre/lib/i386/libJdbcOdbc.so (requires ) done. (/root/.revdep-rebuild.3_rebuild) Assigning files to ebuilds... done. (/root/.revdep-rebuild.4_ebuilds) Evaluating package order... done. (/root/.revdep-rebuild.5_order) All prepared. Starting rebuild... emerge --oneshot --nodeps --pretend =dev-java/sun-jdk-1.4.2.05 These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild RF ] dev-java/sun-jdk-1.4.2.05 Now you can remove -p (or --pretend) from arguments and re-run revdep-rebuild.
Since java is a binary distribution, there isn't anything that revdep-rebuild can do with it anyways. I will add /opt/sun-jdk to the default list of directories to not search in the next revision of the script. In the meantime, you can add the following line to /etc/make.conf to prevent it from looking at sun-jdk. SEARCH_DIRS_MASK="/opt/sun-jdk"
Never mind. After todays updates to my system, revdep-rebuild worked without finding the problem with java. It worked great.
Created attachment 40020 [details] revdep-rebuild Here is the latest and greatest. Changes: * Removed performance enhancement based upon seeing if a file was a shared object. After fixing the bugs noted, the time to run revdep-rebuild increased to the point that performance was worse. * Script obeys and uses PORTAGE_NICENESS * Major changes to the functionality when using --package-names/-X The script should now be able to update slotted packages correctly. I tested this extensively using sys-libs/db. Due to these changes, it no longer uses equery, it uses etcat. emerge is no longer called with --nodeps in this mode. This allows for needed dependencies to be pulled in, if the package is being upgraded. * --keep-temp option (undocumented - i.e not listed as an option with --help). This is primarily useful for debugging/testing. If revdep-rebuild finds nothing to rebuild, it removes its temp files. This option will keep the files. * Changed COMPLETE_LD_LIBRARY_PATH to include the path of all found executibles.
Is working okay here as far as I can tell. One question though.. Why specify the --nodeps option to emerge? If use flags have changed since the package was last built, this could easily cause it to fail. If they have not, there wont be any extra deps brought in and the result will be the same.
As far as using --nodeps, I'm leaning to removing it. What is holding me back is that it was part of the original code and I want to make sure that I don't break something else by removing it. So far I've not noted any issues with removing it.
I think it might be good to have PRELIMINARY_SEARCH_DIRS="$SEARCH_DIRS /bin /sbin /usr/bin /usr/sbin /lib* /usr/lib*" This way it will work for systems with lib64 or lib32 as a libdir. You might also want to consider checking for SEARCH_DIRS in /etc/make.profile/make.defaults (and recursively up the cascading profile)
Created attachment 40741 [details] revdep-rebuild Minor changes removed --nodeps from the emerge command changed the defaults for SEARCH_DIRS to contain /lib* and /usr/lib* Created a man page
Created attachment 40742 [details] revdep-rebuild man page
I'm looking into how to handle also looking at make.defaults with the cascading profiles as I really like that idea.
I'll add the latest version into gentoolkit-0.2.0_pre10 today if I don't find any obvious errors. little note: please use equery instead of etcat (as etcat is also phased out)
mmh, make that pre11 (already uploaded the tarball to the mirrors)
I just opened bug# 67116. Until that issue gets changed, I can't use equery instead of etcat as I'm unable to get the information that I need for revdep-rebuild.
Created attachment 50226 [details] revdep-rebuild Changes: * Updated to ensure that the standard library paths are at the start of the LD_LIBRARY_PATH used to to search for broken dynamic linking. * Script creates and temporary file containing the environment configuration to be used for debugging. * Cleaned up some more grammatical errors in messages.
I have a small problem running it on remote machines, since it closes the connection when exiting...
That is strange, as I run it remotely all of the time and it doesn't close the connection. What shell are you running on the remote systems?
Uh, sorry. Loaded script and without changing permission started it through '. ./revdep-rebuild'... That's why. Just ignore this...
The last attached revdep-rebuild (id=50226) does not fix the vmware lib-dir problem (#80455), as the path-list remains unsorted after 'uniq'. Another thing is that uniq doesn't remove duplicates if unsorted, try "(echo a ; echo b ; echo a ) | uniq" to see. Use 'sort -ru' instead to really remove duplicates and to sort /opt/vmware/lib/lib/ _behind_ /opt/vmware/lib/lib/lib*so.[0-9]/
If you run env SEARCH_DIRS_MASK="/opt/vmware/lib" revdep-rebuild <options> that should completely fix the issue with vmware. (You can also place the SEARCH_DIRS_MASK entry in /etc/make.conf so that you don't have to type it.) However, I'm curious as to how /opt/vmware/lib is getting into the list of directories to search. I have vmware and it isn't in my $PATH or ld.so.conf which is where this version of revdep-rebuild gets the directories to search.
*** Bug 85811 has been marked as a duplicate of this bug. ***
Created attachment 58088 [details] revdep-rebuild Updated version of revdep-rebuild. Contains minor grammatical fixes, updated copyright statement, bug fix for removing duplicates from LD_LIBRARY_PATH, and the patch from bug #91503 to allow revdep-rebuild --soname /path/to/library.so
Created attachment 58170 [details, diff] Patch to portage.py for env-update to support SEARCH_DIRS and SEARCH_DIRS_MASK This patch adds support to env-update to properly process SEARCH_DIRS and SEARCH_DIRS_MASK variables in /etc/env.d files.
(In reply to comment #19) > Created an attachment (id=40742) [edit] > revdep-rebuild man page > I have added a basic man page that much more clearly describes part of the functions and options of revdep-rebuild to bug 40042. I think statements like "recompiles such-and-so" are misleading and should not be used. revdep-rebuild compiles nothing and only passes package names to emerge. Please check out my new man page and point out any mistakes and missing feature descriptions.
Created attachment 60683 [details, diff] cosmetic.patch Maybe it's useful this cosmetic patch to revdep-rebuild :-)
Created attachment 60685 [details, diff] cosmetic.patch Sorry, for the wrong post. Now the patch support the -nc and -q options
I need to add LD_LIBRARY_MASK to the incremental variables supported by portage. I overlooked it when I created the portage.py patch.
Arch teams: I am CC'ing all arch teams that were keyworded in gentoolkit on this bug. I want to verify that I have not broken the portability of revdep-rebuild with this update. I have added gentoolkit_0.2.1_pre3 to the tree package masked with KEYWORDS="-* ~x86". Please test when you have a chance and keyword and add comments to this bug as appropriate. The man page and the gentoolkit ChangeLog have all of the information on the updates to this version of revdep-rebuild.
~amd64 added
added ~ppc64
Marked ~ppc.
Marked ~alpha
~sparc'ed.
Created attachment 61451 [details, diff] Patch to add incremental support to portage for revdep-rebuild enviroment variables
~ia64 done
Sadly, this doesn't apply to ppc-macos. Correct me if I'm wrong.
I still would like arm, hppa, and s390 architectures to test and verify revdep-rebuild in gentoolkit_0.2.1_pre4
Hi, Not a big problem, but the newest (~x86) OOo-bin-1.9.130 installs in /usr/bin not /opt, so revdep-rebuild lists it as a rebuild candidate. Is there a way to use SEARCH_DIRS_MASK to skip OpenOffice-1.9.130. As this is a commonly used package many people may ask/wonder or just rebuild it. Thanks.Rumen
* Make sure that you have >gentoolkit-0.2.0 installed. It is currently KEYWORDED "~arch" * Edit /etc/make.conf and add SEARCH_DIRS_MASK="/usr/lib/openoffice"
Fix is in gentoolkit-0.2.1
