Sorry if this is messy, but I would have just included a diff but i've made ALL sorts of changes/improvements to my qpkg. These are just the ones that I think it would be easy to agree on. Here's what I think is the bug inside the dup checking code. ---- START CODE ---- #Next get all the exact versions duppak=`cat /tmp/qpkg.lst | fgrep "${dups}"` #Now cut that down to the directory name so we can be smart # !! fixing bug to keep from bring ALL the packages back in # !! again when we just went through the trouble to filter them above # dirs=`sed -e 's:/[^/]*$::' /tmp/qpkg.lst` dirs=`echo ${duppak} " " | sed -e "s:/[^/ ]* : :g"` ---- STOP CODE ---- And here is my speed increase (tested with no probs on x86). It's way faster to use BASH's string manipulation inside the loop than call both sed and dirname. ---- START CODE ---- # n=`echo $p | sed -e "s:^/var/db/pkg/${X}/${X}/${X}.ebuild:iii \1/\3:" \ # -e "s:^/usr/portage/${X}/${X}/${X}\.ebuild:uuu \1/\3:"` # accomplishes the same as the above sed command except # bash parameter completetion is used to eliminate external calls n=${p%.ebuild} var_db_pkg="/var/db/pkg/" n=${n/${var_db_pkg}/iii } usr_portage="/usr/portage/" n=${n/${usr_portage}/uuu } n=${n/\/*\//\/} ... d=${p%\/*.ebuild} # faster d=`dirname ${p}` ---- STOP CODE ---- Reproducible: Always Steps to Reproduce: For first bug: any time qpkg is run in duplication checking mode will see a speed up For second speed increase: any parameters that utilize the for that loops through packages will see a speed up Actual Results: Slower speed that we could have, though always correct info. Expected Results: Faster speed.
In the code I pasted I had your original code commented out and my code uncommented...
See http://forums.gentoo.org/viewtopic.php?p=132434#132434 for some additional details and the full extent of my improvements/modifications with a write-up.
You've made other local enhancements to qpkg you say? Do share! I would love it if you posted a complete diff, I can pick and choose hunks, and test locally before committing to gentoolkit. Of course please make the diff based on the latest qpkg at /usr/portage/app-admin/gentoolkit/files/scripts/ Thanks for your help! (Sorry it's been so long on this, I'm a lamer.)
Way busier now than I was back then :( though when I have a second I will glance over the latest code and see if my two biggest _simple_ improvements are still valid and will send you a diff.
Created attachment 11528 [details, diff] Diff between slighly patched qpkg and original *CHANGES* Fixes qpkg db pulling in every package because grep matches /var/"db"/pkg... Speedups by no longer calling dirname and sed inside the main loop... Try "oldqpkg -v" compared to "newqpkg -v"... this speed up is only apparent with large lists... not just with one or two items shown... Drop from main loops sooner if no params are passed...
The qpkg I still use (modified) has caching build in as well as other features like -du (disk usage), -cl (changelog), etc... I can upload the whole thing if your interested but I don't have time to port all my changes to the latest version. Honestly qpkg seems to have changed only slightly anyways since I made my changes... a diff of my changed version vs the latest wasn't SUPER off... mostly typos and spacing corrections and stuff... Let me know.
Please do upload your whole file, and I'll try to forward port any other changes and get it into the next gentoolkit version, thanks. Look around bugzilla, I think someone is working on a python replacement for gentoolkit you might be interested in working on it. later
one more qpkg bug
this tool is deprecated
