Following discussion on bug #130004, i've tried to optimize the new qfile code. The bad news is that i have not any significant optimization for the "qfile /single/file" case. I've removed a few unnecessary strlen() calls, but it really makes no difference. % time ./q file /bin/bash > /dev/null 0.1.18: ======= real 0m0.642s user 0m0.551s sys 0m0.054s ------------------ real 0m0.641s user 0m0.551s sys 0m0.054s 0.1.19: ======= real 0m0.710s user 0m0.620s sys 0m0.055s ------------------ real 0m0.711s user 0m0.612s sys 0m0.063s New patch: ========== real 0m0.707s user 0m0.622s sys 0m0.053s ------------------ real 0m0.711s user 0m0.614s sys 0m0.059s The good news is a big optimization for the "qfile /lot/of/files/*" case. Currently, the CONTENTS file is read and parsed once per commandline argument. With the attached patch, it is read/parsed only once, and an iteration over arguments is done for each CONTENTS line. % ls /bin/* | wc -l 155 % time ./q file /bin/* > /dev/null O.1.18: ======= real 1m31.402s user 1m22.554s sys 0m7.532s ------------------ real 1m30.428s user 1m22.940s sys 0m7.401s 0.1.19: ======= real 1m40.792s user 1m33.380s sys 0m7.342s ------------------ real 1m40.836s user 1m33.448s sys 0m7.327s New patch: ========== real 0m24.725s user 0m24.596s sys 0m0.082s ------------------ real 0m24.826s user 0m24.681s sys 0m0.101s
Created attachment 91320 [details, diff] qfile-0.1.19-optimization.patch
Thanks to OProfile, i've found an other significant optimization in avoiding using "basename()" on CONTENTS entries (because afaik, not all implementations of basename() guarantee they won't modify their parameter, and thus i was doing a lot of "strdup()" calls around it). I will attach a modified patch, where it is replaced by a "strrchr(entry->name,'/')+1". This works just as well since paths in CONTENTS files are clean (absolute, with a no trailing slash). And i did something similar for dirnames, although this one happens much less often (only when basename matches). New bench: % time ./q file /bin/bash > /dev/null real 0m0.594s user 0m0.503s sys 0m0.057s ----------------- real 0m0.601s user 0m0.510s sys 0m0.056s % time ./q file /bin/* > /dev/null real 0m1.237s user 0m1.142s sys 0m0.059s ----------------- real 0m1.239s user 0m1.138s sys 0m0.064s Searching for 155 files is now just twice slower than searching one :) (all figures are with a well filled IO cache, sure)
Created attachment 91385 [details, diff] qfile-0.1.19-optimization.patch
committed InCVS revision: 1.31
Released in portage-utils-0.1.20
