Here is a serie of 6 patches for "qgrep", which adds a few new features. Details about what each patch does follow, in the attachment comments. The first one is for CVS revision 1.17, and then they are incremental. Also, I will attach an equivalent all-in-one patch at the end. (I've thought that opening 6 bugs for that would have been spam, my apologies if you disagree...)
Created attachment 113690 [details, diff] qgrep-01_regexp_compilation_speedup.patch Reduces CPU usage of "-e/--regexp" by compiling the pattern once for all. Before: % time ./orig/q grep -Ee '\<e\(info\|warn\|log\|error\)\>' >/dev/null real 0m1.511s user 0m1.465s sys 0m0.013s After: % time ./q grep -Ee '\<e\(info\|warn\|log\|error\)\>' >/dev/null real 0m0.106s user 0m0.067s sys 0m0.006s
Created attachment 113693 [details, diff] qgrep-02_add_-l_and_-L_options.patch Add some "-l/--list" and "-L/--invert-list" options, similar to GNU Grep "--files-with-matches" and "--files-without-match". Only a list of files will be displayed in this mode. Like in GNU Grep, when this option is use, "--count" is ignored.
Created attachment 113695 [details, diff] qgrep-03_add_a_--skip_option.patch Add a "-S/--skip <pattern>" option. It is a generalisation of the --skip-comments option: lines which match the pattern are ignored. The skip pattern is interpreted just like the search pattern, depending on other options (substring or regexp, case sensitive or insensitive). I find it useful to drop some false-positive when grepping for some actual code: % ./q grep -ves -S '^[ ]*\(e\(info\|warn\|error\|log\)\>\|DESCRIPTION=\)' '\<which\>' app-arch/rpm2targz/rpm2targz-9.0-r2.ebuild: sys-apps/which" app-arch/rpm2targz/rpm2targz-9.0-r3.ebuild: sys-apps/which" app-arch/rpm2targz/rpm2targz-9.0-r4.ebuild: sys-apps/which )" app-arch/rpm2targz/rpm2targz-9.0-r5.ebuild: sys-apps/which )" app-dicts/stardict/stardict-2.4.8.ebuild: export PKG_CONFIG=`which pkg-config` app-dicts/verbiste/verbiste-0.1.16.ebuild: cp $(which gettextize) "${T}"/ || die "could not copy gettextize" ...
Created attachment 113697 [details, diff] qgrep-04_add_a_--with-name_option.patch Add a "-H/--with-name" option, which makes qgrep display cat/pkg-ver instead of ebuild paths. Also, when using -E/--eclass, only the eclass name is displayed: % ./q grep -N foo app-admin/skey-1.1.5-r5: # allow invokation as otp-foo. #71015 app-admin/skey-1.1.5-r6: # allow invokation as otp-foo. #71015 app-admin/system-config-printer-0.7.27: net-print/foomatic ... % ./q grep -NE foo toolchain-funcs: # gen_usr_ldscript libfoo.so subversion: # e.g. http://foo/trunk, svn://bar/trunk kmod: # if is_kernel 2 6; then foo; fi ...
Created attachment 113698 [details, diff] qgrep-05_add_a_--extended_option.patch Add a "-x/--extended" option, to allow extended regular expressions. For instance: % ./q grep -NEx '^[[:space:]]*e(info|warn|error|log)\>' vim-spell: einfo "To enable ${VIM_SPELL_LANGUAGE} spell checking, use" vim-spell: einfo " :setlocal spell spelllang=${VIM_SPELL_CODE}" vim-spell: einfo "The following (Vim internal, not file) encodings are supported for" ...
Created attachment 113699 [details, diff] qgrep-06_take_additional_args_as_a_search_restriction.patch Currently, qgrep only read it's first command line argument, and ignore the later ones. Also, it always search through all ebuilds (or eclasses). With this patch, later arguments are interpreted as the set of files qgrep should search in. Arguments can be: - eclass names (when using "-E"). This one is convenient for instance when you don't remember what functions an eclass define: % ./q grep -Es '()' games egamesconf() { egamesinstall() { gameswrapper() { ... - ebuild names, with or without it's "category/": % ./q grep -Ne '^KEYW' portage-utils app-portage/gentoolkit app-portage/gentoolkit-0.2.2: KEYWORDS="alpha amd64 arm hppa ia64 m68k mips ppc ppc64 s390 sh sparc x86 ~x86-fbsd" app-portage/gentoolkit-0.2.3: KEYWORDS="alpha amd64 arm hppa ia64 m68k mips ppc ppc64 s390 sh sparc ~sparc-fbsd x86 ~x86-fbsd" app-portage/gentoolkit-0.2.4_pre1: KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd" app-portage/portage-utils-0.1.23: KEYWORDS="alpha amd64 arm hppa ia64 m68k mips ppc ppc64 s390 sh sparc ~sparc-fbsd x86 ~x86-fbsd" app-portage/portage-utils-0.1.24: KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~sparc-fbsd ~x86 ~x86-fbsd" - category names (must be followed by a "/", or it will be interpreted as a package name): % ./q grep -Nx '^# \$Header.*\<solar\>' app-portage/ sys-devel/ app-portage/portage-utils-0.1.24: # $Header: /var/cvsroot/gentoo-x86/app-portage/portage-utils/portage-utils-0.1.24.ebuild,v 1.1 2007/02/11 22:49:21 solar Exp $ sys-devel/gcc-3.2.3-r4: # $Header: /var/cvsroot/gentoo-x86/sys-devel/gcc/gcc-3.2.3-r4.ebuild,v 1.22 2006/08/14 16:32:23 solar Exp $ sys-devel/gcc-3.3.5-r1: # $Header: /var/cvsroot/gentoo-x86/sys-devel/gcc/gcc-3.3.5-r1.ebuild,v 1.32 2006/08/14 16:32:23 solar Exp $ ... - full "cat/pkg-ver". I like this one to get a quick stripped view of a particular ebuild: % ./q grep -se . app-portage/portage-utils-0.1.24 | more inherit toolchain-funcs DESCRIPTION="small and fast portage helper tools written in C" HOMEPAGE="http://www.gentoo.org/" SRC_URI="mirror://gentoo/${P}.tar.bz2" ...
Created attachment 113701 [details, diff] qgrep-all_in_one.patch Merge of the 6 previous patches. Don't worry, it's actually not too big: % diffstat qgrep-all_in_one.patch qgrep.c | 160 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 147 insertions(+), 13 deletions(-)
(In reply to comment #1) > Created an attachment (id=113690) [edit] > qgrep-01_regexp_compilation_speedup.patch Pretty impressive speedup. solar@here /var/cvsroot/gentoo-projects/portage-utils $ time ./q grep -e '\<e\(info\|warn\|log\|error\)\>' >/dev/null real 2m36.379s user 0m2.616s sys 0m4.884s solar@here /var/cvsroot/gentoo-projects/portage-utils $ time /usr/bin/q grep -e '\<e\(info\|warn\|log\|error\)\>' >/dev/null real 6m10.265s user 2m27.957s sys 0m10.597s
(In reply to comment #2) > Created an attachment (id=113693) [edit] > qgrep-02_add_-l_and_-L_options.patch I like this. Saves us from having to use awk '{print $1}' however if you could make it work with the -vv that would be handy.
(In reply to comment #4) > Created an attachment (id=113697) [edit] > qgrep-04_add_a_--with-name_option.patch > > Add a "-H/--with-name" option, which makes qgrep display cat/pkg-ver instead > of ebuild paths. I think you mean "Add a -N option" here vs -H. Also if using -H -N -H do you think the last option given should negate the other?
(In reply to comment #6) > Created an attachment (id=113699) [edit] > qgrep-06_take_additional_args_as_a_search_restriction.patch nice I like it.
(In reply to comment #9) > (In reply to comment #2) > > Created an attachment (id=113693) [edit] > > qgrep-02_add_-l_and_-L_options.patch > > however if you could make it work with the -vv that would be handy. > I'm not sure i understand, what output would you expect for -lvv? (In reply to comment #10) > (In reply to comment #4) > > Created an attachment (id=113697) [edit] > > qgrep-04_add_a_--with-name_option.patch > > > > I think you mean "Add a -N option" Oops, yes, sure. > Also if using -H -N -H do you think the last > option given should negate the other? Imho, options order should not change their behavior. I prefer having some arbitrary priorities (-N stronger than -H, like -l is stronger than -c). What about keeping that behavior, but adding some warnings for incoherent combinations?
(In reply to comment #12) > (In reply to comment #9) > > (In reply to comment #2) > > > Created an attachment (id=113693) [edit] > > > qgrep-02_add_-l_and_-L_options.patch > > > > however if you could make it work with the -vv that would be handy. > > I'm not sure i understand, what output would you expect for -lvv? With the two or more -v options we currently display the line number. solar@here ~ $ qgrep -Hvv '$(which ' app-dicts/verbiste/verbiste-0.1.16.ebuild:28: cp $(which gettextize) "${T}"/ || die "could not copy gettextize" user@shell ~ $ qgrep -lvv '$(which ' app-dicts/verbiste/verbiste-0.1.16.ebuild:28:
(In reply to comment #13) > > user@shell ~ $ qgrep -lvv '$(which ' > app-dicts/verbiste/verbiste-0.1.16.ebuild:28: > Ok, i will attach one more patch with the following behavior: * "-l" lists matching files once (no change here): % ./qgrep -l unpack | head -n4 app-accessibility/SphinxTrain/SphinxTrain-0.9.1-r1.ebuild app-accessibility/at-poke/at-poke-0.2.2.ebuild app-accessibility/brltty/brltty-3.7.2.ebuild app-accessibility/dasher/dasher-4.0.2.ebuild * "-lv" lists matching files once per matching line (not really useful, but a logical intermediate between -l and -lvv): % ./qgrep -lv unpack | head -n4 app-accessibility/SphinxTrain/SphinxTrain-0.9.1-r1.ebuild app-accessibility/SphinxTrain/SphinxTrain-0.9.1-r1.ebuild app-accessibility/at-poke/at-poke-0.2.2.ebuild app-accessibility/at-poke/at-poke-0.2.2.ebuild * "-lvv" lists matching files once per matching line, and give the line numbers (what you asked, except i've dropped the colon char at EOL): % ./qgrep -lvv unpack | head -n4 app-accessibility/SphinxTrain/SphinxTrain-0.9.1-r1.ebuild:22 app-accessibility/SphinxTrain/SphinxTrain-0.9.1-r1.ebuild:23 app-accessibility/at-poke/at-poke-0.2.2.ebuild:33 app-accessibility/at-poke/at-poke-0.2.2.ebuild:34 This patch also adds usage warning for two unsupported/meaningless options combinations: * --list (or --invert-list) and --count * --with-name and --with-filename
Created attachment 114221 [details, diff] qgrep-07_misc_options_warnings_and_-lvv_support.patch
Created attachment 114222 [details, diff] qgrep-all_in_one.patch Updated all-in-one patch (includes patches #01 to #07).
Created attachment 114281 [details, diff] qgrep-07_misc_options_warnings_and_-lvv_support.patch I had used some warnf()/errf() where warn()/err() was what i meant.
Created attachment 114282 [details, diff] qgrep-all_in_one.patch
solar@onyx /var/cvsroot/gentoo-projects/portage-utils $ cvs ci -m "- Misc enhancements for qgrep. TGL bug 171374" ... /var/cvsroot/gentoo-projects/portage-utils/qgrep.c,v <-- qgrep.c new revision: 1.18; previous revision: 1.17
Created attachment 114944 [details, diff] strstr_in_qgrep_name_match.patch instead of only matching the exact category, package name, or eclass with strcmp, we can use strstr. this allows us for example to grep * in every vim eclasses instead of just vim.eclass in case we use vim as the optionnal argument. * to grep in every games categories using qgrep searchme games/ and so on...
(In reply to comment #20) > * in every vim eclasses instead of just vim.eclass in case we use vim as the > optionnal argument. well, actually this one is not a good example since we then lose the ability to only grep in the vim eclass. so... I think using strstr instead of strcmp, is only usefull for the CATEGORY
(In reply to comment #20) > we can use strstr. I'm not sure i like this idea, it's a bit to much magic imho. It's like in "qlist cat/pkg", i always forget to add this --exact option which is almost always what i actually want. Here, I expect the usual use case to be searching in "one/some specific category(ies)" rather than in "all categories that match some substrings". For the rare occasions i would want the later, i can live with either this: % cd /usr/portage % qgrep searchme *games*/ or this (to avoid changing dir): % qgrep searchme $(sed -n '/games/s:$:/:p' /usr/portage/profiles/categories) That said, i understand it's really a matter of taste... It's up to upstream :)
This is released in 0.1.25 Bug #168334 ; q -r dies with a segfault after emerge --sync Bug #168442 ; does not properly parse the profile location Bug #170795 ; add a -E/--eclass option to qgrep Bug #170797 ; add a -s/--skip-comments option to qgrep Bug #171024 ; opening '/usr/portage/.metadata.x' failed Bug #171374 ; Misc enhancements for qgrep Bug #172240 ; -A/-B options for qgrep (context lines) Bug #172338 ; qgrepping through installed ebuilds (in the VDB) Bug #173005 ; Colorized output for qgrep.
Closing
