Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 26532

Summary: esearch - Replacement for 'emerge search' with search-index
Product: Gentoo Linux Reporter: David Peter <davidpeter>
Component: New packagesAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: enhancement CC: davidpeter, karltk, seemant
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: http://david-peter.de/esearch
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: esearch-0.3.0.ebuild
esearch-0.4.0.ebuild

Description David Peter 2003-08-13 02:43:11 UTC
esearch is a small python script to search the portage tree. The package
includes 'eupdatedb', which will generate an index-file with package-names,
descriptions, homepage etc. After the index is compiled, you can search with
'esearch'. It behaves exactly like 'emerge search'.

David
Comment 1 David Peter 2003-08-13 02:44:59 UTC
Created attachment 16027 [details]
esearch-0.3.0.ebuild
Comment 2 Shahar Goldin 2003-08-13 03:02:20 UTC
This not only works for me, it's also increadible-- 
time emerge -s moz
real    0m1.736s
user    0m1.524s
sys     0m0.112s

time esearch moz
real    0m0.135s
user    0m0.121s
sys     0m0.009s


time emerge -S browser
real    0m39.918s
user    0m26.439s
sys     0m1.840s

time esearch -S browser
real    0m0.150s
user    0m0.135s
sys     0m0.005s


I think this should replace emerge -s and emerge -S
it takes a while to compile the index
but DAMN its fast :-)
Comment 3 SpanKY gentoo-dev 2003-08-13 15:51:36 UTC
rather than doing this i think we should just work on moving all of portage over 
to a compiled db format (Bug 20330) 
Comment 4 David Peter 2003-08-14 07:52:15 UTC
I think esearch could, at least, provide a temporary solution. I don't think that the portage db will move to a compiled format very soon, although there are even some implementations (http://portagesql.breakmygentoo.net/).
Or are there already some plans to change the format of the portdb?
Furthermore, esearch has some additional functionality (the --compact switch) and maybe I'll add some more in the next versions.

But anyway, if the portage devs are planning to move the portdb to a "real" db, it makes no sense to add esearch to the portage-tree.

Thanks,

David
Comment 5 Masatomo Nakano (RETIRED) gentoo-dev 2003-08-14 08:25:28 UTC
There is one implement for using db in bug 26447
Comment 6 Nicholas Jones (RETIRED) gentoo-dev 2003-08-15 17:26:20 UTC
Karl take a look at this for gentoolkit? Looks interesting.
Portage will be using dbm eventually, but this looks like a
good "now" solution.
Comment 7 Seemant Kulleen (RETIRED) gentoo-dev 2003-09-16 15:34:02 UTC
David, I love this esearch thing.  The only brokenness I've discovered is: 

esearch mysql++

doesn't exit cleanly because I guess it assumes it to be a regex or something.

Also -- eupdatedb takes a long long time -- any way to speed that up?
Comment 8 David Peter 2003-09-17 11:04:30 UTC
Thanks for your comments. I've fixed this "bug" with mysql++. I say "bug", because it isn't really a wrong behaviour, but it's a wrong regex. In fact, emerge -s fixes this regex and adds a slash: ++ -> \+\+.

I've done some profiling and found out some functions, that are really slow. I'll try to work on eupdatedb, to improve speed.

You may try out version 0.3.1. Just copy the esearch-0.3.0.ebuild to esearch-0.3.1.ebuild, and "emerge esearch".

I would be really happy if it would be added to portage, or to gentoolkit.

PS: eupdate *is* slow compared to other search-scripts (like fastsearch for example), but it has the advantage, that it uses the data directly from portage(.py) so you can be sure, it shows you the right version, description and homepage (e.g. fastsearch cannot evaluate eclasses)
Comment 9 Seemant Kulleen (RETIRED) gentoo-dev 2003-09-19 00:55:59 UTC
I added this to portage, the 0.4.0 version is great.
Comment 10 David Peter 2003-09-19 03:03:32 UTC
Hi,

Thanks a lot, Seemant.

But I've got a - maybe stupid - question: Where did my announce of esearch-0.4.0 go? The new ebuild is also missing. However, it seems as if you had the chance to read it, before it disappeared :-)

David
Comment 11 SpanKY gentoo-dev 2003-09-19 05:26:07 UTC
------- Additional Comments From shark@phpwelt.com  2003-18-09 07:43 EST -------
Hi,

After a bit of experimenting today, I found out a weak point in eupdatedb. The little mistake I made, was to write the whole package database into the memory, and then write all at once to the index-file. Now I changed the code and it writes the packages directly into the file. This behaviour does not only save free memory, but it really speeds eupdatedb up alot. Now it runs about 10 times faster on my computer.

I also added a ChangeLog and 2 other features. I'll attach the new ebuild esearch-0.4.0.ebuild.

Thanks for your interest.
Comment 12 David Peter 2003-09-19 06:25:51 UTC
Created attachment 17974 [details]
esearch-0.4.0.ebuild