Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 11981 - -S / --searchdesc speedup?!
Summary: -S / --searchdesc speedup?!
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: High enhancement (vote)
Assignee: Nicholas Jones (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2002-12-11 15:15 UTC by Torgeir Hansen
Modified: 2011-10-30 22:37 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Torgeir Hansen 2002-12-11 15:15:11 UTC
Currently, emerge -S is a nice feature to find packages that a simple -s won't dig up for us (me:)
However, emerge -S takes way, WAY to long time to complete!
I'm currently using gentoo on a variety of systems, from a dual ppro 180 mhz to 800Mhz amd and a 850Mhz PIII laptop - and if I get my way, soon I'll be using it on my 175Mhz MIPS aswell! ;]
some have ide-drive(s) and some have a scsi-drive(s), but the systems have all one thing in common;
it takes way too long!

My suggestion is to speed it up, either with a desc.dat or whatever which can be distributed as the portage-files via rsync,
or by a on-line search application.
The latter is my preferance, as it would probably be the fastest way to search - if build properly :)
Comment 1 Sven Vermeulen (RETIRED) gentoo-dev 2003-01-11 04:48:32 UTC
I'm currently testing various methodes for improving the speed of emerge's "-s",
"--searchdesc", "--upgrade --pretend world", qpkg's "-f" etc... 

The problem is all solutions I have tested yet have some setbacks. Or they make
the system more complicated, or they save information on double locations, or
they require considerate resources (more RAM, f.i.). 

I'll post my results on gentoo-dev after I have finished testing all situations,
but this can still take a lot of time (only 5 of the 12 testcases finished yet).

Some ideas that are already finished testing:
- /usr/portage is on a tmpfs-system (with on-write-disk-access so that it
doesn't disappear when rebooten)
- /usr/portage-files in a mysql-database (+ on disk for when mysql doesn't work)
- portage using specialcrafted binary-file, possibly mounted using loopback and
some special filesystem (requires kerneladjustments) (like rpm's db3 + some
special mount filesystem so that they are seen as regular ebuilds)
- descriptions, rdepend, depend, homepage in one seperate file for all ebuilds
(or mixed: double information storage), configure/make information in seperate
files (as are the ebuilds now)
- caching of the needed information into a temporary file that Portage can use,
and builds during an "emerge (r)sync" (finished: information storage in cache
using B- and B*-trees, still testing other methods)
Comment 2 Alain Penders (RETIRED) gentoo-dev 2003-02-03 01:14:40 UTC
Any other news on the testing?
Comment 3 Sven Vermeulen (RETIRED) gentoo-dev 2003-02-03 07:51:36 UTC
Well, at work we're currently implementing Portage with MySQL so that several systems use 1 central database. The setbacks is that Portage won't work if the network's out or the server's down. The advantages are that it's very fast (the server also compiles the packages, but that's another thing).

Having the system work with tmpfs on /usr/portage and /var/db/pkg (with write-back mechanism) is very fast, but it consumes lots of memory (about 250 meg).

Portage using a binary format (such as db3) makes the system less flexible since it's not that easy to upgrade. That is a serious drawback, especially for Gentoo since it has a fast development cycle.

Caching is the same as using a binary format, but it's more flexible (the cache is generated each time the respective emerge-command is executed). However, emerge (r)syncs and installation of software is a little slower (several seconds).

All other options have been found to be not interesting or flawed. Currently MySQL is on second phase research.
Comment 4 Nicholas Jones (RETIRED) gentoo-dev 2003-02-17 06:44:51 UTC
Athlon700 -- portage-2.0.47-r2

'emerge -s' is less than 2 seconds.

time emerge -S test >/dev/null

real 31.711 seconds
user 23.269 seconds
sys   4.945 seconds