Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 154405 - app-portage/portage-utils: add overlays support
Summary: app-portage/portage-utils: add overlays support
Status: RESOLVED NEEDINFO
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Portage Utils Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-11-07 15:07 UTC by samLT
Modified: 2012-10-28 09:46 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
0.1.21-overlay-support.patch (0.1.21-overlay-support.patch,50.98 KB, patch)
2006-11-07 15:07 UTC, samLT
Details | Diff
overlay-support.patch against portage-utils-20061007 (overlay-support.patch,39.58 KB, patch)
2006-11-08 14:01 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,39.58 KB, patch)
2006-11-08 22:55 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,36.73 KB, patch)
2006-11-09 15:14 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,36.69 KB, patch)
2006-11-11 08:26 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,36.65 KB, patch)
2006-11-11 11:43 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,36.93 KB, patch)
2006-11-12 04:10 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,37.01 KB, patch)
2006-11-12 05:53 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,36.98 KB, patch)
2006-11-12 13:42 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,37.13 KB, patch)
2006-11-14 15:30 UTC, samLT
Details | Diff
FYI: quse.patch (quse.patch,2.20 KB, patch)
2006-11-20 16:19 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,37.51 KB, patch)
2006-11-20 16:20 UTC, samLT
Details | Diff
overlay-manpages.patch (overlay-manpages.patch,5.69 KB, patch)
2006-12-27 11:56 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,71.89 KB, patch)
2007-01-15 20:42 UTC, samLT
Details | Diff
portage2paludis.patch (portage2paludis.patch,45.42 KB, patch)
2007-01-15 20:42 UTC, samLT
Details | Diff
overlay-support.patch (overlay-support.patch,51.95 KB, patch)
2007-01-16 15:52 UTC, samLT
Details | Diff
portage2paludis.patch (portage2paludis.patch,44.42 KB, patch)
2007-01-16 15:53 UTC, samLT
Details | Diff
overlay-support.patch portage2paludis.patch and more.. (url.txt,69 bytes, text/plain)
2007-01-19 15:18 UTC, samLT
Details

Note You need to log in before you can comment on or make changes to this bug.
Description samLT 2006-11-07 15:07:15 UTC
Hi, I was patching portage-utils so that it works with paludis, and thus with overlays, I though others than me might be interested by this. 

basicaly, I changed the syntax of the .ebuild.x a bit: 

overlay::category/packagename/packageversion.ebuild

where 'overlay' is the name of the overlay  (name found in /path/to/overlay/profile/repo_name otherwise basename( /path/to/overlay) )

This file is still in $portdir but contains ebuild from every overlay (every overlay that is defined in the $PORTDIR_OVERLAY)

I also added the option --overlay ( -o ) for the qgrep, qsearch and quse tool (for the last one I just hope it will encourage overlay maintainers to describe their own flag in the correspondant files ( profiles/desc/* and profiles/use.*desc )

This option allow you to do the given action for only one overlay.

AFAIK, portage doesn't have yet an option to list overlay. that's why I added the option --ls-overlays (-l) to base tool 'q' ( q --ls-overlays )

Hope you'll like it. Every comment is welcomed.
Comment 1 samLT 2006-11-07 15:07:52 UTC
Created attachment 101420 [details, diff]
0.1.21-overlay-support.patch
Comment 2 SpanKY gentoo-dev 2006-11-07 15:20:23 UTC
Comment on attachment 101420 [details, diff]
0.1.21-overlay-support.patch

post a new patch that isnt full of pointless whitespace changes
Comment 3 samLT 2006-11-07 15:27:13 UTC
(In reply to comment #2)
> (From update of attachment 101420 [details, diff] [edit])
> post a new patch that isnt full of pointless whitespace changes
> 

white space change are there for the better, just keep them, I just correct what wasn't correct from the beginning..

Anyway, I don't see any problem. but feel free to remove them
Comment 4 SpanKY gentoo-dev 2006-11-07 15:40:56 UTC
i'm not going to pick through and figure out what is legit changes and what is whitespace

as for it being "for the better", that's nice, see above
Comment 5 Thomas Cort (RETIRED) gentoo-dev 2006-11-07 19:06:42 UTC
Your patch is against 0.1.21. A lot of changes have happened since then (for qcache at least). Please make your patch against CVS.

cvs -d cvs.gentoo.org:/var/cvsroot co gentoo-projects
http://sources.gentoo.org/viewcvs.py/gentoo-projects/portage-utils/

I don't know if it will allow you to checkout the sources anonymously, so I created a tarball for you :)
http://dev.gentoo.org/~tcort/tmp/portage-utils-20061007.tar.bz2
Comment 6 samLT 2006-11-08 14:00:44 UTC
ok, I did delete the 'useless' part of the patch, so here is the new against your tarball 
Comment 7 samLT 2006-11-08 14:01:33 UTC
Created attachment 101495 [details, diff]
overlay-support.patch against portage-utils-20061007
Comment 8 SpanKY gentoo-dev 2006-11-08 15:37:55 UTC
but you didnt ... that still contains pointless changes:
 - whitespace changes == noise
 - do not change portage -> paludis
 - do not insert spurious whitespace
 - do not omit whitespace where it belongs
 - cuddle braces (except for func def)
 - do not cast xmalloc() return

ignoring all that, rather than having a portage tree var and an overlay var, it'd probably simplify thing to just have a list of trees to search
Comment 9 samLT 2006-11-08 22:53:51 UTC
(In reply to comment #8)
> but you didnt ... that still contains pointless changes:
>  - do not change portage -> paludis
I'm sorry about that

>  - do not omit whitespace where it belongs
>  - cuddle braces (except for func def)
I don't understand this one:/

>  - do not cast xmalloc() return
actually I tried to take the same syntax as you did. just grep for xmallox and you'll see you're sometimes doing a cast, sometimes not.

 
> ignoring all that, rather than having a portage tree var and an overlay var,
> it'd probably simplify thing to just have a list of trees to search

actually, since it's a simple chained list, the overlay_gentoo is just the start of the list.. that makes things simpler, since the 'repodir' is often use when it HAS to be the portage tree path. (especially for qcache). Doing what you said  would just add useless loop where it is just not needed (I think)

Anyway thanks for your time. 
Comment 10 samLT 2006-11-08 22:55:13 UTC
Created attachment 101520 [details, diff]
overlay-support.patch

without stupid word changes
Comment 11 SpanKY gentoo-dev 2006-11-08 23:46:41 UTC
ive cleaned up the cvs tree as i noticed there were inconsistencies

in the README it tells you how to pull from anoncvs
Comment 12 samLT 2006-11-09 15:11:40 UTC
So here is the patch for the revision of today: 20061110

there shouldn't be any space changes, nor cast'ing (?) on xmalloc() return.
Comment 13 samLT 2006-11-09 15:14:15 UTC
Created attachment 101575 [details, diff]
overlay-support.patch

So here is the patch for the revision of today: 20061110

there shouldn't be any space changes, nor cast'ing (?) on xmalloc() return.

I still don't know what you meant with 'cuddle brace'.. (my dictionnary didn't really help..)
Comment 14 samLT 2006-11-11 03:02:55 UTC
> ignoring all that, rather than having a portage tree var and an overlay var,
> it'd probably simplify thing to just have a list of trees to search

there is an other reason for that, the file .ebuild.x is stored in the $portdir, and there is only one .ebuild.x file for all overlays
Comment 15 samLT 2006-11-11 08:26:25 UTC
Created attachment 101690 [details, diff]
overlay-support.patch

previously, depending on the loop, the last repository was skip, this wrong behaviour is corrected
Comment 16 samLT 2006-11-11 11:43:49 UTC
Created attachment 101699 [details, diff]
overlay-support.patch

Sorry for the bump, this one deals better with no longer existing overlays (but referenced in the .ebuild.x)
Comment 17 SpanKY gentoo-dev 2006-11-12 03:02:25 UTC
why did you change qglsa in applets.h ?  i also still see some xmalloc() casts in there ...

wonder if it'd be useful to make the overlay option a global one ?  maybe not so much ...
Comment 18 samLT 2006-11-12 04:09:08 UTC
(In reply to comment #17)
> why did you change qglsa in applets.h ?
Actually, I probably should not make this change in the public patch, but since qglsa is disabled in the Makefile, it just avoids the qglsa symlink to be created
 
>  i also still see some xmalloc() casts in there ...
oops..

> wonder if it'd be useful to make the overlay option a global one ?  maybe not
> so much ...
I don't think, the only applets concerned are qsearch, qgrep, and quse, for the others it just doesn't have any sense (for qcache: I haven't seen an overlay with metadata/cache yet, and due to portage log format, it's not easily possible to add overlay support to qlop too)


Here is a new one (without the qglsa change). It also stops(with an error message) when the user tries to to select an overlay that doesn't exist ( qapplet --overlay wrong_overlay_name)

The patch also change a little bit test/install/dotest due to a space added in the "Usage:" help

well I think that's it for the portage part.


btw, once all this will be correct, do you think we can add a "paludis" use flag, if you say so, i will post the patch too, but, one thing at a time..
Comment 19 samLT 2006-11-12 04:10:08 UTC
Created attachment 101723 [details, diff]
overlay-support.patch
Comment 20 samLT 2006-11-12 04:19:15 UTC
I forgot to say: 

make -C tests/source/

fails (src.style), but it looks like it's due to some of your recent changes (most of the incorrect lines are comments)
Comment 21 samLT 2006-11-12 05:53:37 UTC
Created attachment 101730 [details, diff]
overlay-support.patch

a little improvement, so that the portroot is now considered
(and that should be enough for now)
Comment 22 samLT 2006-11-12 13:42:59 UTC
Created attachment 101779 [details, diff]
overlay-support.patch
Comment 23 Thomas Cort (RETIRED) gentoo-dev 2006-11-13 15:35:02 UTC
(In reply to comment #18)
> (for qcache: I haven't seen an overlay with metadata/cache yet

CVS log from 8 weeks, 2 days ago...

Re-wrote qcache to use /var/cache/edb/dep instead of
PORTDIR/metadata/cache....
Comment 24 samLT 2006-11-14 15:30:07 UTC
Created attachment 101954 [details, diff]
overlay-support.patch

(In reply to comment #23)
> (In reply to comment #18)
> > (for qcache: I haven't seen an overlay with metadata/cache yet
> 
> CVS log from 8 weeks, 2 days ago(...)

Oh well.. but it is portage which 'generates' /var/cache/edb/dep, So  I just have nothing to do in qcache.c right?

I haven't look in 'the new' qcache.c yet, but do I really need to?

small fix, that so that quse now correctly frees search_vars[idx] when idx=3 (option --format)
Comment 25 samLT 2006-11-20 16:19:50 UTC
Created attachment 102433 [details, diff]
FYI: quse.patch

There was something I've missed a lot, so I added it to quse, you'll tell me if you like, 

I wanted to be able to see the useflag (or whatever search_vars you want) for a particular package instead of every package (works only with the --all switch)

quse  -a rtage-ut
gentoo::app-portage/portage-utils/portage-utils-0.1.20.ebuild python 
gentoo::app-portage/portage-utils/portage-utils-0.1.21.ebuild python 
local::app-portage/portage-utils/portage-utils-20061122.ebuild python paludis

the diff for this particular change is is the quse.patch, but it is already included in the overlay-support.patch


I've also made a small change in quse.c, so that it now uses the different verbosity level.

And, the last change is again in quse.c when doing something like this:

quse -o local -a rtage-ut -vv
local::app-portage/portage-utils/portage-utils-20061122.ebuild python paludis 
 global:python: Adds support/bindings for the Python language
 (local) global:paludis: Add paludis(the other package mangler) support

(with verbose>1 and with the --overlay option ) the flag discription is search in every overlay and not just the one given as argument to the --overlay option
Comment 26 samLT 2006-11-20 16:20:56 UTC
Created attachment 102434 [details, diff]
overlay-support.patch
Comment 27 samLT 2006-12-27 11:56:53 UTC
Created attachment 104816 [details, diff]
overlay-manpages.patch

here is the patch for the man pages
Comment 28 samLT 2007-01-15 20:42:05 UTC
Created attachment 107119 [details, diff]
overlay-support.patch

hey, I'm just dropping the latest patch here in case someone wanted to look into it

portage-utils does almost no longer depend on the gentoo overlay, it's now a simple chain list as suggested.

If you know where to store .ebuild.x and .metadata.x instead of inside the gentoo tree, then portage-utils won't depend anymore on it.

many things changed, probably too many so that you want review/integrate it. anyway...
Comment 29 samLT 2007-01-15 20:42:57 UTC
Created attachment 107120 [details, diff]
portage2paludis.patch

and the paludis patch, just in case some one was looking for it on BGO.
Comment 30 samLT 2007-01-16 15:52:58 UTC
Created attachment 107183 [details, diff]
overlay-support.patch
Comment 31 samLT 2007-01-16 15:53:20 UTC
Created attachment 107184 [details, diff]
portage2paludis.patch
Comment 32 samLT 2007-01-19 15:18:21 UTC
Created attachment 107450 [details]
overlay-support.patch portage2paludis.patch and more..

again a new bump:
portage-utils now use the overlay/profiles/categories, if it exists

when searching the name ( ie not the DESCRIPTION )  it is now possible to make full use of the atom_explode thing:

eg: 
To list every package in the sys-apps category
qsearch sys-apps/
To list every package in the sys-apps category in the overlays the regexp match pal.*a.$  
qsearch sys-apps/::pal.*a.$

or even exlude an overlay while searching
qsearch sthg::[^gentoo]


and so on, a kind of changelog can be found here: http://fatimademe.free.fr/portage-utils/changelog
Comment 33 SpanKY gentoo-dev 2011-03-17 03:05:55 UTC
Comment on attachment 107450 [details]
overlay-support.patch portage2paludis.patch and more..

not interested in links to dead sites