Bug 395359 – Document metadata/layout.conf in detail (was: Devmanual should document the metadata/layout.conf file format in detail)

• Product: Documentation
• Status: RESOLVED FIXED

[Description Kent Fredric (IRC: kent\n) (RETIRED) 2011-12-20 12:11:55 UTC]

At present, the only documentation I can find to layout.conf and repos.conf are in man 5 portage, and *both* of these are insufficient in detail or quality to really do anything with without also needing to look into the guts of portage.

Example:

# eclasses provided by java-overlay take precedence over identically named
# eclasses that are provided by gentoo
masters = gentoo java-overlay


Sure, that's nice, but there are many questions I have, none of which are explained in portage(5), and are totally missing from the devmanual, which considering the power the leverage over a repository, I would imagine the devmanual would be an important place for it. 

Such as: which repository is that example snippet from? Is that from the repository "java-overlay" itself? Or some other overlay, "java-experiemental" ?

If this is for the "java-overlay" itself, then it follows that its saying "yes, its ok to name your own repository in this file, and doing so tells portage/* that the eclasses shipped in that repository supercede the ones in gentoo".

OR, 

Is it for "java-experimental", and is it not sensible to list oneself in this file like that, and is there some implicit behaviour given by simply stating "master = gentoo" that isn't identical to what you had *before* you added the layout.conf file. 

( Reading the layout.conf from sunrise tells me that "master = gentoo" is indeed the default behaviour, and the sunrise layout.conf is just stating that to be explicit, but I shouldn't have to dig into random repos to see how it works, it should be documented. )



And this: Confuses me senseless:

"Site-specific overrides to layout.conf settings may be specified in /etc/portage/repos.conf. Settings in repos.conf take precedence over settings in layout.conf,"

^^^

Because there are way more settings in use in layout.conf than there are in the same documents repos.conf,  and I'm assuming that, given this statement, all fields that are valid in layout.conf are *also* valid in repos.conf, 

Just ... they're not documented in repos.conf 's documentation either.


Yes, portage documentation itself could be better, but I feel for files like layout.conf which are not intrinsically portage specific, they should be in the devmanual. 

As it is, I didn't find the documentation in portage by searching for it in portage, I searched the *internet* which turned me up at zmedico's html'ised copies of the portage man-pages.

[Comment 1 Markos Chandras (RETIRED) 2012-11-02 17:47:36 UTC]

Care to submit a patch? :)

[Comment 2 Kent Fredric (IRC: kent\n) (RETIRED) 2012-11-03 16:35:30 UTC]

(In reply to comment #1)
> Care to submit a patch? :)

Well, for me to do that, I'd have to first know 

a) a valid list of settings for layout.conf  ( Which, as I stated earlier, I can't divine )

b) what the valid values are for each of those settings ( Which is the same problem "I don't know" )

I can draw assumptions from the information in man 5 portage, but those are likely wrong assumptions,  and I can't *know* without reading the source to portage itself ( Which I'm not about to do )

'man 5 portage' doesn't even document these values really, it just has an "example" , which I can only /assume/ contains all the valid values, and ONLY valid values. 

Essentially, for me to provide a patch, I would literally be transcribing the contents of man 5 portage, which as I stated earlier, are inadequate. 

Either that, or I would have to read the source of portage itself ( which, I am not about to do ).



To exacerbate issues, isn't "layout.conf" a PMS agnostic file, and "repos.conf" a PMS specific file? 

And if "layout.conf" is an Inter-PMS configuration format, why is it completely undocumented, bar scant information in only *one* package manager?

[Comment 3 Kent Fredric (IRC: kent\n) (RETIRED) 2012-11-03 16:37:17 UTC]

I think the simplest I can put it: 

You're suggesting I, who can't find documentation about X , and has no idea how X works, document X, despite not having any clue about X .

Forgive me if I find that a bit of a tall order =)

[Comment 4 Markos Chandras (RETIRED) 2014-01-05 00:29:10 UTC]

I am not sure about the layout.conf format myself either. CC'ing portage in case they are interested to submit a patch for that.

[Comment 5 SpanKY 2014-01-05 13:48:01 UTC]

we document all related files in the portage(5) man page

[Comment 6 Kent Fredric (IRC: kent\n) (RETIRED) 2014-01-05 14:05:12 UTC]

> I can draw assumptions from the information in man 5 portage, but those are
> likely wrong assumptions,  and I can't *know* without reading the source to
> portage itself ( Which I'm not about to do )
> 
> 'man 5 portage' doesn't even document these values really, it just has an
> "example" , which I can only /assume/ contains all the valid values, and
> ONLY valid values. 

Granted, in the 3 years passed since I opened this bug, the listing has become more detailed, but its still with questions.

For instance:


> # indicate that this repo requires manifests for each package, and is
> # considered a failure if a manifest file is missing/incorrect
> use-manifests = strict

What are the valid values for this field? "strict" seems to be one, but its arbitrariness suggest there are others.

> # customize the set of hashes generated for Manifest entries
> manifest-hashes = SHA256 SHA512 WHIRLPOOL

This too is obviously a subset of options, where do I look to find the complete list ( ie: is it something that is a function of some libraries feature set, or is it some python-specific thing, etc )

> # indicate that this repo contains both md5-dict and pms cache formats,
> # which may be generated by egencache(1)
> cache-formats = md5-dict pms

Is this a complete list of formats? I look at egencache and it lists only those two, so I assume so, but I also notice `pms` is considered obsolete, so should be removed from the example.

Perhaps information to what the default behaviour is when a field is not explicitly stated would be helpful.

[Comment 7 Sebastian Luther (few) 2014-01-05 14:31:52 UTC]

If anywhere else, layout.conf should be documented in PMS, otherwise it's just a duplicate that inevitably runs out of sync.

You're raising valid points what concerns the man page in comment 6.

[Comment 8 SpanKY 2014-01-05 15:42:30 UTC]

(In reply to Sebastian Luther (few) from comment #7)

erm, no, layout.conf has nothing to do with PMS.  there is no duplicate information here as only portage utilizes it.

[Comment 9 Sebastian Luther (few) 2014-01-05 17:01:13 UTC]

(In reply to SpanKY from comment #8)
> (In reply to Sebastian Luther (few) from comment #7)
> 
> erm, no, layout.conf has nothing to do with PMS.  there is no duplicate
> information here as only portage utilizes it.

That's not correct. Both other PMs use it [1] [2]. A PM cannot really go without layout.conf these days.


[1] search for layout.conf in https://code.google.com/p/pkgcore/source/browse/NEWS

[2] search for layout.conf in http://paludis.exherbo.org/configuration/repositories/e.html
and here it meantions the masters key:
http://paludis.exherbo.org/configuration/repositories/repository.html

[Comment 10 SpanKY 2014-01-05 19:11:34 UTC]

(In reply to Kent Fredric from comment #6)

i've cleaned up the existing documentation in git now

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;h=77c4a5b31fb5b42e1e4969bf1747cc5e416eca2f

(In reply to Sebastian Luther (few) from comment #9)

its adoption by other PM's is irrelevant as long as it's not in the PMS.  file another bug if you want PMS to pick up the layout.conf format.

[Comment 11 Markos Chandras (RETIRED) 2014-01-05 19:42:34 UTC]

(In reply to SpanKY from comment #10)
> (In reply to Kent Fredric from comment #6)
> 
> i've cleaned up the existing documentation in git now
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;
> h=77c4a5b31fb5b42e1e4969bf1747cc5e416eca2f
> 
> (In reply to Sebastian Luther (few) from comment #9)
> 
> its adoption by other PM's is irrelevant as long as it's not in the PMS. 
> file another bug if you want PMS to pick up the layout.conf format.


Thanks. Having said that, I am not sure if layout.conf needs to be in the devmanual. It's not a file needed specifically for ebuild or eclass development which the devmanual mainly deals with. Users and/or developers who maintain the overlay can get the information from the portage man pages. I am inclined to close this as WONTFIX from the devmanual's point of view.

[Comment 12 Sebastian Luther (few) 2014-01-05 20:42:18 UTC]

(In reply to SpanKY from comment #10)
> (In reply to Sebastian Luther (few) from comment #9)
> 
> its adoption by other PM's is irrelevant as long as it's not in the PMS. 
> file another bug if you want PMS to pick up the layout.conf format.

Well, the adoption in the repositories is relevant, because some of the keys render the repo useless if the PM doesn't understand them.

I didn't mean to make this part of this bug, though.


(In reply to Markos Chandras from comment #11)
> [...]
> I am inclined to close this as WONTFIX from the devmanual's point of view.

This is now dev-portage's bug.

[Comment 13 Markos Chandras (RETIRED) 2014-01-05 21:08:56 UTC]

(In reply to Sebastian Luther (few) from comment #12)
> 
> (In reply to Markos Chandras from comment #11)
> > [...]
> > I am inclined to close this as WONTFIX from the devmanual's point of view.
> 
> This is now dev-portage's bug.

OK fine by me. So there is nothing left for us to do here. Portage documentation does not belong to devmanual but rather in manpages

[Comment 14 Brian Dolbec (RETIRED) 2014-01-06 00:45:27 UTC]

Released in sys-apps/portage-2.2.8.