Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 436918 - egencache subcommand needed to do thin manifest -> thick manifest conversion
Summary: egencache subcommand needed to do thin manifest -> thick manifest conversion
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 333701 431026
  Show dependency tree
 
Reported: 2012-10-02 04:12 UTC by Brian Harring (RETIRED)
Modified: 2012-10-17 04:36 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Harring (RETIRED) gentoo-dev 2012-10-02 04:12:28 UTC
As it sounds; thin manifest (used for git) doesn't need checksums for AUX/MISC/EBUILD- they're implicitly provided by git.

Thick/Full/traditional manifests that are used in cvs, and more importantly, rsync, however require those checksums.

We need a conversion subcommand that takes existing thin manifest, and regenerates a thick manifest using the given DIST data.

This must also be able to use the existing signing pathways, since ultimately infra's usage of this will involve signing it.
Comment 1 Zac Medico gentoo-dev 2012-10-02 05:10:42 UTC
Will fold this into egencache, since it already does the metadata and use.local.desc generation.

Is our tool supposed to internally disregard "thin-manifests = true" in metadata/layout.conf, or are we going to leave it to the caller to adjust the layout.conf settings appropriately with sed or something?
Comment 2 Zac Medico gentoo-dev 2012-10-02 06:34:27 UTC
As discussed on irc, the user (infra) needs to merge changes into the tree such that unmodified thick manifests from the previous run are preserved with their timestamps and signatures. This way, egencache only has to generate/sign manifests for the manifests that have changed since the previous run, and the preserved timestamps of the unmodified manifests will serve to avoid unnecessary rsync transfers.
Comment 3 Brian Harring (RETIRED) gentoo-dev 2012-10-02 09:30:03 UTC
worst case, a git merge -X theirs <ToT> # will reset the conflicting manifests back to what they've got.

Downside, is if our machinery adds a Manifest to a pathway, but there is no content in that pathway, the delete won't fire.
Comment 4 Zac Medico gentoo-dev 2012-10-03 09:39:03 UTC
This is fixed in git:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=15a799b52155a47568f4b049ff8487a2718b270c

egencache: add --update-manifests

Update manifest files, and sign them if signing is enabled. This
supports parallelization if enabled via the --jobs option. The
--thin-manifests and --sign-manifests options may be used to manually
override layout.conf settings. There's also a new --strict-manifests
option that may be used to manually override the "strict" FEATURES
setting, a --gpg-key option to override PORTAGE_GPG_KEY, and a
--gpg-dir option to override PORTAGE_GPG_DIR.
Comment 5 Zac Medico gentoo-dev 2012-10-04 00:13:49 UTC
This is fixed in 2.1.11.24 and 2.2.0_alpha135.
Comment 6 Zac Medico gentoo-dev 2012-10-16 06:49:30 UTC
Here's a fix to make it re-sign Manifests with the current key when necessary:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=84ea673c632dea0bc9af61c371f71ca86cc19e1a
Comment 7 Zac Medico gentoo-dev 2012-10-17 00:05:26 UTC
(In reply to comment #6)
> Here's a fix to make it re-sign Manifests with the current key when
> necessary:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=84ea673c632dea0bc9af61c371f71ca86cc19e1a

This is release in 2.1.11.28 and 2.2.0_alpha139.
Comment 8 Zac Medico gentoo-dev 2012-10-17 02:04:08 UTC
Here's a fix for key id parsing. Without this fix, failure to parse the
key results in manifest being re-signed even though it already has
a signature with the correct key.

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=8a28c000ce225b79076e36f212162f58e01188b7
Comment 9 Zac Medico gentoo-dev 2012-10-17 04:36:04 UTC
(In reply to comment #8)
> Here's a fix for key id parsing. Without this fix, failure to parse the
> key results in manifest being re-signed even though it already has
> a signature with the correct key.
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=8a28c000ce225b79076e36f212162f58e01188b7

This is in 2.1.11.29 and 2.2.0_alpha140.