| Summary: | sys-apps/portage-3.0.47: emerge-webrsync fails to sync if gemato is not installed | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Pacho Ramos <pacho> |
| Component: | Core | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | CC: | sam |
| Priority: | Normal | Keywords: | InVCS |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=905358 https://github.com/gentoo/portage/pull/1039 https://github.com/gentoo/portage/pull/1042 |
||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | 905355 | ||
| Bug Blocks: | 905356 | ||
|
Description
Pacho Ramos
2023-05-07 10:29:30 UTC
Would you mind quickly trying portage-9999? It looks to have the same issue (now is for example trying to download the snapshot from the 3rd May and goes on) :/ (In reply to Pacho Ramos from comment #2) > It looks to have the same issue (now is for example trying to download the > snapshot from the 3rd May and goes on) :/ no worries, thanks, I just wanted to check because I'd changed a lot in git too. I'll check this out today (or at worst, tomorrow). Thank you for spotting this now, I was planning on cutting a new release shortly! The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=66c00b2e3d72bc8947fc802b0403687853e16e13 commit 66c00b2e3d72bc8947fc802b0403687853e16e13 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-05-17 06:18:25 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-05-17 06:20:16 +0000 emerge-webrsync: add fallback error case This would've helped avoid a loop where we keep trying old snapshots if gemato wasn't installed. We already have a fix for that separately with a more specific error, but a fallback is good for unexpected ones. Bug: https://bugs.gentoo.org/905868 Closes: https://github.com/gentoo/portage/pull/1039 Signed-off-by: Sam James <sam@gentoo.org> bin/emerge-webrsync | 5 +++++ 1 file changed, 5 insertions(+) https://gitweb.gentoo.org/proj/portage.git/commit/?id=b444a4baa113dcf9f779fa68b056b8ac5e9ea5ea commit b444a4baa113dcf9f779fa68b056b8ac5e9ea5ea Author: Sam James <sam@gentoo.org> AuthorDate: 2023-05-17 06:12:32 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-05-17 06:20:12 +0000 emerge-webrsync: fall back correctly to manual gpg if no gemato Bug: https://bugs.gentoo.org/905868 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 3 +++ bin/emerge-webrsync | 9 +++++++++ 2 files changed, 12 insertions(+) Thanks Pacho. I could reproduce it but I'd appreciate it if you could verify portage-9999 is OK before I cut a release? Thanks! Thanks I hit two problems: * PGP verification method: gemato * Fetching most recent snapshot ... /usr/bin/emerge-webrsync: line 577: [[: 08: value too great for base (error token is "08") * Trying to retrieve 20230517 snapshot from http://gentoo.mirrors.ovh.net/gentoo-distfiles ... And, later, after downloading the right file: * Checking digest ... * Checking signature ... * Falling back to gpg as gemato is not installed gpg: WARNING: unsafe ownership on homedir '/home/pacho/.gnupg' gpg: Signature made Thu May 18 02:56:47 2023 CEST gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 gpg: Can't check signature: No public key * ERROR: /:: failed: * signature verification failed * * If you need support, post the output of `emerge --info '=/::'`, * the complete build log and the output of `emerge -pqv '=/::'`. * Working directory: '/var/tmp/portage/webrsync-mZMbAr' It tries to use my gnupg because I have PORTAGE_GPG_DIR and PORTAGE_GPG_KEY in my make.conf. If I drop both (as most users will have) I have this error: * Checking digest ... * Checking signature ... * Falling back to gpg as gemato is not installed gpg: directory '/root/.gnupg' created gpg: keybox '/root/.gnupg/pubring.kbx' created gpg: Signature made Thu May 18 02:56:47 2023 CEST gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 gpg: Can't check signature: No public key * ERROR: /:: failed: * signature verification failed * * If you need support, post the output of `emerge --info '=/::'`, * the complete build log and the output of `emerge -pqv '=/::'`. * Working directory: '/var/tmp/portage/webrsync-8JPMhr' Thanks for your help thanks, I'll take a look tonight! (In reply to Pacho Ramos from comment #6) > Thanks > > I hit two problems: > * PGP verification method: gemato > * Fetching most recent snapshot ... > /usr/bin/emerge-webrsync: line 577: [[: 08: value too great for base (error > token is "08") > * Trying to retrieve 20230517 snapshot from > http://gentoo.mirrors.ovh.net/gentoo-distfiles ... existing_timestamp=$(get_repository_timestamp) start_time=$(get_utc_date_in_seconds) start_hour=$(get_date_part "${start_time}" "%H") # Daily snapshots are created at 00:45 and are not # available until after 01:00. Don't waste time trying # to fetch a snapshot before it's been created. if [[ ${start_hour} -lt 1 ]] ; then I'm guessing that get_date_part is being affected by locale or similar. If you prefix the date command in get_date_part with LC_ALL=C, does it help? > > And, later, after downloading the right file: > * Checking digest ... > * Checking signature ... > * Falling back to gpg as gemato is not installed > gpg: WARNING: unsafe ownership on homedir '/home/pacho/.gnupg' > gpg: Signature made Thu May 18 02:56:47 2023 CEST > gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 > gpg: Can't check signature: No public key > * ERROR: /:: failed: > * signature verification failed > * looking at the other issue now ignore my earlier question :) Can you try https://github.com/gentoo/portage/pull/1042 please? (on top of 9999) You can get it as a patch by appending .patch to the URL The problem with the hours if fixed, thanks! But the problem with the fallback remain. With PORTAGE_GPG_DIR being set I get: * Falling back to gpg as gemato is not installed gpg: WARNING: unsafe ownership on homedir '/home/pacho/.gnupg' gpg: Signature made Sat May 20 02:56:46 2023 CEST gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 gpg: Can't check signature: No public key I guess the error that makes it die is gpg: Can't check signature: No public key Without it being set I get a variant of the same (missing public key) problem: * Falling back to gpg as gemato is not installed gpg: Signature made Sat May 20 02:56:46 2023 CEST gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 gpg: Can't check signature: No public key (In reply to Pacho Ramos from comment #10) > The problem with the hours if fixed, thanks! > > But the problem with the fallback remain. With PORTAGE_GPG_DIR being set I > get: > * Falling back to gpg as gemato is not installed > gpg: WARNING: unsafe ownership on homedir '/home/pacho/.gnupg' > gpg: Signature made Sat May 20 02:56:46 2023 CEST > gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 > gpg: Can't check signature: No public key > I think this part is correct behaviour. If you're setting PORTAGE_GPG_DIR, you're telling it to use that keyring. And that keyring apparently doesn't have it imported. > > Without it being set I get a variant of the same (missing public key) > problem: > * Falling back to gpg as gemato is not installed > gpg: Signature made Sat May 20 02:56:46 2023 CEST > gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 > gpg: Can't check signature: No public key This part is interesting and I think I need to add an import or something. I don't get why I can't hit this part yet... (In reply to Sam James from comment #11) [...] > I think this part is correct behaviour. If you're setting PORTAGE_GPG_DIR, > you're telling it to use that keyring. And that keyring apparently doesn't > have it imported. > I don't remember when I added those lines... I think they were needed for pushing to the tree... but maybe in repoman times and I can simply drop it Could you try the PR again? The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=b8ab8e1c850b773dd17e503a22902b52a2d3a868 commit b8ab8e1c850b773dd17e503a22902b52a2d3a868 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-05-20 08:13:29 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-05-26 01:33:03 +0000 emerge-webrsync: create a new temporary dir for legacy gpg verification It's possible that we can't read /root/.gnupg and we shouldn't be poking around in there anyway. However, if the user is setting PORTAGE_GPG_DIR by themselves, it's their responsibility to handle the directory being in the right state (e.g. has the right keys imported). - If PORTAGE_GPG_DIR is unset, make a tmpdir w/ mktemp. - If we're using that temporary directory we just created, import PORTAGE_GPG_KEY, as before defaulting to /usr/share/openpgp-keys/gentoo-release.asc. Bug: https://bugs.gentoo.org/905868 Signed-off-by: Sam James <sam@gentoo.org> Closes: https://github.com/gentoo/portage/pull/1042 Signed-off-by: Sam James <sam@gentoo.org> bin/emerge-webrsync | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) https://gitweb.gentoo.org/proj/portage.git/commit/?id=2eef717c4b630f359235f2801fafdc9e63c546fb commit 2eef717c4b630f359235f2801fafdc9e63c546fb Author: Sam James <sam@gentoo.org> AuthorDate: 2023-05-20 08:17:38 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-05-26 01:33:03 +0000 emerge-webrsync: handle early hours correctly One of the perils of only doing development late... We would error out on '08' etc as the hour. Strip the 0. Bug: https://bugs.gentoo.org/905868 Signed-off-by: Sam James <sam@gentoo.org> bin/emerge-webrsync | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=08be91eebdbff0de0e033efe30c633219a9859ca commit 08be91eebdbff0de0e033efe30c633219a9859ca Author: Sam James <sam@gentoo.org> AuthorDate: 2023-06-01 01:22:47 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-01 01:23:18 +0000 sys-apps/portage: add 3.0.48 Closes: https://bugs.gentoo.org/722270 Closes: https://bugs.gentoo.org/879687 Closes: https://bugs.gentoo.org/898232 Closes: https://bugs.gentoo.org/898366 Closes: https://bugs.gentoo.org/905355 Closes: https://bugs.gentoo.org/905358 Closes: https://bugs.gentoo.org/905868 Closes: https://bugs.gentoo.org/906129 Closes: https://bugs.gentoo.org/906156 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.48.ebuild | 296 +++++++++++++++++++++++++++++++++ 2 files changed, 297 insertions(+) Sorry for the delay, I couldn't test on my computer But it still fails... in a different way: * Checking digest ... * Checking signature ... * Falling back to gpg as gemato is not installed gpg: keybox '/var/tmp/portage/webrsync-jUG5k9/pubring.kbx' created gpg: can't open '//usr/share/openpgp-keys/gentoo-release.asc': No such file or directory gpg: Total number processed: 0 gpg: Signature made Wed Jun 7 02:56:30 2023 CEST gpg: using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250 gpg: Can't check signature: No public key * ERROR: /:: failed: * signature verification failed * * If you need support, post the output of `emerge --info '=/::'`, * the complete build log and the output of `emerge -pqv '=/::'`. * Working directory: '/var/tmp/portage/webrsync-uAKw3D' I guess you need to pull in sec-keys/openpgp-keys-gentoo-release unconditionally Other option is to change the logic of the "rsync-verify" USE, I would change it to a more general "sync-verify" and, when disabled, emerge-webrsync should behave as running it with --no-pgp-verify Thanks a lot That's going to be quite brittle (it'd involve essentially sedding in the ebuild). Instead, let's just unconditionally depend on sec-keys/openpgp-keys-gentoo-release. It doesn't cost anything anyway. The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=32964d0c92402bd84b164852ca2a408f01211020 commit 32964d0c92402bd84b164852ca2a408f01211020 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-06-09 12:57:21 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-09 12:58:17 +0000 sys-apps/portage: unconditionally depend on sec-keys/openpgp-keys-gentoo-release It's useful for people to have it installed and this keeps emerge-webrsync working even with USE=-rsync-verify. The keys are tiny and have no dependencies themselves, so I don't see the value in trying to mangle the script with sed to default to --no-pgp-verify or similar. (It'd be different if we had a proper build system which would let us do it. Maybe.) Closes: https://bugs.gentoo.org/905868 Signed-off-by: Sam James <sam@gentoo.org> .../portage/{portage-3.0.48.1.ebuild => portage-3.0.48.1-r1.ebuild} | 6 +++--- sys-apps/portage/portage-9999.ebuild | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) Let's hope that's it ;) Thanks for your continued testing! It's a tricky topic and I hope we've got there in the end. (In reply to Sam James from comment #19) > Let's hope that's it ;) > > Thanks for your continued testing! It's a tricky topic and I hope we've got > there in the end. I'll also add a friendlier error message. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=12164035655e5cea4f83f9955bdb4db3369af7e3 commit 12164035655e5cea4f83f9955bdb4db3369af7e3 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-06-09 13:03:44 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-06-09 13:03:44 +0000 emerge-webrsync: improve error message when key is missing Bug: https://bugs.gentoo.org/905868 Signed-off-by: Sam James <sam@gentoo.org> NEWS | 3 +++ bin/emerge-webrsync | 10 ++++++++++ 2 files changed, 13 insertions(+) It works fine, thanks! |
