Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 723966 - portage without use flag rsync-verify always runs verify after rsync
Summary: portage without use flag rsync-verify always runs verify after rsync
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Repoman (show other bugs)
Hardware: AMD64 Linux
: Normal major (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-19 11:05 UTC by hsn
Modified: 2020-11-14 15:54 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description hsn 2020-05-19 11:05:41 UTC
emaint sync does rsync verify when it should not.

Reproducible: Always

Steps to Reproduce:
hsn@gentoo ~ $ equery u portage
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-apps/portage-2.3.99-r2:
 U I
 - - apidoc                   : Build html API documentation with
                                sphinx-apidoc.
 - - build                    : !!internal use only!! DO NOT SET THIS FLAG
                                YOURSELF!, used for creating build images and
                                the first half of bootstrapping [make stage1]
 + + doc                      : Add extra documentation (API, Javadoc, etc). It
                                is recommended to enable per package instead of
                                globally
 + + gentoo-dev               : Enable features required for Gentoo ebuild
                                development.
 + + native-extensions        : Compiles native "C" extensions (speedups,
                                instead of using python backup code). Currently
                                includes libc-locales. This should only be
                                temporarily disabled for some bootstrapping
                                operations. Cross-compilation is not supported.
 - - python_targets_python3_6 : Build with Python 3.6
 + + python_targets_python3_7 : Build with Python 3.7
 - - rsync-verify             : Enable full-tree cryptographic verification of
                                Gentoo repository rsync checkouts using
                                app-portage/gemato.
 + + xattr                    : Preserve extended attributes (filesystem-stored
                                metadata) when installing files. Usually only
                                required for hardened systems.

hsn@gentoo ~ $ emaint sync -A

Actual Results:  
* Verifying /usr/portage/.tmp-unverified-download-quarantine ...

Expected Results:  
quick operation, no verify shit

software should not spend 45 minutes with meta data verifications.
Comment 1 hsn 2020-05-19 11:58:08 UTC
After close system inspection with help provided by iamben it was found that in /etc/portage/repos.conf there is sync-rsync-verify-metamanifest = yes in repo configuration.
Comment 2 Brian Dolbec gentoo-dev 2020-05-19 13:36:01 UTC
Emaint sync and emerge sync are essentially the same tool (almost) there is only one sync code, just 2 tools to run it.  

So, what you are wanting is the ability for emaint to have a cli option to NOT rsync-verify a repo that is otherwise set to verify in repos.conf?

Or have you found out how to turn off that feature, so you are now good?  If so, please close this bug
Comment 3 Zac Medico gentoo-dev 2020-05-19 17:00:53 UTC
(In reply to hsn from comment #1)
> After close system inspection with help provided by iamben it was found that
> in /etc/portage/repos.conf there is sync-rsync-verify-metamanifest = yes in
> repo configuration.

This behavior is intended. You can set "sync-rsync-verify-metamanifest = no" if you want to disable it.

The rsync-verify USE flag serves to control the default setting found in /usr/share/portage/config/make.globals, and also ensures the the required dependencies (gemato and gnupg) are installed.
Comment 4 hsn 2020-05-19 17:19:54 UTC
> So, what you are wanting is the ability for emaint to have a cli option to NOT rsync-verify a repo that is otherwise set to verify in repos.conf?

No, i want that if there is no rsync-verify use flag compiled in then software will return hard error when it sees repo with verification wanted because as you said it might or might not work due to missing dependencies that are pulled in only when there is +rsync-verify use flag.
Comment 5 Zac Medico gentoo-dev 2020-05-19 17:37:35 UTC
(In reply to hsn from comment #4)
> > So, what you are wanting is the ability for emaint to have a cli option to NOT rsync-verify a repo that is otherwise set to verify in repos.conf?
> 
> No, i want that if there is no rsync-verify use flag compiled in then
> software will return hard error when it sees repo with verification wanted
> because as you said it might or might not work due to missing dependencies
> that are pulled in only when there is +rsync-verify use flag.

It *will* fail hard if sync-rsync-verify-metamanifest is enabled and the dependencies are not present.

The intention of the rsync-verify is not to prevent rsync verification. It's only to enable it.
Comment 6 Walter Dnes 2020-09-06 01:31:57 UTC
If you're not going to change the new behaviour, it at least warrants a news item. Not only is the new behaviour counter-intuitive, but a config that used to work in the past is now brokem. That goes against "the principle of least surprise". Let people know that emerging portage with USE="-rsync-verify" also requires setting

sync-rsync-verify-metamanifest = no

in /etc/portage/repos.conf/gentoo.conf

  I assume that you have to set

sync-rsync-verify-metamanifest = yes

in /etc/portage/repos.conf/gentoo.conf *IN ADDITION TO* emerging portage with USE="-rsync-verify" when you want verification.

I can see this being useful in a scenario where you have multiple repos where you want some verified, and others not verified. But still, let people know about this change in default behaviour.

Another option... hard-code "rsync-verify" into the ebuild, so that it's not physically possible to come up with a non-working ebuild. Even in that case, a news item should be sent out letting people know that verification is now controlled in /etc/portage/repos.conf/gentoo.conf rather than by a portage USE flag.
Comment 7 Walter Dnes 2020-09-06 01:36:25 UTC
Edit:

in /etc/portage/repos.conf/gentoo.conf *IN ADDITION TO* emerging portage with USE="rsync-verify" when you want verification.

That should be USE="rsync-verify".
Comment 8 Zac Medico gentoo-dev 2020-09-06 17:50:41 UTC
(In reply to Walter Dnes from comment #7)
> If you're not going to change the new behaviour, it at least warrants a news
> item. Not only is the new behaviour counter-intuitive, but a config that
> used to work in the past is now brokem. That goes against "the principle of
> least surprise". Let people know that emerging portage with
> USE="-rsync-verify" also requires setting
> 
> sync-rsync-verify-metamanifest = no
> 
> in /etc/portage/repos.conf/gentoo.conf
> 
>   I assume that you have to set
> 
> sync-rsync-verify-metamanifest = yes
> 
> in /etc/portage/repos.conf/gentoo.conf *IN ADDITION TO* emerging portage
> with USE="rsync-verify" when you want verification.

That's not true, because sync-rsync-verify-metamanifest is normally set to yes by default. It only defaults to no when you disable the rsync-verify USE flag, via this code in the ebuild:


	if use build || ! use rsync-verify; then
		sed -e '/^sync-rsync-verify-metamanifest/s|yes|no|' \
			-e '/^sync-webrsync-verify-signature/s|yes|no|' \
			-i cnf/repos.conf || die "sed failed"
	fi

> I can see this being useful in a scenario where you have multiple repos
> where you want some verified, and others not verified. But still, let people
> know about this change in default behaviour.
> 
> Another option... hard-code "rsync-verify" into the ebuild, so that it's not
> physically possible to come up with a non-working ebuild. Even in that case,
> a news item should be sent out letting people know that verification is now
> controlled in /etc/portage/repos.conf/gentoo.conf rather than by a portage
> USE flag.

With USE=rsync-verify enabled, verification is enabled by default, and the user does not need to intervene in any way.
Comment 9 Walter Dnes 2020-11-14 12:02:39 UTC
I did a fresh install recently on an older machine, and ran into this again, having forgotten about it. A Google search turned up news item https://www.gentoo.org/support/news-items/2018-01-30-portage-rsync-verification.html which says...

> If you wish to disable it, you can disable the 'rsync-verify' USE flag
> on sys-apps/portage ***OR*** set 'sync-rsync-verify-metamanifest = no'
> in your repos.conf.

That's my emphasis. It wasn't until I checked bugzilla that I remembered. I'm not asking for a change in the way "emerge" works; I'm asking that the documentation be updated. Replacing the "or" with an "and" should be sufficient.
Comment 10 Mike Gilbert gentoo-dev 2020-11-14 15:54:49 UTC
(In reply to Walter Dnes from comment #9)
> That's my emphasis. It wasn't until I checked bugzilla that I remembered.
> I'm not asking for a change in the way "emerge" works; I'm asking that the
> documentation be updated. Replacing the "or" with an "and" should be
> sufficient.

But "and" isn't correct: it is not necessary to both disable the USE flag and set the sync-rsync-verify-metamanifest setting in /etc/portage/repos.conf.