Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 691482 - gemato: [Errno 2] No such file or directory: '/usr/portage/.tmp-unverified-download-quarantine/Manifest'
Summary: gemato: [Errno 2] No such file or directory: '/usr/portage/.tmp-unverified-do...
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-08-05 09:28 UTC by Hadrien Lacour
Modified: 2019-08-08 21:30 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hadrien Lacour 2019-08-05 09:28:35 UTC
Hello,

I just set a local portage tree mirror via rsyncd, but when syncing on the client, I get:

>$ sudo emerge --sync
>>>> Syncing repository 'gentoo' into '/usr/portage'...
> * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
> * Refreshing keys via WKD ...                                                                                                                        [ ok ]
>Do you want to sync your ebuild repository with the mirror at
>rsync://192.168.1.2/portage-tree? [Yes/No]
>>>> Starting rsync with rsync://192.168.1.2/portage-tree...
>>>> Checking server timestamp ...
>receiving incremental file list
>...
>...
>...
>sent 54.04K bytes  received 6.83M bytes  1.97M bytes/sec
>total size is 188.36M  speedup is 27.37
>Traceback (most recent call last):
>  File "/usr/lib64/python3.6/site-packages/portage/util/_async/AsyncFunction.py", line 39, in _run
>    result = self.target(*(self.args or []), **(self.kwargs or {}))
>  File "/usr/lib64/python3.6/site-packages/portage/sync/controller.py", line 169, in sync
>    taskmaster.run_tasks(tasks, func, status, options=task_opts)
>  File "/usr/lib64/python3.6/site-packages/portage/sync/controller.py", line 68, in run_tasks
>    result = getattr(inst, func)(**kwargs)
>  File "/usr/lib64/python3.6/site-packages/portage/sync/syncbase.py", line 311, in sync
>    return self.update()
>  File "/usr/lib64/python3.6/site-packages/portage/sync/modules/rsync/rsync.py", line 373, in update
>    max_jobs=self.verify_jobs)
>  File "/usr/lib64/python3.6/site-packages/gemato/recursiveloader.py", line 265, in __init__
>    allow_create=allow_create, store_dev=not allow_xdev)
>  File "/usr/lib64/python3.6/site-packages/gemato/recursiveloader.py", line 303, in load_manifest
>    raise err
>  File "/usr/lib64/python3.6/site-packages/gemato/recursiveloader.py", line 287, in load_manifest
>    relpath, verify_entry)
>  File "/usr/lib64/python3.6/site-packages/gemato/recursiveloader.py", line 58, in verify_and_load
>    path, 'r', encoding='utf8') as f:
>  File "/usr/lib64/python3.6/site-packages/gemato/compression.py", line 107, in open_potentially_compressed_path
>    return io.open(path, mode, **kwargs)
>FileNotFoundError: [Errno 2] No such file or directory: '/usr/portage/.tmp-unverified-download-quarantine/Manifest'
>
>Action: sync for repo: gentoo, returned code = 1

at the end, and the sync isn't applied. Here are the important config files:
Client make.conf
># These settings were set by the catalyst build script that automatically
># built this stage.
># Please consult /usr/share/portage/config/make.conf.example for a more
># detailed example.
>COMMON_FLAGS="-O3 -march=native -pipe"
>CFLAGS="${COMMON_FLAGS}"
>CXXFLAGS="${COMMON_FLAGS}"
>FCFLAGS="${COMMON_FLAGS}"
>FFLAGS="${COMMON_FLAGS}"
>
>MAKEOPTS="-j12 -l12"
>
>PORTDIR="/usr/portage"
>DISTDIR="/usr/portage/distfiles"
>PKGDIR="/usr/portage/packages"
>GENTOO_MIRRORS="rsync://workstation/portage-mirror ftp://ftp.free.fr/mirrors/ftp.gentoo.org/"
>FEATURES="network-sandbox news parallel-fetch parallel-install splitdebug userfetch userpriv usersandbox usersync"
>PORTAGE_BINHOST="rsync://workstation/portage-binhost"
>EMERGE_DEFAULT_OPTS="--getbinpkg --ask --verbose --tree --unordered-display --jobs=12 --load-average=12"
>PORTAGE_NICENESS="10"
>PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"
>
>USE="logrotate pgo savedconfig ssl threads udev zsh-completion -bindist -dbus -introspection -opengl -systemd -X"
>L10N="en fr"
>GRUB_PLATFORMS="pc"
>CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
>
># This sets the language of build output to English.
># Please keep this setting intact when reporting bugs.
>LC_MESSAGES=C

Client repos.conf/gentoo
>[DEFAULT]
>main-repo = gentoo
>
>[gentoo]
>location = /usr/portage
>sync-type = rsync
>sync-uri = rsync://workstation/portage-tree
>auto-sync = yes

Mirror (workstation) rsyncd.conf
># /etc/rsyncd.conf
>
># Minimal configuration file for rsync daemon
># See rsync(1) and rsyncd.conf(5) man pages for help
>
># This line is required by the /etc/init.d/rsyncd script
>pid file = /run/rsyncd.pid
>use chroot = yes
>read only = yes
>
>reverse lookup = no
>hosts allow = 192.168.1.0/24
>hosts deny = *
>
>[portage-tree]
>    path = /usr/portage/
>    comment = Portage ebuild tree
>    exclude = /distfiles /packages
>
>[portage-mirror]
>    path = /usr/portage/
>    comment = Portage distfiles
>    exclude = /*
>    include = /distfiles
>
>[portage-binhost]
>    path = /usr/portage/packages
>    comment = Portage binhost (Zen+)

Mirror repos.conf/gentoo
>[DEFAULT]
>main-repo = gentoo
>
>[gentoo]
>location = /usr/portage
>sync-type = git
>sync-uri = https://anongit.gentoo.org/git/repo/sync/gentoo.git
>clone-depth = 1
>sync-git-verify-commit-signature = yes
>auto-sync = yes


Any idea of what the problem is?
Comment 1 Zac Medico gentoo-dev 2019-08-05 17:49:04 UTC
Maybe you've excluded the Manifest file via PORTAGE_RSYNC_EXTRA_OPTS or PORTAGE_RSYNC_OPTS?

Check the output of this command:

> portageq envvar -v PORTAGE_RSYNC_EXTRA_OPTS PORTAGE_RSYNC_OPTS
Comment 2 Hadrien Lacour 2019-08-06 06:51:21 UTC
I get

PORTAGE_RSYNC_EXTRA_OPTS=
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"

on both systems. Looks normal to me. It looks like this directory isn't on any of the systems, including the server (which doesn't have problem syncing).
Comment 3 Zac Medico gentoo-dev 2019-08-07 17:55:42 UTC
Do you know what mirror your server last synced from? All mirrors should have the Manifest. Does the server have sync-rsync-verify-metamanifest disabled in /etc/portage/repos.conf?
Comment 4 Hadrien Lacour 2019-08-07 20:13:08 UTC
As written, I sync via https://anongit.gentoo.org/git/repo/sync/gentoo.git on the server.
Comment 5 Hadrien Lacour 2019-08-07 20:14:45 UTC
In the end, I simply disabled verif on the client. Doing the verification two times seems useless. Would still like to know why it doesn't work, though.
Comment 6 Zac Medico gentoo-dev 2019-08-07 20:21:36 UTC
For git, use sync-type = git and sync-git-verify-commit-signature as described here:

https://wiki.gentoo.org/wiki/Portage_Security#git-mirror_repositories
Comment 7 Hadrien Lacour 2019-08-08 07:10:28 UTC
I already use this, please look at the end of the first post where I put the server repos.conf.
Comment 8 Zac Medico gentoo-dev 2019-08-08 18:08:30 UTC
(In reply to Hadrien Lacour from comment #7)
> I already use this, please look at the end of the first post where I put the
> server repos.conf.

Since the Manifest file is not included in the sync/gentoo git tree, there's no way for the client to verify unless they client also uses sync-type = git.

Alternatively, on you server you could use gemato to generate a signed Manifest, and set sync-openpgp-key-path on your client to refer to your exported key.
Comment 9 Hadrien Lacour 2019-08-08 21:30:50 UTC
I see, it makes sense for git and rsync to use different verification methods. Thanks for the help