Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 562264 - sys-apps/portage-2.2.22: First post-sync repo is synced at the same time as the cache is being updated
Summary: sys-apps/portage-2.2.22: First post-sync repo is synced at the same time as t...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2015-10-05 01:37 UTC by Nikos Chantziaras
Modified: 2016-03-14 02:56 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nikos Chantziaras 2015-10-05 01:37:13 UTC
When doing an "emerge --sync", the first layman repo is synced at the same time as the portage cache is being updated. You can see how the cache update progress bar is split as the first repo (vmware) gets synced.

This only affects the first repo. All remaining repos are synced only after the cache update completes.


...
sent 143.52K bytes  received 9.89M bytes  211.29K bytes/sec
total size is 334.03M  speedup is 33.28
=== Sync completed for gentoo

>>> Updating Portage cache
>>> Syncing repository 'vmware' into '/var/lib/layman/vmware'...
>>> Starting layman sync for vmware...
 * Running Git... # ( cd /var/lib/layman/vmware  && /usr/bin/git pull )
  0% [>                                                                       ]Already up-to-date.
 * 
 * Succeeded:
 * ------
 * Successfully synchronized overlay "vmware".
 * 
>>> layman sync succeeded: vmware
>>> laymansync sez... "Hasta la sync ya, baby!"
=== Sync completed for vmware
100% [=======================================================================>]
>>> Syncing repository 'steam-overlay' into '/var/lib/layman/steam-overlay'...
>>> Starting layman sync for steam-overlay...
 * Running Git... # ( cd /var/lib/layman/steam-overlay  && /usr/bin/git pull )
Already up-to-date.
 * 
 * Succeeded:
 * ------
 * Successfully synchronized overlay "steam-overlay".
 * 
>>> layman sync succeeded: steam-overlay
>>> laymansync sez... "Hasta la sync ya, baby!"
=== Sync completed for steam-overlay
>>> Syncing repository 'interactive-fiction' into '/var/lib/layman/interactive-fiction'...
>>> Starting layman sync for interactive-fiction...
 * Running Git... # ( cd /var/lib/layman/interactive-fiction  && /usr/bin/git pull )
...
Comment 1 Nikos Chantziaras 2015-10-05 01:38:03 UTC
$ emerge --info portage

Portage 2.2.22 (python 3.4.3-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-4.9.3, glibc-2.21-r1, 4.1.9-gentoo-r1 x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.1.9-gentoo-r1-x86_64-Intel-R-_Core-TM-_i5-2500K_CPU_@_3.30GHz-with-gentoo-2.2
KiB Mem:    16385360 total,  12972912 free
KiB Swap:   25166912 total,  25166912 free
Timestamp of repository gentoo: Mon, 05 Oct 2015 00:45:01 +0000
sh bash 4.3_p42
ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1
app-shells/bash:          4.3_p42::gentoo
dev-lang/perl:            5.22.0::gentoo
dev-lang/python:          2.7.10::gentoo, 3.3.5-r2::gentoo, 3.4.3::gentoo
dev-util/cmake:           3.3.2::gentoo
dev-util/pkgconfig:       0.29::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.17::gentoo
sys-apps/sandbox:         2.9::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r1::gentoo
sys-devel/automake:       1.11.6-r1::gentoo, 1.14.1::gentoo, 1.15::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            4.9.3::gentoo
sys-devel/gcc-config:     1.8::gentoo
sys-devel/libtool:        2.4.6-r1::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.21-r1::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.europe.gentoo.org/gentoo-portage
    priority: -1000

interactive-fiction
    location: /var/lib/layman/interactive-fiction
    sync-type: laymansync
    sync-uri: git://repo.or.cz/gentoo-interactive-fiction.git
    masters: gentoo
    priority: 50

steam-overlay
    location: /var/lib/layman/steam-overlay
    sync-type: laymansync
    sync-uri: git://github.com/anyc/steam-overlay.git
    masters: gentoo
    priority: 50

vmware
    location: /var/lib/layman/vmware
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/vmware.git
    masters: gentoo
    priority: 50

Local
    location: /usr/local/portage
    masters: gentoo
    priority: 99999999

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-pipe -mtune=native -march=native -O2 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-pipe -mtune=native -march=native -O2 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block"
DISTDIR="/mnt/Data/portage/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.ntua.gr/pub/linux/gentoo http://mirror.ovh.net/gentoo-distfiles http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -Wl,--sort-common"
MAKEOPTS="-j4"
PKGDIR="/mnt/Data/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi aes aes-ni alsa amd64 audiofile avx bash-completion branding bzip2 cairo cdr cli consolekit cracklib crypt cxx dbus declarative dri dts dvd dvdr emboss encode exif fam fbcondecor flac gdbm gif glamor gpm gtk iconv ipv6 jpeg kde kipi lcdfilter lcms libnotify libsamplerate lm_sensors mad mmx mmxext mng modules mp3 mp4 mpeg multilib ncurses nptl nvidia offensive ogg opengl openmp pam pango pcre pdf pgo phonon plasma png policykit popcnt ppds pulseaudio qml qt4 qt5 readline seccomp session spell sse sse2 sse3 sse4 sse4_1 sse4_2 ssl ssse3 startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vdpau vorbis widgets wxwidgets x264 xcb xcomposite xml xv xvid xvmc zlib" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes aes-ni avx mmx mmxext popcnt sse sse2 sse3 sse4 sse4_1 sse4_2 ssse3" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="emu efi-32 efi-64 pc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_US" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4" RUBY_TARGETS="ruby22" SANE_BACKENDS="nothing" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="3.4 3.3 2.7"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

sys-apps/portage-2.2.22::gentoo was built with the following:
USE="(ipc) -build -doc -epydoc (-selinux) -xattr" ABI_X86="64" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 python3_4 (-pypy)"
Comment 2 Zac Medico gentoo-dev 2015-10-05 21:58:13 UTC
This is a minor defect related to bug 557426, which is only noticeable when you are not using the --jobs option (or --jobs=1 which is equivalent).

What we need to do is change SyncManager.async to return a CompositeTask instance which encapsulates the existing AsyncFunction instance and also calls the SyncManager._sync_callback code before it exits. This will ensure that the _sync_callback code (which updates the cache) will execute before a particular repo's sync operation is considered 'complete'.
Comment 3 Zac Medico gentoo-dev 2015-10-05 23:07:42 UTC
There's a patch in the following branch:

https://github.com/zmedico/portage/tree/bug_562264

You can test it like this:

echo '=sys-apps/portage-9999 **' >> /etc/portage/package.accept_keywords
portage_LIVE_BRANCH=bug_562264 \
portage_LIVE_REPO=https://github.com/zmedico/portage.git \
emerge -1 =sys-apps/portage-9999

I've posted it for review here:

https://archives.gentoo.org/gentoo-portage-dev/message/7badb8ddf0f6861a7318be4f60aab7c1
Comment 5 Nikos Chantziaras 2015-10-07 02:32:37 UTC
I've put:

https://github.com/zmedico/portage/commit/4205ec912aebf2e3b0bd673fcacb576b1f344329.patch

into:

/etc/portage/patches/sys-apps/portage/

and emerged portage again. It fixes the issue for me.
Comment 6 Zac Medico gentoo-dev 2016-03-14 02:56:49 UTC
Fixed in 2.2.23.