Bug 496244

Summary:	dev-vcs/git-annex-5.20131130 git-annex infinitely trying to retrieve a file from remote
Product:	Gentoo Linux	Reporter:	headcrabextra
Component:	[OLD] Development	Assignee:	Gentoo's Haskell Language team <haskell>
Status:	RESOLVED OBSOLETE
Severity:	normal	CC:	jstein
Priority:	Normal
Version:	unspecified
Hardware:	AMD64
OS:	Linux
Whiteboard:
Package list:		Runtime testing required:	---
Attachments:	A screenshot of mentioned transfers The log from the same time from my debian computer

Description headcrabextra 2013-12-28 08:46:29 UTC

Created attachment 366354 [details]
A screenshot of mentioned transfers

I have git-annex with version mentioned in the summary on gentoo and git-annex 4.20131106 on Debian Jessie x86_64 on another computer. They are in the same LAN and paired. Today this happened:
I had my gentoo computer turned on. I turned on my Debian computer, git-annex started there in autostart. They started syncing, synced some files (maybe all files - not sure), Debian computer checked integrity. After that on Debian I have git-annex dashboard showing no notifications and (No file transfers running), on the Gentoo computer I have git-annex and git processes each taking 13% of CPU, there is this one file stuck in Transfers as
filename <- ip_adress_annex queued (size)
Also for about 15 minutes there were appearing and disappearing 2 more transfers with the same filename and ip_adress, queued, but different sizes.
Then those 2 more transfers were no longer there but 1 was still there.
Now those 2 appeared again.

Here is a piece of log starting from the moment I turned on second computer (distinguished by timemark)

[2013-12-28 12:03:56 MSK] main: Syncing with 192.168.1.246_annex 
ssh: connect to host 192.168.1.246 port 22: No route to host
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Already up-to-date.
ssh: connect to host 192.168.1.246 port 22: No route to host
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
From ssh://git-annex-192.168.1.246-crabman_annex/~/annex
   9119db7..b0978af  git-annex  -> 192.168.1.246_annex/git-annex
   cd37e54..9dddca4  master     -> 192.168.1.246_annex/master
   9ba4c0e..9dddca4  synced/master -> 192.168.1.246_annex/synced/master
Already up-to-date.
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   d2ab506..602d90a  git-annex -> synced/git-annex
   9dddca4..26ee326  annex/direct/master -> synced/master
Already up-to-date.
[2013-12-28 12:05:10 MSK] read: git ["--git-dir=/home/crabman/annex/.git","--work-tree=/home/crabman/annex","-c","core.bare=false","show-ref","git-annex"]
[2013-12-28 12:05:10 MSK] read: git ["--git-dir=/home/crabman/annex/.git","--work-tree=/home/crabman/annex","-c","core.bare=false","show-ref","--hash","refs/heads/git-annex"]
[2013-12-28 12:05:10 MSK] read: git ["--git-dir=/home/crabman/annex/.git","--work-tree=/home/crabman/annex","-c","core.bare=false","log","refs/heads/git-annex..602d90a15a10edec43c1bac432a122d4eeeb5ddd","--oneline","-n1"]
[2013-12-28 12:05:10 MSK] read: git ["--git-dir=/home/crabman/annex/.git","--work-tree=/home/crabman/annex","-c","core.bare=false","log","refs/heads/git-annex..b0978af6db1732f9edb07e069a5607c757e0ccd8","--oneline","-n1"]
[2013-12-28 12:05:10 MSK] chat: git ["--git-dir=/home/crabman/annex/.git","--work-tree=/home/crabman/annex","-c","core.bare=false","cat-file","--batch"]
[2013-12-28 12:05:10 MSK] read: rsync ["--progress","--inplace","-e","'ssh' '-S' '.git/annex/ssh/[censored]' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'crabman@git-annex-192.168.1.246-crabman_annex' 'git-annex-shell ''recvkey'' ''/~/annex/'' ''SHA256E-[censored].kdb'' --uuid [censored] ''--'' ''remoteuuid=[censored]'' ''direct=1'' ''associatedfile=Documents/Keys/kpdb.kdb'' ''--'''","--","/home/crabman/annex/Documents/Keys/kpdb.kdb","dummy:"]
[2013-12-28 12:05:48 MSK] call: rsync ["--server","-t","--inplace","-e.Lsf",".","//home/crabman/annex/.git/annex/tmp/SHA256E-[censored].kdb"]

kpdb.kdb
       32768  24%    0.00kB/s    0:00:00
      134556 100%   97.07MB/s    0:00:00 (xfer#1, to-check=0/1)
[2013-12-28 12:05:48 MSK] chat: git ["--git-dir=//home/crabman/annex/.git","--work-tree=//home/crabman/annex/","cat-file","--batch"]

sent 134644 bytes  received 31 bytes  29927.78 bytes/sec
total size is 134556  speedup is 1.00
[2013-12-28 12:05:14 MSK] Transferrer: Uploaded kpdb.kdb
[2013-12-28 12:05:14 MSK] Pusher: Syncing with 192.168.1.246_annex 
[2013-12-28 12:05:14 MSK] read: rsync ["--progress","--inplace","-e","'ssh' '-S' '.git/annex/ssh/[censored]' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'crabman@git-annex-192.168.1.246-crabman_annex' 'git-annex-shell ''recvkey'' ''/~/annex/'' ''SHA256E-[censored].m4a'' --uuid [censored] ''--'' ''remoteuuid=[censored]'' ''direct=1'' ''associatedfile=Music/Saints Row: The Third OST/Saints Row - The Third [Soundtrack] - I Need a Hero-XNEDoARuzxE.m4a'' ''--'''","--","/home/crabman/annex/Music/Saints Row: The Third OST/Saints Row - The Third [Soundtrack] - I Need a Hero-XNEDoARuzxE.m4a","dummy:"]
[2013-12-28 12:05:48 MSK] call: rsync ["--server","-t","--inplace","-e.Lsf",".","//home/crabman/annex/.git/annex/tmp/SHA256E-[censored].m4a"]

Saints Row - The Third [Soundtrack] - I Need a Hero-XNEDoARuzxE.m4a
       32768   0%    0.00kB/s    0:00:00
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   602d90a..58431ef  git-annex -> synced/git-annex
     3932160  42%    3.71MB/s    0:00:01
     9202426 100%    4.43MB/s    0:00:01 (xfer#1, to-check=0/1)
[2013-12-28 12:05:50 MSK] read: sha256sum ["//home/crabman/annex/.git/annex/tmp/SHA256E-s9202426--1e79f1c98a97e4fcd67bed6fa209c0e182ee5ec7232d7a3540fb184f2d93b642.m4a"]
[2013-12-28 12:05:51 MSK] chat: git ["--git-dir=//home/crabman/annex/.git","--work-tree=//home/crabman/annex/","cat-file","--batch"]

sent 9203678 bytes  received 31 bytes  2629631.14 bytes/sec
total size is 9202426  speedup is 1.00
[2013-12-28 12:05:19 MSK] Transferrer: Uploaded Saints Ro..RuzxE.m4a
[2013-12-28 12:05:19 MSK] Pusher: Syncing with 192.168.1.246_annex 
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   58431ef..9f691b0  git-annex -> synced/git-annex
[2013-12-28 12:06:02 MSK] read: rsync ["--progress","--inplace","-e","'ssh' '-S' '.git/annex/ssh/[censored]' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'crabman@git-annex-192.168.1.246-crabman_annex' 'git-annex-shell ''recvkey'' ''/~/annex/'' ''SHA256E-s110081--3a5bad0aa9f89b88f9f602c377346e0517aa7555e18c3f6311fc2a2cfb5092d8.png'' --uuid 59849d30-12c9-4af6-abb0-6b2558d5725e ''--'' ''remoteuuid=5e3abb9b-e0e2-415c-9463-26a200fb19dd'' ''direct=1'' ''associatedfile=chan/s/femi tux.png'' ''--'''","--","/home/crabman/annex/chan/s/femi tux.png","dummy:"]
[2013-12-28 12:06:35 MSK] call: rsync ["--server","-t","--inplace","-e.Lsf",".","//home/crabman/annex/.git/annex/tmp/SHA256E-s110081--3a5bad0aa9f89b88f9f602c377346e0517aa7555e18c3f6311fc2a2cfb5092d8.png"]

femi tux.png
       32768  29%    0.00kB/s    0:00:00
      110081 100%   73.73MB/s    0:00:00 (xfer#1, to-check=0/1)
[2013-12-28 12:06:35 MSK] chat: git ["--git-dir=//home/crabman/annex/.git","--work-tree=//home/crabman/annex/","cat-file","--batch"]

sent 110169 bytes  received 31 bytes  220400.00 bytes/sec
total size is 110081  speedup is 1.00
[2013-12-28 12:06:02 MSK] Transferrer: Uploaded femi tux.png
[2013-12-28 12:06:02 MSK] Pusher: Syncing with 192.168.1.246_annex 
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   9f691b0..0678c8c  git-annex -> synced/git-annex
[2013-12-28 12:10:18 MSK] Committer: Adding .~lock.теормин.odt#
ok
(Recording state in git...)
(Recording state in git...)



(Recording state in git...)
(Recording state in git...)
(merging synced/git-annex into git-annex...)
(Recording state in git...)
(Recording state in git...)
add Documents/МГУ/5 семестр/физика/.~lock.теормин.odt# (checksum...) ok
add Documents/МГУ/5 семестр/физика/.~lock.теормин.odt# (checksum...) [2013-12-28 12:10:18 MSK] Committer: Committing changes to git
[2013-12-28 12:10:18 MSK] Pusher: Syncing with 192.168.1.246_annex 
[2013-12-28 12:10:18 MSK] read: rsync ["--progress","--inplace","-e","'ssh' '-S' '.git/annex/ssh/[censored]' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'crabman@git-annex-192.168.1.246-crabman_annex' 'git-annex-shell ''recvkey'' ''/~/annex/'' ''SHA256E-s75--d7b9f18fd9a0936aa795f62e556ba77142dfcf68c34ddd1ffe3902041f63b146.odt'' --uuid 59849d30-12c9-4af6-abb0-6b2558d5725e ''--'' ''remoteuuid=5e3abb9b-e0e2-415c-9463-26a200fb19dd'' ''direct=1'' ''associatedfile=Documents/\1052\1043\1059/5 \1089\1077\1084\1077\1089\1090\1088/\1092\1080\1079\1080\1082\1072/.~lock.\1090\1077\1086\1088\1084\1080\1085.odt#'' ''--'''","--","/home/crabman/annex/Documents/\1052\1043\1059/5 \1089\1077\1084\1077\1089\1090\1088/\1092\1080\1079\1080\1082\1072/.~lock.\1090\1077\1086\1088\1084\1080\1085.odt#","dummy:"]
[2013-12-28 12:10:51 MSK] call: rsync ["--server","-t","--inplace","-e.Lsf",".","//home/crabman/annex/.git/annex/tmp/SHA256E-s75--d7b9f18fd9a0936aa795f62e556ba77142dfcf68c34ddd1ffe3902041f63b146.odt"]

.~lock.ÑÐµÐ¾ÑÐ¼Ð¸Ð½.odt#
          75 100%    0.00kB/s    0:00:00
          75 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)
[2013-12-28 12:10:52 MSK] chat: git ["--git-dir=//home/crabman/annex/.git","--work-tree=//home/crabman/annex/","cat-file","--batch"]

sent 165 bytes  received 31 bytes  392.00 bytes/sec
total size is 75  speedup is 0.38
[2013-12-28 12:10:18 MSK] Transferrer: Uploaded .~lock.теормин.odt#
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   0678c8c..b2d71d4  git-annex -> synced/git-annex
   26ee326..8c2cffa  annex/direct/master -> synced/master
[2013-12-28 12:10:21 MSK] Pusher: Syncing with 192.168.1.246_annex 
[2013-12-28 12:10:22 MSK] Committer: Committing changes to git
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   b2d71d4..a3781a2  git-annex -> synced/git-annex
[2013-12-28 12:10:23 MSK] Committer: Committing changes to git
[2013-12-28 12:10:24 MSK] Pusher: Syncing with 192.168.1.246_annex 
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   a3781a2..a1a1fac  git-annex -> synced/git-annex
   8c2cffa..42734ef  annex/direct/master -> synced/master
[2013-12-28 12:32:45 MSK] NetWatcherFallback: Syncing with 192.168.1.246_annex 
From ssh://git-annex-192.168.1.246-crabman_annex/~/annex
   b0978af..a9aceac  git-annex  -> 192.168.1.246_annex/git-annex
   26ee326..42734ef  master     -> 192.168.1.246_annex/master
To ssh://crabman@git-annex-192.168.1.246-crabman_annex/~/annex/
   a1a1fac..a9aceac  git-annex -> synced/git-annex


$ emerge --info git-annex
Portage 2.2.7 (default/linux/amd64/13.0/desktop/kde, gcc-4.7.3, glibc-2.16.0, 3.10.17-gentoogg5 x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-3.10.17-gentoogg5-x86_64-Intel-R-_Core-TM-_i7-3517U_CPU_@_1.90GHz-with-gentoo-2.2
KiB Mem:     3936720 total,     67048 free
KiB Swap:    4509692 total,   4329040 free
Timestamp of tree: Sat, 28 Dec 2013 06:15:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
app-shells/bash:          4.2_p45
dev-lang/python:          2.7.5-r3, 3.3.2-r2
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.10.3, 1.11.6, 1.12.6, 1.13.4
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.9 (virtual/os-headers)
sys-libs/glibc:           2.16.0
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe -ggdb"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe -ggdb"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--ask-enter-invalid"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp.vectranet.pl/gentoo/ http://gentoo.bloodhost.ru/ http://mirror2.corbina.ru/gentoo-distfiles/ http://mirror.yandex.ru/gentoo-distfiles/ http://tux.rainside.sk/gentoo/ http://ftp.df.lth.se/pub/gentoo/ http://gentoo.kiev.ua/ftp/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j9"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.ru.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa amd64 berkdb bindist bluetooth branding bzip2 cairo cdda cdr cli cmake consolekit cracklib crypt cryptsetup cups cxx dbus declarative dri dts dvd dvdr emacs emboss encode exif fam firefox flac fortran gdbm gif git gpm gtk iconv ipv6 jpeg kde kipi lcms ldap libnotify mad mmx mng modules mp3 mp4 mpeg mudflap multilib ncurses nls nptl ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds pulseaudio qt3support qt4 readline sdl semantic-desktop session spell sse sse1 sse2 ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb valgrind vorbis wxwidgets x264 xcb xcomposite xinerama xml xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en ru" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="intel" XFCE_PLUGINS="brightness clock trash" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

dev-vcs/git-annex-5.20131130 was built with the following:
USE="assistant cryptohash dbus dns feed inotify pairing production quvi s3 tdfa testsuite webapp webdav xmpp -android -doc"
CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="-march=native -O2 -pipe"

Comment 1 headcrabextra 2013-12-28 09:00:54 UTC

Created attachment 366356 [details]
The log from the same time from my debian computer

Here is the log from the 2nd computer. BTW the name of the file that caused this problem is Documents/МГУ/5 семестр/физика/теормин.odt

Comment 2 headcrabextra 2013-12-28 09:06:57 UTC

Also after I shut down the daemon and started git-annex again, there were no transfers on both machines and nothing taking so much cpu time.

Comment 3 Sergei Trofimovich (RETIRED) gentoo-dev

2014-07-19 06:35:04 UTC

May I ask you to check last 5.20140717 portage release
and report that bug upstream if it fails?

Comment 4 headcrabextra 2014-07-19 11:58:58 UTC

(In reply to Sergei Trofimovich from comment #3)
> May I ask you to check last 5.20140717 portage release
> and report that bug upstream if it fails?

As far as I remember this bug happened randomly, not all the time. I'll upgrade to the version you mentioned and if it happens again, I'll report it upstream.

Comment 5 Jonas Stein gentoo-dev

2016-08-21 22:59:51 UTC

dev-vcs/git-annex-5.20131130 is no longer in the tree.
any news? Can we close this ticket?