when using geo-replication with glusterfs the master look for `gsyncd` in /usr/local/libexec/ when it should look for it in /usr/libexec/. The replication obviously fail. log exerpt: [2013-04-02 15:01:45.205017] I [monitor(monitor):80:monitor] Monitor: ------------------------------------------------------------ [2013-04-02 15:01:45.205230] I [monitor(monitor):81:monitor] Monitor: starting gsyncd worker [2013-04-02 15:01:45.246262] I [gsyncd:354:main_i] <top>: syncing: gluster://localhost:geo1 -> ssh://10.7.0.6:/export/brick1_slave [2013-04-02 15:01:45.996646] E [syncdutils:173:log_raise_exception] <top>: connection to peer is broken [2013-04-02 15:01:45.996880] E [resource:191:errlog] Popen: command "ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-G2wNeF/gsycnd-ssh-%r@%h:%p 10.7.0.6 /usr/local/libexec/glusterfs/gsyncd --session-owner b7f48a07-990b-4cb7-8421-fb68775762fc -N -- listen --timeout 120 file:///export/brick1_slave" returned with 127, saying: [2013-04-02 15:01:45.997013] E [resource:194:logerr] Popen: ssh> bash: /usr/local/libexec/glusterfs/gsyncd: No such file or directory [2013-04-02 15:01:45.997153] I [syncdutils:142:finalize] <top>: exiting. $ emerge --info Portage 2.2.0_alpha163 (default/linux/amd64/13.0/no-multilib, gcc-4.6.3, glibc-2.15-r3, 3.7.9-vs2.3.5.6 x86_64) ================================================================= System uname: Linux-3.7.9-vs2.3.5.6-x86_64-Intel-R-_Core-TM-_i7-3930K_CPU_@_3.20GHz-with-gentoo-2.1 KiB Mem: 65358672 total, 10996084 free KiB Swap: 0 total, 0 free Timestamp of tree: Mon, 01 Apr 2013 20:00:01 +0000 ld GNU ld (GNU Binutils) 2.22 app-shells/bash: 4.2_p37 dev-lang/python: 2.7.3-r2 dev-util/cmake: 2.8.9 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.11.8 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.69 sys-devel/automake: 1.11.6 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.5.4, 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.6 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo tmp ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=core2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=core2 -pipe" DISTDIR="/g/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="it_IT.UTF-8" LC_ALL="it_IT.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j12" PKGDIR="/g/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/g/tmp" PORTDIR="/g/portage" PORTDIR_OVERLAY="/g/ovl" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="acl amd64 berkdb bitmap-fonts bzip2 caps cli cracklib crypt cups cxx dri gdbm gpm hpn iconv ipv6 isdnlog midi mmx modules mudflap ncurses nls nptl nptlonly openmp pcre perl ppds pppd python readline reflection session spl sse sse2 ssl tcpd truetype-fonts type1-fonts unicode userlocales xattr xorg zlib" ABI_X86="64" ALSA_PCM_PLUGINS="null" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="caps mdraid ssh-client net syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev wacom" KERNEL="linux" LCD_DEVICES="ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="it" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7" QEMU_SOFTMMU_TARGETS="x86_64" QEMU_USER_TARGETS="x86_64" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="dummy fbdev vesa vga radeon radeonhd" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
it's fixated in sources: .../glusterfs-3.3.1/xlators/mgmt/glusterd/src/glusterd.c:465 probably it's configurable somewhere at runtime, but Gentoo should default to /usr/libexec/glusterfs/gsyncd because it's the place it use by default. /************ * master pre-configuration ************/ /* remote-gsyncd */ runinit_gsyncd_setrx (&runner, conf); runner_add_args (&runner, "remote-gsyncd", GSYNCD_PREFIX"/gsyncd", ".", ".", NULL); RUN_GSYNCD_CMD; runinit_gsyncd_setrx (&runner, conf); runner_add_args (&runner, "remote-gsyncd", "/usr/local/libexec/glusterfs/gsyncd", ".", "^ssh:", NULL); RUN_GSYNCD_CMD;
Please attach full build.log. I get -DGSYNCD_PREFIX=\"/usr/libexec/glusterfs\" out of the box
Created attachment 344162 [details] glusterfs.log in glusterd.c:465 replace: runner_add_args (&runner, "remote-gsyncd", "/usr/local/libexec/glusterfs/gsyncd", ".", "^ssh:", NULL); with: runner_add_args (&runner, "remote-gsyncd", GSYNCD_PREFIX"/gsyncd", ".", "^ssh:", NULL); GSYNCD_PREFIX actually is used for the _local_ gsyncd server, it use ssh to call gsyncd on the remote server too but use the /usr/local/ path hardcoded. RedHat decided this way because the source built packages could expect to be located in usr/local, the point is Gentoo use source build and not rpm. An alternative is to add einfo to modify the "remote_gsyncd =" line in /var/lib/glusterd/geo-replication/gsyncd.conf . But this must be carefully worded because the file is created at runtime after a geo-replication is requested, so it can be confusing to explain.
(In reply to comment #3) > Created attachment 344162 [details] > glusterfs.log > > in glusterd.c:465 > replace: > runner_add_args (&runner, "remote-gsyncd", > "/usr/local/libexec/glusterfs/gsyncd", ".", "^ssh:", NULL); > with: > runner_add_args (&runner, "remote-gsyncd", > GSYNCD_PREFIX"/gsyncd", ".", "^ssh:", NULL); I've misunderstood you first time, I'll add it shortly.
+*glusterfs-3.3.1-r2 (03 Apr 2013) + + 03 Apr 2013; Kacper Kowalik <xarthisius@gentoo.org> + +glusterfs-3.3.1-r2.ebuild, -glusterfs-3.3.1-r1.ebuild, + glusterfs-3.3.0.ebuild: + Expect that gsync resides in /usr/libexec/glusterfs on remote machine or in + other words that the remote machine runs Gentoo too. Fixes #464196 by + Francesco Riosa <francesco+gentoo_bugzilla@pnpitalia.it> who also provided the + patch
