Certain ciphers that sshd uses apparently use pthreads. When a connection using one is these ciphers is rekeyed, sshd (or more likely libcrypto from openssl) attempts to call pthread_cancel. That doesn't work if the user is chrooted. I hit this when using WinSCP as a client which negotiated the aes256-ctr, and transferring >1GB of data (its default rekey interval), but it can be easily reproduced with the vanilla sftp client. This is an interaction between sshd, libcrypto, pthreads, and the client (negotiated cipher and rekey interval). In my opinion, though, this is pthreads' fault - if they want to dlopen libgcc_s for pthread_cancel, they should do that when libpthread.so is loaded, not lazily when pthread_cancel is called. Other apps using chroot() are also affected by this. Reproducible: Always Steps to Reproduce: 1. Create a test user, e.g. "jailed" 2. Set it up as a chrooted user with internal-sftp in /etc/ssh/sshd_config: Subsystem sftp internal-sftp Match User jailed # Anything root-owned should work ChrootDirectory /opt ForceCommand internal-sftp 3. /etc/init.d/sshd restart 4. sftp -o Ciphers=aes256-ctr -o RekeyLimit=100 -v localhost@hub Actual Results: Client: debug1: need rekeying debug1: SSH2_MSG_KEXINIT sent debug1: rekeying in progress debug1: SSH2_MSG_KEXINIT received debug1: AUTH STATE IS 1 debug1: REQUESTED ENC.NAME is 'aes256-ctr' debug1: kex: server->client aes256-ctr hmac-sha2-512-etm@openssh.com none debug1: REQUESTED ENC.NAME is 'aes256-ctr' debug1: kex: client->server aes256-ctr hmac-sha2-512-etm@openssh.com none debug1: sending SSH2_MSG_KEX_ECDH_INIT debug1: expecting SSH2_MSG_KEX_ECDH_REPLY Connection closed by 127.0.0.1 Couldn't read packet: Connection reset by peer Server (in debug mode): debug1: SSH2_MSG_KEXINIT received debug1: SSH2_MSG_KEXINIT sent debug1: AUTH STATE IS 1 debug1: REQUESTED ENC.NAME is 'aes256-ctr' debug1: kex: client->server aes256-ctr hmac-sha2-512-etm@openssh.com none SSH: Server;Ltype: Kex;Remote: 127.0.0.1-50047;Enc: aes256-ctr;MAC: hmac-sha2-512-etm@openssh.com;Comp: none debug1: REQUESTED ENC.NAME is 'aes256-ctr' debug1: kex: server->client aes256-ctr hmac-sha2-512-etm@openssh.com none debug1: expecting SSH2_MSG_KEX_ECDH_INIT debug1: set_newkeys: rekeying libgcc_s.so.1 must be installed for pthread_cancel to work debug1: do_cleanup debug1: PAM: cleanup debug1: PAM: closing session debug1: PAM: deleting credentials Expected Results: Rekeying completes successfully and the connection is not closed. Ugly workaround: preload libgcc_s.so in /etc/init.d/sshd. This does not propagate to child binaries, since sshd cleans the environment, but does prevent sshd and its internal-sftp from hitting the bug. [...] LD_PRELOAD=libgcc_s.so \ start-stop-daemon --start --exec "${SSHD_BINARY}" \ [...] Portage 2.2.16 (python 2.7.9-final-0, hardened/linux/amd64, gcc-4.8.4, glibc-2.20-r1, 3.18.4-hardened x86_64) ================================================================= System uname: Linux-3.18.4-hardened-x86_64-Intel-R-_Core-TM-_i7-3820QM_CPU_@_2.70GHz-with-gentoo-2.2 KiB Mem: 32844268 total, 4450452 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Mon, 09 Feb 2015 08:15:01 +0000 sh bash 4.3_p33-r1 ld GNU ld (Gentoo 2.24 p1.4) 2.24 distcc 3.2rc1 x86_64-pc-linux-gnu [disabled] ccache version 3.2.1 [disabled] app-shells/bash: 4.3_p33-r1::gentoo dev-java/java-config: 2.2.0::gentoo dev-lang/perl: 5.20.1-r4::gentoo dev-lang/python: 2.7.9-r2::gentoo, 3.3.5-r1::gentoo, 3.4.2::gentoo dev-util/ccache: 3.2.1-r1::gentoo dev-util/cmake: 3.1.0::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.13.8::gentoo sys-apps/sandbox: 2.6-r1::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69::gentoo sys-devel/automake: 1.11.6-r1::gentoo, 1.12.6::gentoo, 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.24-r3::gentoo sys-devel/gcc: 4.8.4::gentoo, 4.9.2::gentoo sys-devel/gcc-config: 1.8::gentoo sys-devel/libtool: 2.4.5::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 3.18::gentoo (virtual/os-headers) sys-libs/glibc: 2.20-r1::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.jp.gentoo.org/gentoo-portage priority: -1000 local location: /usr/local/portage masters: gentoo proaudio priority: 0 klondike location: /var/lib/layman/klondike masters: gentoo priority: 1 steam-overlay location: /var/lib/layman/steam masters: gentoo priority: 2 bumblebee location: /var/lib/layman/bumblebee masters: gentoo priority: 3 proaudio location: /var/lib/layman/pro-audio masters: gentoo priority: 4 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA Google-TOS ut2003 PUEL Q3AEULA Introversion dlj-1.1 skype-eula cadsoft AdobeFlash-10.1 AdobeFlash-10 AdobeFlash-9.0.31.0 googleearth google-talkplugin Oracle-BCLA-JavaSE AdobeFlash-10.3 AdobeFlash-11.x skype-4.0.0.7-copyright google-chrome NVIDIA-CUDA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.1/conf /usr/share/maven-bin-3.2/conf /usr/share/themes/oxygen-gtk/gtk-2.0 /usr/share/themes/oxygen-gtk/gtk-3.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://gentoo.gg3.net/ ftp://ftp.iij.ad.jp/pub/linux/gentoo/ ftp://ftp.jaist.ac.jp/pub/Linux/Gentoo/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j9" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--timeout=180" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" USE="7zip X a52 aac aalib acl acpi addbookmarks additions addressbook akode alias alsa amarok amd64 amd64codecs anthy apache2 apng artswrappersuid asf aspell audiofile autoreplace avahi avi bash-completion berkdb bitmap-fonts bittorrent-external blender-game bluetooth bonjour branding bzip2 bzlib cairo calendar canna cdda cddb cdio cdparanoia cdr cgi chm cjk clamav cli connectionstatus consolekit contactnotes cracklib crypt css ctype cups curl cvs cxx daap dba dbus declarative device-mapper dillo djvu dlloader dri dts dv dvb dvd dvdr dvdread embedded enca encode equalizer examples exif faad fam fasttrack fbcondecor ffmpeg firefox3 flac flash fluidsynth font-server foomaticdb frei0r ftdi ftp g3dvl gallium gcrypt gd gdbm gif git glamor gles2 glitz gmp gnutella go gphoto2 gpm gs gsm gtk gtk2 h323 hackrf hardened hash highlight history httpd ibus iconv icu id3tag ieee1394 imagemagick imap immqt-bc inifile inkjar innodb ipv6 irc ithreads jabber jack jadetex java java6 javascript jingle joystick jpeg jpeg2k justify kate kde kipi kpathsea kqemu kvm ladspa laptop lcms lensfun libass libnotify libsamplerate libtiger libv4l2 lisamplerate llvm lm_sensors logrotate lv2 lvm lzma m17n-lib mad maildir matroska mbox mbrola mdnsresponder-compat melt midi mikmod mjpeg mmx mng mod modemmanager modplug modules moodbar moviepack moviesounds mozbranding mozcalendar mozdevelop mozsvg mozxmlterm mp3 mp4 mpeg mpeg4 mplayer msn mtp multilib musicbrainz mysql mysqli mythtv ncurses network networkmanager nfs nls nodrm nowlistening nptl nptlonly ntp numa nvidia offensive ogg oggvorbis openal openct openexr opengl openmp opus oscar otr pam pango parted pax_kernel pcap pcre pcsc-lite pdf perl phonon plasma plotutils png policykit posix ppds prime projectm protutils pulseaudio pyqt4 python qemu qemu-ifup qt qt3support qt4 quicktime rar readline realtime resolvconf rt2500pci rtc rtlsdr rtsp samba scanner sdk sdl session shout skins slang sna sndfile soap sockets sox speex spell sql sqlite srt sse sse2 ssl startup-notification statistics stream subtitles subversion svg swat system-libvncserver taglib tcpd test-programs texteffect tga theora threads tiff tk transcode transmitter truetype truetype-fonts twolame type1-fonts udev udisks unicode upnp upower urandom usb utempter utf8 uxa v4l v4l2 video vim-syntax virt-network visualization vlm vorbis vpx wavpack webkit webpresence wifi wininst wmf wxwidgets wxwindows x264 xattr xcb xcomposite xforms xinerama xml xmlreader xmlrpc xmlwriter xmp xprint xpsmall xscreensaver xsl xtpax xulrunner xv xvid yahoo yv12 zeroconf zip zlib zsh-completion zvbi" ABI_X86="64" ALSA_CARDS="usb-audio virmidi loopback hda-intel" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias slotmem_shm unixd" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="canon" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx sse sse2 sse3 ssse3 avx sse4_1 sse4_2" ELIBC="glibc" FOO2ZJS_DEVICES="sa315" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INITNG_PLUGINS="also bash_launcher chdir chroot conflict cpout critical ctrlaltdel daemon_clean debug_commands envparser find fmon fstat history idleprobe initctl interactive iparser last limit lockfile logfile netdev netprobe ngc4 ngcs nge pause provide reload renice rlparser simple_launcher stcmd stdout suid syncron syslog sysreq unneeded usplash" INPUT_DEVICES="evdev keyboard mouse wacom synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_US es es_ES ja ja_JP" LIRC_DEVICES="serial devinput" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" QEMU_SOFTMMU_TARGETS="x86_64 arm armeb ppc ppc64 mips mipsel s390x aarch64" QEMU_USER_TARGETS="x86_64 s390x arm armeb mips mipsel ppc ppc64 ppc64abi32 aarch64" RUBY_TARGETS="ruby22 ruby21 ruby20" SANE_BACKENDS="plustek" USERLAND="GNU" VIDEO_CARDS="intel nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7 3.3" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
Silly typo in step 4: $ sftp -o Ciphers=aes256-ctr -o RekeyLimit=100 -v jailed@localhost
I'm no longer using the LD_PRELOAD workaround and cannot reproduce, so I assume this got fixed at some point in the packages involved.
