When emerging binary packages from PORTAGE_BINHOST, FTP protocol errors are not being catched. This leads to a ugly stack trace and emerge fails immediately instead of proceeding with the next package. The problem is the "except" line in /usr/lib64/portage/pym/portage/getbinpkg.py +504. socket.error exceptions are catched, but protocol errors are not catched. I think that ftplib.Error should be catched as well. All exceptions which are raised from ftplib can be found in ftplib.all_errors. Reproducible: Always Steps to Reproduce: export PORTAGE_BINHOST="ftp://anonymous:portage@services03/pub/packages/d9b4d6a87b064ce0" # binary server in our network emerge eix # any ebuild for which a binpkg exists Actual Results: # With a defective FTP server (e.g., chmod -x pure-ftpd), I get the following stack trace: !!! Error fetching binhost package info from 'ftp://anonymous:portage@services03/pub/packages/d9b4d6a87b064ce0' !!! [Errno ftp error] Fetching bininfo from ftp://anonymous:*password*@services03/pub/packages/d9b4d6a87b064ce0 Traceback (most recent call last): File "/usr/bin/emerge", line 42, in <module> retval = emerge_main() File "/usr/lib/portage/pym/_emerge/main.py", line 1119, in emerge_main getbinpkgs="--getbinpkg" in myopts) File "/usr/lib/portage/pym/portage/dbapi/bintree.py", line 482, in populate self._populate(getbinpkgs) File "/usr/lib/portage/pym/portage/dbapi/bintree.py", line 835, in _populate self.settings["PORTAGE_BINHOST"], chunk_size=chunk_size) File "/usr/lib/portage/pym/portage/getbinpkg.py", line 503, in dir_get_metadata conn, protocol, address, params, headers = create_conn(baseurl, conn) File "/usr/lib/portage/pym/portage/getbinpkg.py", line 163, in create_conn conn = ftplib.FTP(host) File "/usr/lib/python2.6/ftplib.py", line 116, in __init__ self.connect(host) File "/usr/lib/python2.6/ftplib.py", line 134, in connect self.welcome = self.getresp() File "/usr/lib/python2.6/ftplib.py", line 209, in getresp resp = self.getmultiline() File "/usr/lib/python2.6/ftplib.py", line 195, in getmultiline line = self.getline() File "/usr/lib/python2.6/ftplib.py", line 185, in getline if not line: raise EOFError EOFError Expected Results: # With a working FTP server, I get the normal binpkg installation output: Calculating dependencies... done! >>> Emerging binary (1 of 1) app-portage/eix-0.20.5 --2010-09-20 14:44:04-- ftp://anonymous:*password*@services03/pub/packages/d9b4d6a87b064ce0/app-portage/eix-0.20.5.tbz2 => “/tmp/portage/packages/d9b4d6a87b064ce0/app-portage/eix-0.20.5.tbz2” Resolving services03... 172.20.3.4 Connecting to services03|172.20.3.4|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/packages/d9b4d6a87b064ce0/app-portage ... done. ==> SIZE eix-0.20.5.tbz2 ... 528553 ==> PASV ... done. ==> RETR eix-0.20.5.tbz2 ... done. 0K .......... .......... .......... .......... .......... 796K 50K .......... .......... .......... .......... .......... 76.2M 100K .......... .......... .......... .......... .......... 91.9M 150K .......... .......... .......... .......... .......... 110M 200K .......... .......... .......... .......... .......... 101M 250K .......... .......... .......... .......... .......... 1.82M 300K .......... .......... .......... .......... .......... 88.7M 350K .......... .......... .......... .......... .......... 49.8M 400K .......... .......... .......... .......... .......... 41.0M 450K .......... .......... .......... .......... .......... 25.7M 500K .......... ...... 18.9M=0.1s 2010-09-20 14:44:04 (5.18 MB/s) - “/tmp/portage/packages/d9b4d6a87b064ce0/app-portage/eix-0.20.5.tbz2” saved [528553] * eix-0.20.5.tbz2 MD5 SHA1 size ;-) ... [ ok ] >>> Extracting info * CPV: app-portage/eix-0.20.5 * REPO: gentoo * USE: bzip2 elibc_glibc kernel_linux nls userland_GNU x86 >>> Extracting app-portage/eix-0.20.5 >>> Installing (1 of 1) app-portage/eix-0.20.5 >>> Auto-cleaning packages... >>> No outdated packages were found on your system. * GNU info directory index is up-to-date. Portage 2.1.8.3 (default/linux/x86/10.0, gcc-4.4.3, glibc-2.11.2-r0, 2.6.34-gentoo-r1 i686) ================================================================= System uname: Linux-2.6.34-gentoo-r1-i686-QEMU_Virtual_CPU_version_0.12.5-with-gentoo-1.12.13 Timestamp of tree: Fri, 13 Aug 2010 07:00:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 4.0_p37 dev-java/java-config: 2.1.11 dev-lang/python: 2.4.6, 2.5.4-r4, 2.6.4-r1, 3.1.2-r4 dev-util/ccache: 2.4-r7 dev-util/cmake: 2.8.1-r2 sys-apps/baselayout: 1.12.13 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.65 sys-devel/automake: 1.11.1 sys-devel/binutils: 2.20.1-r1 sys-devel/gcc: 4.4.3-r2 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6b virtual/os-headers: 2.6.30-r1 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="* -@EULA dlj-1.1" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -pipe -march=i686" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc" services03/pub/packages/d9b4d6aCONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/conf.d/apache2 /etc/eclean /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts /etc/fonts/fonts.conf /etc/gconf /etc/init.d /etc/openldap/schema /etc/portage/savedconfig /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=i686" DISTDIR="/tmp/portage/distfiles" EMERGE_DEFAULT_OPTS="--binpkg-respect-use y --with-bdeps y --jobs=1 --load-average=1 --getbinpkg" FEATURES="assume-digests buildpkg ccache distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch usersandbox" GENTOO_MIRRORS="http://download.gocept.com/gentoo/mirror/ ftp://de-mirror.org/distro/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo http://mirrors.sec.informatik.tu-darmstadt.de/gentoo/ " LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="de en" MAKEOPTS="-j1 -l1" PKGDIR="/tmp/portage/packages/d9b4d6a87b064ce0" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="acl acpi apache2 bash-completion berkdb bzip2 cli cracklib crypt ctype curl cxx device-mapper dri filter fortran gd gdbm geoip gnutls iconv ipv6 jpeg json kvm ldap logrotate mercurial mhash mudflap nagios-dns nagios-ntp nagios-ping nagios-ssh ncurses nls nptl nptlonly openmp pam pcre perl png pppd python readline reflection sasl session simplexml snmp spl sqlite3 sse sse2 ssl symlink sysfs syslog tcpd truetype unicode utils vhosts vim-syntax x86 xinetd xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias proxy proxy_balancer proxy_connect proxy_ftp proxy_http" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de en" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa via vmware voodoo" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
The exception indicates that you're triggering the old protocol, which happens whenever $PORTAGE_BINHOST/Packages can't be fetched. Make sure that you have $PORTAGE_BINHOST set correctly so that the Packages file can be fetched. You can run `emaint --fix binhost` on the server in order to generate or update the Packages file. This file is updated automatically with new packages when they are built, but you need to call `emaint --fix binhost` if it is out of sync with the packages in $PKGDIR for any reason. For example, eclean-pkg calls `emaint --fix binhost` after it removes old binary packages.
(In reply to comment #1) > The exception indicates that you're triggering the old protocol. I'm trying to clarify. The bug is triggered when a defective FTP server gives *no* sensible answers at all. I think that emerge should not die in such cases. Instead, it should either try to compile the package or proceed with the next one.
Thanks, there's a fix in git now: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=bf5877484cbca0b59c1b9d3688d338465083bcf9
This is fixed in 2.1.9.10 and 2.2_rc86.