Summary: | sys-apps/portage-2.1.9.26 emerge freezes w. pure-ftpd binhost | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Jarno Mäkelä <jarno.makela> |
Component: | Core | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | vmatare+gbug |
Priority: | High | ||
Version: | unspecified | ||
Hardware: | AMD64 | ||
OS: | Linux | ||
URL: | http://bugs.python.org/issue11199 | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=437112 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 377365, 349307 | ||
Attachments: |
emerge backtrace
add timeout when closing binhost connection download failed session log successful download session log A simple python script for ftp testing three repeated successes with ftptest.py pure-ftpd conf |
Description
Jarno Mäkelä
2010-12-30 10:16:44 UTC
In order to create a backtrace, open a separate shell from the frozen emerge and run `kill -s SIGUSR1 <pid>` for the emerge pid. That will make the frozen emerge start the debugger (Pdb). At the (Pdb) prompt, type the "bt" command and press Enter. Please copy the backtrace and post it here. Created attachment 258487 [details]
emerge backtrace
Created attachment 258490 [details, diff]
add timeout when closing binhost connection
Save as /tmp/binhost_timeout.patch and apply as follows:
patch /usr/lib/portage/pym/portage/dbapi/bintree.py /tmp/binhost_timeout.patch
Timeout works. Download still fails unless the local copy of Packages is deleted. I'll post ftp logs.. Created attachment 258492 [details]
download failed session log
Created attachment 258493 [details]
successful download session log
After the timeout, does emerge output some more error info? It seems like the ftp server and/or python's ftplib are behaving incorrectly. Maybe try another ftp server program? It seems the person in the other forums thread was having the same problem with pure-ftp. (In reply to comment #7) > After the timeout, does emerge output some more error info? Nope. Just '!!! Timed out while closing connection to binhost' I'll try an another software. Might take some time. Meanwhile I tested with a simple python script. Worked fine. Created attachment 258496 [details]
A simple python script for ftp testing
Did ran with python2.7:
$ python2.7 ftptest.py
open connection
cwd packages
open Packages
get Packages
close Packages
close connection
Created attachment 258497 [details]
three repeated successes with ftptest.py
(In reply to comment #4) > Timeout works. Download still fails unless the local copy of Packages is > deleted. I'll post ftp logs.. It's supposed to use the cached copy of Packages. Does it contain the correct content or not? (In reply to comment #11) > It's supposed to use the cached copy of Packages. Does it contain the correct > content or not? Yes, diff returns empty when compared to server's copy. Emerge updates cached copy correctly (not freezing) when server side gets updated. I've tested locally with pure-ftpd-1.0.29-r2 and an anonymous ftp account, and I didn't have any freeze. Please post full details about your server configuration. Created attachment 258504 [details]
pure-ftpd conf
Client has
PORTAGE_BINHOST="ftp://<user>:<pass>@<server>/"
(In reply to comment #7) > Maybe try another ftp server program? I tried with an another server and net-ftp/oftpd. Works fine. Pure-fptd seems to work with small files but freezes on files >100k when using urllib or urllib2. Ftplib works ok. I'd say that your patch fixes the problem fine. (In reply to comment #15) > I tried with an another server and net-ftp/oftpd. Works fine. > > Pure-fptd seems to work with small files but freezes on files >100k when using > urllib or urllib2. Ftplib works ok. My guess is that the interaction between urllib and pure-fptd is triggering some sort of deadlock in the communication protocol. It could be either one, both, or the protocol spec that's at fault. > I'd say that your patch fixes the problem fine. Great, then I'll mark this bug as fixed. The issue between urllib and pure-fptd is really beyond the scope of what I want to handle on the portage side. *** Bug 353189 has been marked as a duplicate of this bug. *** The ftp hang is supposed to be fixed in Python 3.3: http://bugs.python.org/issue11199 http://hg.python.org/cpython/file/bd8afb90ebf2/Misc/NEWS That fix is not included in Python 2.7.3 or 3.2.3, since their NEWS files do not mention it. I am seeing this issue re-occur when using python-2.7.5, while python-2.7.3 and python-3.2.5 work fine. Can anyone verify? Or is it just me. |