|Summary:||dev-lang/python-2.7.2-r3 fix for exception: "handle out of range in select()"|
|Product:||Gentoo Linux||Reporter:||Keith Harrison <KeithBHarrison>|
|Component:||[OLD] Core system||Assignee:||Python Gentoo Team <python>|
|Package list:||Runtime testing required:||---|
|Attachments:||based on multiproc2.patch with minor changes|
Description Keith Harrison 2011-12-09 06:49:31 UTC
This is bug/limitation when deploying software that has over 1023 file descriptors open AND when using a multiprocessing Pipe or Queue. I am currently encountering this issue with software I am developing. The patches on the site for python 3 apply and compile fine after minor modification of the directories in the patch header. I will attach the patch I am currently using in my local repo. I will give further feedback after more thorough testing. Reproducible: Always Steps to Reproduce: 1. Open 1024 file descriptors 2. Create a multiprocessing Queue and Process 3. Attempt to read from and write to the Queue Actual Results: IOException: "handle out of range in select()" Expected Results: No exception, successful reading and writing to Queue http://bugs.python.org/issue10527 "I have a code that uses multiprocessing.Pipe to communicate with subprocesses. Spawning 500 subprocesses this way works like a charm, but when spawning about 600 of them the pipe ends raise the exception: "handle out of range in select()". I realized that this is because of the FD_SETSIZE limit. To address the situation I quickly hacked together a patch that uses poll() instead of select(), which solves the problem for me. I don't know the reason why select() was chosen for this task (maybe because of windows) but wouldn't it be better to use polling where possible?"
Comment 1 Keith Harrison 2011-12-09 06:53:13 UTC
Created attachment 295261 [details, diff] based on multiproc2.patch with minor changes based on multiproc2.patch with minor changes in the header for the directory names so that it would apply with portage for python 2.7
Comment 2 Arfrever Frehtes Taifersar Arahesis 2011-12-10 21:27:35 UTC
This change is not yet accepted by upstream. Please create a patch for Python 3.3 and send it to upstream.