Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 394103

Summary: dev-lang/python-2.7.2-r3 fix for exception: "handle out of range in select()"
Product: Gentoo Linux Reporter: Keith Harrison <KeithBHarrison>
Component: [OLD] Core systemAssignee: Python Gentoo Team <python>
Severity: normal    
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Package list:
Runtime testing required: ---
Attachments: based on multiproc2.patch with minor changes

Description Keith Harrison 2011-12-09 06:49:31 UTC
This is bug/limitation when deploying software that has over 1023 file descriptors open AND when using a multiprocessing Pipe or Queue.

I am currently encountering this issue with software I am developing.

The patches on the site for python 3 apply and compile fine after minor modification of the directories in the patch header.  I will attach the patch I am currently using in my local repo.  I will give further feedback after more thorough testing.

Reproducible: Always

Steps to Reproduce:
1.  Open 1024 file descriptors
2.  Create a multiprocessing Queue and Process
3.  Attempt to read from and write to the Queue
Actual Results:  
IOException: "handle out of range in select()"

Expected Results:  
No exception, successful reading and writing to Queue

"I have a code that uses multiprocessing.Pipe to communicate with subprocesses. Spawning 500 subprocesses this way works like a charm, but when spawning about 600 of them the pipe ends raise the exception: "handle out of range in select()". I realized that this is because of the FD_SETSIZE limit. To address the situation I quickly hacked together a patch that uses poll() instead of select(), which solves the problem for me. I don't know the reason why select() was chosen for this task (maybe because of windows) but wouldn't it be better to use polling where possible?"
Comment 1 Keith Harrison 2011-12-09 06:53:13 UTC
Created attachment 295261 [details, diff]
based on multiproc2.patch with minor changes

based on multiproc2.patch with minor changes in the header for the directory names so that it would apply with portage for python 2.7
Comment 2 Arfrever Frehtes Taifersar Arahesis 2011-12-10 21:27:35 UTC
This change is not yet accepted by upstream. Please create a patch for Python 3.3 and send it to upstream.
Comment 3 Patrice Clement gentoo-dev 2016-10-10 20:35:08 UTC
Marked as FIXED by upstream in Closing.