Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 394103 - dev-lang/python-2.7.2-r3 fix for exception: "handle out of range in select()"
Summary: dev-lang/python-2.7.2-r3 fix for exception: "handle out of range in select()"
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
Depends on:
Reported: 2011-12-09 06:49 UTC by Keith Harrison
Modified: 2016-10-10 20:35 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---

based on multiproc2.patch with minor changes (multiproc3.patch,2.00 KB, patch)
2011-12-09 06:53 UTC, Keith Harrison
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Keith Harrison 2011-12-09 06:49:31 UTC
This is bug/limitation when deploying software that has over 1023 file descriptors open AND when using a multiprocessing Pipe or Queue.

I am currently encountering this issue with software I am developing.

The patches on the site for python 3 apply and compile fine after minor modification of the directories in the patch header.  I will attach the patch I am currently using in my local repo.  I will give further feedback after more thorough testing.

Reproducible: Always

Steps to Reproduce:
1.  Open 1024 file descriptors
2.  Create a multiprocessing Queue and Process
3.  Attempt to read from and write to the Queue
Actual Results:  
IOException: "handle out of range in select()"

Expected Results:  
No exception, successful reading and writing to Queue

"I have a code that uses multiprocessing.Pipe to communicate with subprocesses. Spawning 500 subprocesses this way works like a charm, but when spawning about 600 of them the pipe ends raise the exception: "handle out of range in select()". I realized that this is because of the FD_SETSIZE limit. To address the situation I quickly hacked together a patch that uses poll() instead of select(), which solves the problem for me. I don't know the reason why select() was chosen for this task (maybe because of windows) but wouldn't it be better to use polling where possible?"
Comment 1 Keith Harrison 2011-12-09 06:53:13 UTC
Created attachment 295261 [details, diff]
based on multiproc2.patch with minor changes

based on multiproc2.patch with minor changes in the header for the directory names so that it would apply with portage for python 2.7
Comment 2 Arfrever Frehtes Taifersar Arahesis 2011-12-10 21:27:35 UTC
This change is not yet accepted by upstream. Please create a patch for Python 3.3 and send it to upstream.
Comment 3 Patrice Clement gentoo-dev 2016-10-10 20:35:08 UTC
Marked as FIXED by upstream in Closing.