Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bugzilla DB migration completed. Please report issues to Infra team via email via infra@gentoo.org or IRC
Bug 662340 - dev-python/twisted-18.7.0: test failures
Summary: dev-python/twisted-18.7.0: test failures
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords: TESTFAILURE
Depends on:
Blocks:
 
Reported: 2018-07-28 13:09 UTC by Thomas Deutschmann
Modified: 2019-11-28 17:20 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log.gz,127.67 KB, application/gzip)
2018-07-28 13:09 UTC, Thomas Deutschmann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Deutschmann gentoo-dev Security 2018-07-28 13:09:08 UTC
Created attachment 541512 [details]
build.log

test_closePeerOnEMFILE test is failing multiple times and test suite is failing with

[ERROR]
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/twisted-18.7.0/work/Twisted-18.7.0-python2_7/test/lib/Twisted-18.7.0-py2.7-linux-i686.egg/twisted/internet/base.py", line 896, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/var/tmp/portage/dev-python/twisted-18.7.0/work/Twisted-18.7.0-python2_7/test/lib/Twisted-18.7.0-py2.7-linux-i686.egg/twisted/internet/posixbase.py", line 298, in _handleSignals
    self._childWaker = _SIGCHLDWaker(self)
  File "/var/tmp/portage/dev-python/twisted-18.7.0/work/Twisted-18.7.0-python2_7/test/lib/Twisted-18.7.0-py2.7-linux-i686.egg/twisted/internet/posixbase.py", line 200, in __init__
    _FDWaker.__init__(self, reactor)
  File "/var/tmp/portage/dev-python/twisted-18.7.0/work/Twisted-18.7.0-python2_7/test/lib/Twisted-18.7.0-py2.7-linux-i686.egg/twisted/internet/posixbase.py", line 133, in __init__
    self.i, self.o = os.pipe()
exceptions.OSError: [Errno 24] Too many open files
Comment 1 Thomas Deutschmann gentoo-dev Security 2018-07-28 13:09:22 UTC
Portage 2.3.40 (python 3.6.5-final-0, default/linux/x86/17.0, gcc-7.3.0, glibc-2.26-r7, 4.14.52-gentoo i686)
=================================================================
System uname: Linux-4.14.52-gentoo-i686-Intel-R-_Core-TM-_i7-3770K_CPU_@_3.50GHz-with-gentoo-2.4.1
KiB Mem:     3107600 total,    997312 free
KiB Swap:     488276 total,    486988 free
Timestamp of repository gentoo: Sat, 28 Jul 2018 07:04:25 +0000
Head commit of repository gentoo: c2598d764aa0836607778c15bdb07d03c0497de5

sh bash 4.4_p12
ld GNU ld (Gentoo 2.30 p2) 2.30.0
app-shells/bash:          4.4_p12::gentoo
dev-lang/perl:            5.24.3-r1::gentoo
dev-lang/python:          2.7.14-r1::gentoo, 3.6.5::gentoo
dev-util/cmake:           3.9.6::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.13::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.15.1-r2::gentoo
sys-devel/binutils:       2.30-r2::gentoo
sys-devel/gcc:            7.3.0-r3::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers)
sys-libs/glibc:           2.26-r7::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000

ABI="x86"
ABI_X86="32"
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
ACCEPT_PROPERTIES="*"
ACCEPT_RESTRICT="*"
ARCH="x86"
BROOT=""
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=pentium4m -mtune=pentium4m"
CHOST="i686-pc-linux-gnu"
CHOST_x86="i686-pc-linux-gnu"
COLLISION_IGNORE="/lib/modules/* *.py[co] *$py.class */dropin.cache"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CPU_FLAGS_X86="mmx mmxext sse sse2"
CXXFLAGS="-O2 -pipe -march=pentium4m -mtune=pentium4m"
DEFAULT_ABI="x86"
EDITOR="/usr/bin/mcedit"
ELIBC="glibc"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
EPREFIX=""
EROOT="/"
ESYSROOT="/"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs cgroup config-protect-if-modified distlocks downgrade-backup ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -march=i686 -pipe"
GCC_SPECS=""
GRUB_PLATFORMS="efi-32 pc"
GSETTINGS_BACKEND="dconf"
HOME="/root"
INFOPATH="/usr/share/gcc-data/i686-pc-linux-gnu/7.3.0/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.30/info:/usr/share/info"
INPUT_DEVICES="libinput keyboard mouse"
IUSE_IMPLICIT="abi_x86_32 prefix prefix-chain prefix-guest"
KERNEL="linux"
L10N="en en-US de de-DE"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LC_MESSAGES="C"
LC_PAPER="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LIBDIR_x86="lib"
LINGUAS="en de"
LOGNAME="root"
MAIL="/var/mail/root"
MAKEOPTS="--jobs 5 --load-average 7.95"
MULTILIB_ABIS="x86"
NOCOLOR="true"
OFFICE_IMPLEMENTATION="libreoffice"
OLDPWD="/root"
OPENCL_PROFILE="mesa"
OPENGL_PROFILE="xorg-x11"
PAGER="/usr/bin/less"
PATH="/usr/i686-pc-linux-gnu/gcc-bin/7.3.0:/usr/lib/llvm/6/bin:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
PHP_TARGETS="php5-6 php7-0"
POSTGRES_TARGETS="postgres9_5 postgres10"
PWD="/tmp"
PYTHONDONTWRITEBYTECODE="1"
PYTHON_SINGLE_TARGET="python3_6"
PYTHON_TARGETS="python2_7 python3_6"
QT_GRAPHICSSYSTEM="raster"
ROOT="/"
ROOTPATH="/usr/i686-pc-linux-gnu/gcc-bin/7.3.0:/usr/lib/llvm/6/bin:/usr/lib/llvm/5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin"
RUBY_TARGETS="ruby22 ruby23"
SHELL="/bin/bash"
SHLVL="2"
SSH_TTY="/dev/pts/0"
SYSROOT="/"
TERM="tmux-256color"
TMUX="/tmp/tmux-0/default,5404,0"
TMUX_PANE="%3"
TWISTED_DISABLE_WRITING_OF_PLUGIN_CACHE="1"
USER="root"
USERLAND="GNU"
VIDEO_CARDS="vmware"
Comment 2 Brian Dolbec gentoo-dev 2018-07-28 15:27:45 UTC
Tests pass here on multiple systems.

The full test suite can only run when you run them as a user after install.

run "trial twisted" or "python2.7 /usr/bin/trial twisted" as a user to run the full test suite.  Many tests are disabled for FEATURES=test due to network sandbox and the fact they are being run as root.

What is the output of:  $ ulimit -n

I have the stock 1024 on my systems without experiencing this issue.

Perhaps you had too many other apps/emerges/... running at the test run time


On python2.7 there are 2 expected unicode test fails due to utf-8, their test suite is not expecting (designed for) a utf-8 based system so the results don't match even though they are just a different form of the same characters.
Comment 3 Thomas Deutschmann gentoo-dev Security 2018-07-28 18:43:55 UTC
(In reply to Brian Dolbec from comment #2)
> What is the output of:  $ ulimit -n

Tried with default 1024, then I raised limit 4096, 10000, 100000 and 500000. Always the same.

Of course, this does only work if the ebuild tries to raise ulimit as well.

I.e. in percona-server ebuild I am doing https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-db/percona-server/percona-server-5.6.40.84.0.ebuild?id=2f9bf6dc3151d2844259ea264f89283195348eb7#n516 which only works when I call "ulimit -n <wantedValue>" in the shell from where I'll call emerge later.


> Perhaps you had too many other apps/emerges/... running at the test run time

No, only twisted-18.7.0 was running. I am currently able to reproduce.
Comment 4 Brian Dolbec gentoo-dev 2018-07-28 20:06:33 UTC
this is strange, I have run twisted tests on numerous systems without this issue.  Including an arm64 odroid-c2 2GB ram 16GB emmc.

going to re-test again now on the odroid while I'm gone out for the afternoon.

I re-tested already on my old core2 box without issue already.
Comment 5 Brian Dolbec gentoo-dev 2018-07-29 00:54:14 UTC
Again, only the expected few unicode errors.

brian@casper /var/tmp $ ulimit -n
1024


So, there has to be something on that system causing the problem.
Comment 6 Thomas Deutschmann gentoo-dev Security 2018-08-16 12:30:07 UTC
(In reply to Brian Dolbec from comment #5)
> So, there has to be something on that system causing the problem.

Still happening in a new stage3 on real x86 hardware.
Comment 7 Brian Dolbec gentoo-dev 2018-08-16 15:23:02 UTC
ugh...I am leaving on a holiday today.  I won't be able to do anything about it till September.

If you think it is an upstream issue, then please open an issue on twisted's issue tracker.
Comment 8 Larry the Git Cow gentoo-dev 2019-11-28 03:16:51 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=538d1343448d7d69ebc75afe857c6601eaa844fa

commit 538d1343448d7d69ebc75afe857c6601eaa844fa
Author:     Aaron Bauman <bman@gentoo.org>
AuthorDate: 2019-11-28 03:01:49 +0000
Commit:     Aaron Bauman <bman@gentoo.org>
CommitDate: 2019-11-28 03:16:34 +0000

    dev-python/twisted: bump to 19.10.0
    
    * Remove net-misc/openssh version constraint
    * dev-python/bcrypt is now in DEPEND
    * Drop all patches which are no longer required
    * Clean python_test() per previous comment
    
    Package is still not compatible with py3.8 but is very close. We should see full
    support soon. Most tests pass with py3.8. Additionally, dev-python/automat will
    require a bump to >=0.8.0 in order to support py3.8 within twisted.
    
    Bug: https://bugs.gentoo.org/661258
    Bug: https://bugs.gentoo.org/662340
    Bug: https://bugs.gentoo.org/677226
    Bug: https://bugs.gentoo.org/690790
    Bug: https://bugs.gentoo.org/683910
    
    Signed-off-by: Aaron Bauman <bman@gentoo.org>

 dev-python/twisted/Manifest               |   1 +
 dev-python/twisted/twisted-19.10.0.ebuild | 173 ++++++++++++++++++++++++++++++
 2 files changed, 174 insertions(+)