* Package: sci-chemistry/GromacsWrapper-0.8.0 * Repository: gentoo * Maintainer: alexxy@gentoo.org sci-chemistry@gentoo.org * USE: abi_x86_64 amd64 elibc_glibc kernel_linux python_targets_python3_6 python_targets_python3_7 test userland_GNU * FEATURES: network-sandbox preserve-libs sandbox test userpriv usersandbox @@@@@ PLEASE PAY ATTENTION HERE!!! @@@@@ This information may help you to understand if this is a duplicate or if this bug exists after you pushed a fix; This ebuild was merged at the following commit: https://github.com/gentoo-mirror/gentoo/commit/a25a6fcdd571968b88adaddb860c649a07a13cd4 (Tue Sep 8 14:35:15 UTC 2020) @@@@@ END @@@@@ emerge --info: Portage 3.0.6 (python 3.8.5-final-0, default/linux/amd64/17.1, gcc-10.2.0, glibc-2.32-r1, 5.4.0-1021-aws x86_64) ================================================================= System uname: Linux-5.4.0-1021-aws-x86_64-Intel-R-_Xeon-R-_Platinum_8175M_CPU_@_2.50GHz-with-glibc2.2.5 KiB Mem: 64359960 total, 25605692 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Tue, 08 Sep 2020 14:05:37 +0000 sh bash 5.0_p18 ld GNU ld (Gentoo 2.34 p6) 2.34.0 app-shells/bash: 5.0_p18::gentoo dev-lang/perl: 5.30.3-r1::gentoo dev-lang/python: 2.7.18-r1::gentoo, 3.6.12::gentoo, 3.7.9::gentoo, 3.8.5::gentoo, 3.9.0_rc1::gentoo dev-util/cmake: 3.18.2::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.20::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.16.2::gentoo sys-devel/binutils: 2.34-r2::gentoo sys-devel/gcc: 10.2.0-r1::gentoo sys-devel/gcc-config: 2.3.2::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.8::gentoo (virtual/os-headers) sys-libs/glibc: 2.32-r1::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: yes sync-rsync-verify-jobs: 1 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* GPL-3 LGPL-3" CBUILD="x86_64-pc-linux-gnu" CC="x86_64-pc-linux-gnu-clang" CFLAGS="-O2 -pipe -march=native -frecord-gcc-switches -fno-diagnostics-color" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXX="x86_64-pc-linux-gnu-clang++" CXXFLAGS="-O2 -pipe -march=native -frecord-gcc-switches -fno-diagnostics-color" DISTDIR="/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/distdir" EMERGE_DEFAULT_OPTS="--with-bdeps=y -1 -b -k" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -march=native -frecord-gcc-switches -fno-diagnostics-color" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms sign split-log strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=native -frecord-gcc-switches -fno-diagnostics-color" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="C.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0 -fuse-ld=lld" MAKEOPTS="-j16 V=1" PKGDIR="/root/.packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl amd64 berkdb bzip2 cli crypt dri elogind fortran gdbm iconv ipv6 jumbo-build libglvnd libtirpc multilib native-symlinks ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl tcpd test unicode xattr zlib" ABI_X86="64" ELIBC="glibc" KERNEL="linux" PYTHON_TARGETS="python3_6 python3_7" USERLAND="GNU" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS >>> Unpacking source... * Unpacking GromacsWrapper-0.8.0.tar.gz >>> Source unpacked in /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work >>> Preparing source in /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0 ... * Applying GromacsWrapper-0.8.0-tests-package.patch ... [ ok ] >>> Source prepared. >>> Configuring source in /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0 ... >>> Source configured. >>> Compiling source in /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0 ... * python3_6: running distutils-r1_run_phase distutils-r1_python_compile python3.6 setup.py build -j 16 running build running build_py creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/__init__.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/scaling.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/setup.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/qsub.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/_version.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/config.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/cbook.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/exceptions.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/run.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/tools.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/environment.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/core.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/collections.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/utilities.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/formats.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs copying gromacs/log.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/mdp.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/__init__.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/convert.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/top.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/xvg.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/ndx.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/xpm.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats copying gromacs/fileformats/blocks.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/fileformats creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/local.sh -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/darwin.sh -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/md_OPLSAA.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/md_CHARMM27_gpu.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/md_G43a1.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/md_CHARMM27.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/em.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/md_OPLSAA_gpu.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/gromacswrapper_465.cfg -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates copying gromacs/templates/gromacswrapper.cfg -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/templates warning: cmd_build_py: byte-compiling is disabled, skipping. UPDATING /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/_version.py set /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/lib/gromacs/_version.py to '0.8.0' running build_scripts creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts copying and adjusting scripts/gw-join_parts.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts copying and adjusting scripts/gw-merge_topologies.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts copying scripts/gw-forcefield.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts copying scripts/gw-partial_tempering.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts changing mode of /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts/gw-join_parts.py from 644 to 755 changing mode of /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_6/scripts/gw-merge_topologies.py from 644 to 755 * python3_7: running distutils-r1_run_phase distutils-r1_python_compile python3.7 setup.py build -j 16 running build running build_py creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/__init__.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/scaling.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/setup.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/qsub.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/_version.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/config.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/cbook.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/exceptions.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/run.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/tools.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/environment.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/core.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/collections.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/utilities.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/formats.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs copying gromacs/log.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/mdp.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/__init__.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/convert.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/top.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/xvg.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/ndx.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/xpm.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats copying gromacs/fileformats/blocks.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/fileformats creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/local.sh -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/darwin.sh -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/md_OPLSAA.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/md_CHARMM27_gpu.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/md_G43a1.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/md_CHARMM27.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/em.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/md_OPLSAA_gpu.mdp -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/gromacswrapper_465.cfg -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates copying gromacs/templates/gromacswrapper.cfg -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/templates warning: cmd_build_py: byte-compiling is disabled, skipping. UPDATING /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/_version.py set /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/lib/gromacs/_version.py to '0.8.0' running build_scripts creating /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts copying and adjusting scripts/gw-join_parts.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts copying and adjusting scripts/gw-merge_topologies.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts copying scripts/gw-forcefield.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts copying scripts/gw-partial_tempering.py -> /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts changing mode of /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts/gw-join_parts.py from 644 to 755 changing mode of /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0-python3_7/scripts/gw-merge_topologies.py from 644 to 755 >>> Source compiled. >>> Test phase: sci-chemistry/GromacsWrapper-0.8.0 * python3_6: running distutils-r1_run_phase python_test ============================= test session starts ============================== platform linux -- Python 3.6.12, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3.6 cachedir: .pytest_cache rootdir: /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0 collecting ... collected 252 items tests/test_cbook.py::test_grompp_qtot FAILED [ 0%] tests/test_collections.py::TestCollection::test_list_like[things0] PASSED [ 0%] tests/test_collections.py::TestCollection::test_list_like[things1] PASSED [ 1%] tests/test_collections.py::TestCollection::test_list_like[things2] PASSED [ 1%] tests/test_collections.py::TestCollection::test_list_like[things3] PASSED [ 1%] tests/test_collections.py::TestCollection::test_list_like[things4] PASSED [ 2%] tests/test_collections.py::TestCollection::test_tolist[things0] PASSED [ 2%] tests/test_collections.py::TestCollection::test_tolist[things1] PASSED [ 3%] tests/test_collections.py::TestCollection::test_tolist[things2] PASSED [ 3%] tests/test_collections.py::TestCollection::test_tolist[things3] PASSED [ 3%] tests/test_collections.py::TestCollection::test_tolist[things4] PASSED [ 4%] tests/test_collections.py::TestCollection::test_save_load[things0] PASSED [ 4%] tests/test_collections.py::TestCollection::test_save_load[things1] PASSED [ 5%] tests/test_collections.py::TestCollection::test_save_load[things2] PASSED [ 5%] tests/test_collections.py::TestCollection::test_save_load[things3] PASSED [ 5%] tests/test_collections.py::TestCollection::test_save_load[things4] PASSED [ 6%] tests/test_collections.py::TestCollection::test_method_pass_through[textthings0-startswith-args0] PASSED [ 6%] tests/test_collections.py::TestCollection::test_method_pass_through[textthings0-upper-args1] PASSED [ 7%] tests/test_collections.py::TestCollection::test_method_pass_through[textthings0-capitalize-args2] PASSED [ 7%] tests/test_collections.py::TestCollection::test_method_pass_through[textthings1-startswith-args0] PASSED [ 7%] tests/test_collections.py::TestCollection::test_method_pass_through[textthings1-upper-args1] PASSED [ 8%] tests/test_collections.py::TestCollection::test_method_pass_through[textthings1-capitalize-args2] PASSED [ 8%] tests/test_collections.py::TestCollection::test_attribute_pass_through[textthings0-__doc__] PASSED [ 9%] tests/test_collections.py::TestCollection::test_attribute_pass_through[textthings1-__doc__] PASSED [ 9%] tests/test_collections.py::TestCollection::test_add[textthings0] PASSED [ 9%] tests/test_collections.py::TestCollection::test_add[textthings1] PASSED [ 10%] tests/test_config.py::test_set_gmxrc_environment ERROR [ 10%] tests/test_config.py::test_check_setup PASSED [ 11%] tests/test_config.py::test_get_configuration PASSED [ 11%] tests/test_config.py::test_modified_config PASSED [ 11%] tests/test_config.py::test_get_boolean PASSED [ 12%] tests/test_core.py::TestCommand::test_run_default PASSED [ 12%] tests/test_core.py::TestCommand::test_run_with_args PASSED [ 13%] tests/test_core.py::TestCommand::test_run_capture_stdout PASSED [ 13%] tests/test_core.py::TestCommand::test_run_capture_stderr PASSED [ 13%] tests/test_core.py::TestCommand::test_run_with_input[not_used] PASSED [ 14%] tests/test_core.py::TestCommand::test_run_with_input[inp1] PASSED [ 14%] tests/test_core.py::TestCommand::test_run_with_input[inp2] PASSED [ 15%] tests/test_core.py::TestCommand::test_Popen_with_input[not_used] PASSED [ 15%] tests/test_core.py::TestCommand::test_Popen_with_input[inp1] PASSED [ 15%] tests/test_core.py::TestCommand::test_Popen_with_input[inp2] PASSED [ 16%] tests/test_core.py::TestCommand::test_help_short PASSED [ 16%] tests/test_core.py::TestCommand::test_help_long PASSED [ 17%] tests/test_log.py::test_create PASSED [ 17%] tests/test_log.py::test_clear_handlers PASSED [ 17%] tests/test_log.py::test_NullHandler PASSED [ 18%] tests/test_qsub.py::test_queuing_systems PASSED [ 18%] tests/test_qsub.py::test_detect_queuing_system[foo.sge-Sun Gridengine] PASSED [ 19%] tests/test_qsub.py::test_detect_queuing_system[foo.pbs-PBS] PASSED [ 19%] tests/test_qsub.py::test_detect_queuing_system[foo.ll-LoadLeveler] PASSED [ 19%] tests/test_qsub.py::test_detect_queuing_system[foo.slu-Slurm] PASSED [ 20%] tests/test_qsub.py::test_generate_submit_scripts PASSED [ 20%] tests/test_run.py::Test_check_mdrun_success::test_no_logfile PASSED [ 21%] tests/test_run.py::Test_check_mdrun_success::test_success_Gromacs4 PASSED [ 21%] tests/test_run.py::Test_check_mdrun_success::test_incomplete_Gromacs4 PASSED [ 21%] tests/test_run.py::Test_check_mdrun_success::test_success_Gromacs5 PASSED [ 22%] tests/test_run.py::Test_check_mdrun_success::test_incomplete_Gromacs5 PASSED [ 22%] tests/test_run.py::test_MDRunner FAILED [ 23%] tests/test_run.py::Test_find_gromacs_command::test_find FAILED [ 23%] tests/test_run.py::Test_find_gromacs_command::test_raises_ValueError PASSED [ 23%] tests/test_run.py::test_get_double_or_single_prec_mdrun PASSED [ 24%] tests/test_setup.py::test_trj_compact_main FAILED [ 24%] tests/test_setup.py::test_topology ERROR [ 25%] tests/test_setup.py::test_solvate ERROR [ 25%] tests/test_setup.py::test_energy_minimize ERROR [ 25%] tests/test_setup.py::test_energy_minimize_custom_mdp ERROR [ 26%] tests/test_tools.py::test_tools_help[trjconv] FAILED [ 26%] tests/test_tools.py::test_tools_help[gmxcheck] FAILED [ 26%] tests/test_tools.py::test_tools_help[trjorder] FAILED [ 27%] tests/test_tools.py::test_tools_help[g_sgangle] FAILED [ 27%] tests/test_tools.py::test_tools_help[genconf] FAILED [ 28%] tests/test_tools.py::test_tools_help[grompp] FAILED [ 28%] tests/test_tools.py::test_tools_help[do_dssp] FAILED [ 28%] tests/test_tools.py::test_tools_help[make_ndx] FAILED [ 29%] tests/test_tools.py::test_tools_help[tpbconv] FAILED [ 29%] tests/test_tools.py::test_tools_help[genion] FAILED [ 30%] tests/test_tools.py::test_tools_help[eneconv] FAILED [ 30%] tests/test_tools.py::test_tools_help[editconf] FAILED [ 30%] tests/test_tools.py::test_tools_help[make_edi] FAILED [ 31%] tests/test_tools.py::test_tools_help[gmxdump] FAILED [ 31%] tests/test_tools.py::test_tools_help[g_sas] FAILED [ 32%] tests/test_tools.py::test_tools_help[pdb2gmx] FAILED [ 32%] tests/test_tools.py::test_tools_help[genrestr] FAILED [ 32%] tests/test_tools.py::test_tools_help[g_dist] FAILED [ 33%] tests/test_tools.py::test_tools_help[mdrun] FAILED [ 33%] tests/test_tools.py::test_tools_help[trjcat] FAILED [ 34%] tests/test_tools.py::test_tools_help[xpm2ps] FAILED [ 34%] tests/test_tools.py::test_tools_help[genbox] FAILED [ 34%] tests/test_tools.py::test_failure_raises FAILED [ 35%] tests/test_tools.py::test_failure_warns FAILED [ 35%] tests/test_tools.py::test_failure_ignore FAILED [ 36%] tests/test_tools.py::TestRelease::test_release FAILED [ 36%] tests/test_tools.py::TestRelease::test_release_startswith FAILED [ 36%] tests/test_tools.py::TestRelease::test_str FAILED [ 37%] tests/test_utilities.py::test_which PASSED [ 37%] tests/test_utilities.py::test_realpath[~/whatever] PASSED [ 38%] tests/test_utilities.py::test_realpath[$HOME/whatever] PASSED [ 38%] tests/test_utilities.py::TestAttributeDict::test_attribute_get PASSED [ 38%] tests/test_utilities.py::TestAttributeDict::test_dict_get PASSED [ 39%] tests/test_utilities.py::TestAttributeDict::test_attribute_set PASSED [ 39%] tests/test_utilities.py::TestAttributeDict::test_dict_set PASSED [ 40%] tests/test_utilities.py::TestAttributeDict::test_pickle PASSED [ 40%] tests/test_utilities.py::test_autoconvert[int -> int] PASSED [ 40%] tests/test_utilities.py::test_autoconvert[str -> int] PASSED [ 41%] tests/test_utilities.py::test_autoconvert[int list -> int list] PASSED [ 41%] tests/test_utilities.py::test_autoconvert[float -> float] PASSED [ 42%] tests/test_utilities.py::test_autoconvert[str -> float] PASSED [ 42%] tests/test_utilities.py::test_autoconvert[float list -> float list] PASSED [ 42%] tests/test_utilities.py::test_autoconvert[str -> str] PASSED [ 43%] tests/test_utilities.py::test_autoconvert[str list -> str list] PASSED [ 43%] tests/test_utilities.py::test_autoconvert[str -> int list] PASSED [ 44%] tests/test_utilities.py::test_autoconvert[str -> float list] PASSED [ 44%] tests/test_utilities.py::test_autoconvert[str -> str list] PASSED [ 44%] tests/test_utilities.py::TestOpenAny::test_file[] PASSED [ 45%] tests/test_utilities.py::TestOpenAny::test_file[gz] PASSED [ 45%] tests/test_utilities.py::TestOpenAny::test_file[bz2] PASSED [ 46%] tests/test_utilities.py::TestOpenAny::test_stream_write PASSED [ 46%] tests/test_utilities.py::TestOpenAny::test_stream_read PASSED [ 46%] tests/test_utilities.py::test_number_pdbs[args0] PASSED [ 47%] tests/test_utilities.py::test_number_pdbs[args1] PASSED [ 47%] tests/test_utilities.py::test_number_pdbs[args2] PASSED [ 48%] tests/test_utilities.py::test_number_pdbs[args3] PASSED [ 48%] tests/test_utilities.py::test_cat PASSED [ 48%] tests/test_utilities.py::test_cat_fail PASSED [ 49%] tests/test_utilities.py::test_cat_None_out PASSED [ 49%] tests/test_utilities.py::test_cat_None_in PASSED [ 50%] tests/test_utilities.py::test_unlink PASSED [ 50%] tests/test_utilities.py::test_unlink_nonexistant PASSED [ 50%] tests/test_utilities.py::test_unlink_gmx_backups PASSED [ 51%] tests/test_utilities.py::test_unlink_gmx PASSED [ 51%] tests/test_utilities.py::test_firstof[this-this] PASSED [ 51%] tests/test_utilities.py::test_firstof[iterable1-this] PASSED [ 52%] tests/test_utilities.py::test_firstof[iterable2-1] PASSED [ 52%] tests/test_utilities.py::test_firstof[iterable3-0] PASSED [ 53%] tests/test_utilities.py::test_conv_aa_code[a-ALA] PASSED [ 53%] tests/test_utilities.py::test_conv_aa_code[A-ALA] PASSED [ 53%] tests/test_utilities.py::test_conv_aa_code[ala-A] PASSED [ 54%] tests/test_utilities.py::test_conv_aa_code[ALA-A] PASSED [ 54%] tests/test_utilities.py::test_conv_aa_code[Ala-A] PASSED [ 55%] tests/test_utilities.py::test_conv_aa_code[Q-GLN] PASSED [ 55%] tests/test_utilities.py::test_conv_aa_code[q-GLN] PASSED [ 55%] tests/test_utilities.py::test_conv_aa_code[GLN-Q] PASSED [ 56%] tests/test_utilities.py::test_conv_aa_code[gln-Q] PASSED [ 56%] tests/test_utilities.py::test_conv_aa_code[Gln-Q] PASSED [ 57%] tests/test_utilities.py::test_conv_aa_code_VE[ALAA] PASSED [ 57%] tests/test_utilities.py::test_conv_aa_code_VE[] PASSED [ 57%] tests/test_utilities.py::test_FileUtils_filename[None-None-simple] PASSED [ 58%] tests/test_utilities.py::test_FileUtils_filename[other-None-other] PASSED [ 58%] tests/test_utilities.py::test_FileUtils_filename[None-pdf-simple.pdf] PASSED [ 59%] tests/test_utilities.py::test_FileUtils_filename[other-pdf-other.pdf] PASSED [ 59%] tests/test_utilities.py::test_FileUtils_filename_VE PASSED [ 59%] tests/test_utilities.py::test_check_file_exists_ignore[exists.txt] PASSED [ 60%] tests/test_utilities.py::test_check_file_exists_ignore[nonexistant.txt] PASSED [ 60%] tests/test_utilities.py::test_check_file_exists_force[exists.txt] PASSED [ 61%] tests/test_utilities.py::test_check_file_exists_force[nonexistant.txt] PASSED [ 61%] tests/test_utilities.py::test_check_file_exists_indicate[exists.txt-True] PASSED [ 61%] tests/test_utilities.py::test_check_file_exists_indicate[nonexistant.txt-False] PASSED [ 62%] tests/test_utilities.py::test_check_file_exists_warn[warn] PASSED [ 62%] tests/test_utilities.py::test_check_file_exists_warn[warning] PASSED [ 63%] tests/test_utilities.py::test_check_file_exists_raise[exception] PASSED [ 63%] tests/test_utilities.py::test_check_file_exists_raise[raise] PASSED [ 63%] tests/test_version.py::test_version PASSED [ 64%] tests/fileformats/test_convert.py::test_to_unicode[100-100] PASSED [ 64%] tests/fileformats/test_convert.py::test_to_unicode[Jabberwock-Jabberwock] PASSED [ 65%] tests/fileformats/test_convert.py::test_to_unicode[\xc5ngstr\xf6m-\xc5ngstr\xf6m] PASSED [ 65%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[True-foo bar 22 boing ----expected0] PASSED [ 65%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[True-1 2 3 4-expected1] PASSED [ 66%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[True-1 2 3 4-expected2] PASSED [ 66%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[True-True x X yes Present-expected3] PASSED [ 67%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[True-False no - None none-expected4] PASSED [ 67%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[None-foo bar 22 boing ----expected0] PASSED [ 67%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[None-1 2 3 4-expected1] PASSED [ 68%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[None-1 2 3 4-expected2] PASSED [ 68%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[None-True x X yes Present-expected3] PASSED [ 69%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default[None-False no - None none-expected4] PASSED [ 69%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default_sep[1,2,3,4-expected0] PASSED [ 69%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_default_sep[1 2,3,4-expected1] PASSED [ 70%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_numbers[True-2.71213 3.14-expected0] PASSED [ 70%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_numbers[True-1000 -234 987654-expected1] PASSED [ 71%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_numbers[None-2.71213 3.14-expected0] PASSED [ 71%] tests/fileformats/test_convert.py::TestAutoconverter::test_convert_numbers[None-1000 -234 987654-expected1] PASSED [ 71%] tests/fileformats/test_mdp.py::TestMDP::test_values[original] PASSED [ 72%] tests/fileformats/test_mdp.py::TestMDP::test_values[written] PASSED [ 72%] tests/fileformats/test_mdp.py::TestMDP::test_values[no_autoconvert] PASSED [ 73%] tests/fileformats/test_mdp.py::TestMDP::test_comments[original] PASSED [ 73%] tests/fileformats/test_mdp.py::TestMDP::test_comments[written] PASSED [ 73%] tests/fileformats/test_mdp.py::TestMDP::test_comments[no_autoconvert] PASSED [ 74%] tests/fileformats/test_mdp.py::TestMDP::test_blank_lines[original] PASSED [ 74%] tests/fileformats/test_mdp.py::TestMDP::test_blank_lines[written] PASSED [ 75%] tests/fileformats/test_mdp.py::TestMDP::test_blank_lines[no_autoconvert] PASSED [ 75%] tests/fileformats/test_mdp.py::TestMDP::test_no_filename PASSED [ 75%] tests/fileformats/test_mdp.py::test_bad_mdp PASSED [ 76%] tests/fileformats/test_ndx.py::test_read[original] PASSED [ 76%] tests/fileformats/test_ndx.py::test_read[nofilename] PASSED [ 76%] tests/fileformats/test_ndx.py::test_read[written] PASSED [ 77%] tests/fileformats/test_ndx.py::test_get[original] PASSED [ 77%] tests/fileformats/test_ndx.py::test_get[nofilename] PASSED [ 78%] tests/fileformats/test_ndx.py::test_get[written] PASSED [ 78%] tests/fileformats/test_ndx.py::test_set[original] PASSED [ 78%] tests/fileformats/test_ndx.py::test_set[nofilename] PASSED [ 79%] tests/fileformats/test_ndx.py::test_set[written] PASSED [ 79%] tests/fileformats/test_ndx.py::test_size[original] PASSED [ 80%] tests/fileformats/test_ndx.py::test_size[nofilename] PASSED [ 80%] tests/fileformats/test_ndx.py::test_size[written] PASSED [ 80%] tests/fileformats/test_ndx.py::test_sizes[original] PASSED [ 81%] tests/fileformats/test_ndx.py::test_sizes[nofilename] PASSED [ 81%] tests/fileformats/test_ndx.py::test_sizes[written] PASSED [ 82%] tests/fileformats/test_ndx.py::test_groups[original] PASSED [ 82%] tests/fileformats/test_ndx.py::test_groups[nofilename] PASSED [ 82%] tests/fileformats/test_ndx.py::test_groups[written] PASSED [ 83%] tests/fileformats/test_xpm.py::TestXPM::test_constructor PASSED [ 83%] tests/fileformats/test_xpm.py::TestXPM::test_read PASSED [ 84%] tests/fileformats/test_xpm.py::TestXPM::test_reversed_by_default PASSED [ 84%] tests/fileformats/test_xpm.py::TestXPM::test_to_pd FAILED [ 84%] tests/fileformats/test_xvg.py::TestXVG_array::test_names PASSED [ 85%] tests/fileformats/test_xvg.py::TestXVG_array::test_array PASSED [ 85%] tests/fileformats/test_xvg.py::TestXVG_array::test_props[mean] PASSED [ 86%] tests/fileformats/test_xvg.py::TestXVG_array::test_props[max] PASSED [ 86%] tests/fileformats/test_xvg.py::TestXVG_array::test_props[min] PASSED [ 86%] tests/fileformats/test_xvg.py::TestXVG_array::test_props[std] PASSED [ 87%] tests/fileformats/test_xvg.py::TestXVG_array::test_write_read PASSED [ 87%] tests/fileformats/test_xvg.py::TestXVG_array::test_correl PASSED [ 88%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[mean] PASSED [ 88%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[circmean] PASSED [ 88%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[min] PASSED [ 89%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[max] PASSED [ 89%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[rms] PASSED [ 90%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[percentile] PASSED [ 90%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[smooth] PASSED [ 90%] tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[error] PASSED [ 91%] tests/fileformats/test_xvg.py::TestXVG_array::test_plot FAILED [ 91%] tests/fileformats/test_xvg.py::TestXVG_array::test_errorplot PASSED [ 92%] tests/fileformats/test_xvg.py::TestXVG_array::test_plot_coarsend FAILED [ 92%] tests/fileformats/test_xvg.py::test_break_array PASSED [ 92%] tests/fileformats/top/test_amber03star.py::TestAmber03star::test_basic <- tests/fileformats/top/top.py PASSED [ 93%] tests/fileformats/top/test_amber03star.py::TestAmber03star::test_equal <- tests/fileformats/top/top.py PASSED [ 93%] tests/fileformats/top/test_amber03star.py::TestAmber03star::test_read <- tests/fileformats/top/top.py PASSED [ 94%] tests/fileformats/top/test_amber03star.py::TestAmber03star::test_read_write <- tests/fileformats/top/top.py PASSED [ 94%] tests/fileformats/top/test_amber03star.py::TestAmber03star::test_grompp <- tests/fileformats/top/top.py FAILED [ 94%] tests/fileformats/top/test_amber03star.py::TestAmber03star::test_mdrun <- tests/fileformats/top/top.py FAILED [ 95%] tests/fileformats/top/test_amber03w.py::TestAmber03w::test_basic <- tests/fileformats/top/top.py PASSED [ 95%] tests/fileformats/top/test_amber03w.py::TestAmber03w::test_equal <- tests/fileformats/top/top.py PASSED [ 96%] tests/fileformats/top/test_amber03w.py::TestAmber03w::test_read <- tests/fileformats/top/top.py PASSED [ 96%] tests/fileformats/top/test_amber03w.py::TestAmber03w::test_read_write <- tests/fileformats/top/top.py PASSED [ 96%] tests/fileformats/top/test_amber03w.py::TestAmber03w::test_grompp <- tests/fileformats/top/top.py FAILED [ 97%] tests/fileformats/top/test_amber03w.py::TestAmber03w::test_mdrun <- tests/fileformats/top/top.py FAILED [ 97%] tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_basic <- tests/fileformats/top/top.py PASSED [ 98%] tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_equal <- tests/fileformats/top/top.py PASSED [ 98%] tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_read <- tests/fileformats/top/top.py PASSED [ 98%] tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_read_write <- tests/fileformats/top/top.py PASSED [ 99%] tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_grompp <- tests/fileformats/top/top.py FAILED [ 99%] tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_mdrun <- tests/fileformats/top/top.py FAILED [100%] ==================================== ERRORS ==================================== _________________ ERROR at setup of test_set_gmxrc_environment _________________ @pytest.fixture def GMXRC(): # Try using GMXRC in config file: GMXRC = gromacs.config.cfg.get('Gromacs', 'gmxrc') if GMXRC: return GMXRC # get GMXRC from installed Gromacs conda package for gmxexe in ('gmx', 'gmx_d', 'gmx_mpi', 'gmx_mpi_d', 'grompp', 'mdrun'): path = gromacs.utilities.which(gmxexe) if path is not None: break else: > raise RuntimeError("Cannot find Gromacs installation") E RuntimeError: Cannot find Gromacs installation tests/test_config.py:36: RuntimeError _______________________ ERROR at setup of test_topology ________________________ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] bufsize = -1, executable = None, stdin = None, stdout = None, stderr = None preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] executable = b'pdb2gmx', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1 errwrite = -1, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: tmpdir_factory = TempdirFactory(_tmppath_factory=TempPathFactory(_given_basetemp=None, _trace=, _basetemp=PosixPath('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0'))) struct = '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb' @pytest.fixture(scope="session") def topology(tmpdir_factory, struct=datafile("1ake_A_protein.pdb")): # note: use protein-only input 1ake_A_protein.pdb because solvation fails # if crystal waters are included (in 1ake_A.pdb) TMPDIR = tmpdir_factory.mktemp('1ake') with TMPDIR.as_cwd(): > topol_args = gromacs.setup.topology(struct=struct, ff="oplsaa", water="tip4p") tests/test_setup.py:38: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/setup.py:215: in topology gromacs.pdb2gmx(**pdb2gmx_args) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log setup ------------------------------ ERROR gromacs.core:core.py:290 pdb2gmx -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb -o protein.pdb -p system.top -i posres.itp -ff oplsaa -water tip4p CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? ________________________ ERROR at setup of test_solvate ________________________ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] bufsize = -1, executable = None, stdin = None, stdout = None, stderr = None preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] executable = b'pdb2gmx', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1 errwrite = -1, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: tmpdir_factory = TempdirFactory(_tmppath_factory=TempPathFactory(_given_basetemp=None, _trace=, _basetemp=PosixPath('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0'))) struct = '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb' @pytest.fixture(scope="session") def topology(tmpdir_factory, struct=datafile("1ake_A_protein.pdb")): # note: use protein-only input 1ake_A_protein.pdb because solvation fails # if crystal waters are included (in 1ake_A.pdb) TMPDIR = tmpdir_factory.mktemp('1ake') with TMPDIR.as_cwd(): > topol_args = gromacs.setup.topology(struct=struct, ff="oplsaa", water="tip4p") tests/test_setup.py:38: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/setup.py:215: in topology gromacs.pdb2gmx(**pdb2gmx_args) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ____________________ ERROR at setup of test_energy_minimize ____________________ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] bufsize = -1, executable = None, stdin = None, stdout = None, stderr = None preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] executable = b'pdb2gmx', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1 errwrite = -1, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: tmpdir_factory = TempdirFactory(_tmppath_factory=TempPathFactory(_given_basetemp=None, _trace=, _basetemp=PosixPath('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0'))) struct = '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb' @pytest.fixture(scope="session") def topology(tmpdir_factory, struct=datafile("1ake_A_protein.pdb")): # note: use protein-only input 1ake_A_protein.pdb because solvation fails # if crystal waters are included (in 1ake_A.pdb) TMPDIR = tmpdir_factory.mktemp('1ake') with TMPDIR.as_cwd(): > topol_args = gromacs.setup.topology(struct=struct, ff="oplsaa", water="tip4p") tests/test_setup.py:38: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/setup.py:215: in topology gromacs.pdb2gmx(**pdb2gmx_args) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ______________ ERROR at setup of test_energy_minimize_custom_mdp _______________ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] bufsize = -1, executable = None, stdin = None, stdout = None, stderr = None preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] executable = b'pdb2gmx', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1 errwrite = -1, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: tmpdir_factory = TempdirFactory(_tmppath_factory=TempPathFactory(_given_basetemp=None, _trace=, _basetemp=PosixPath('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0'))) struct = '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb' @pytest.fixture(scope="session") def topology(tmpdir_factory, struct=datafile("1ake_A_protein.pdb")): # note: use protein-only input 1ake_A_protein.pdb because solvation fails # if crystal waters are included (in 1ake_A.pdb) TMPDIR = tmpdir_factory.mktemp('1ake') with TMPDIR.as_cwd(): > topol_args = gromacs.setup.topology(struct=struct, ff="oplsaa", water="tip4p") tests/test_setup.py:38: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/setup.py:215: in topology gromacs.pdb2gmx(**pdb2gmx_args) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A_protein.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError =================================== FAILURES =================================== _______________________________ test_grompp_qtot _______________________________ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] bufsize = -1, executable = None, stdin = None, stdout = None, stderr = None preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] executable = b'pdb2gmx', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1 errwrite = -1, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_grompp_qtot0') def test_grompp_qtot(tmpdir): pdb = datafile("1ake_A.pdb") top = tmpdir.mkdir("top") with top.as_cwd(): > f = gromacs.setup.topology(struct=pdb, ff="oplsaa", water="tip4p") tests/test_cbook.py:19: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/setup.py:215: in topology gromacs.pdb2gmx(**pdb2gmx_args) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 pdb2gmx -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb -o protein.pdb -p system.top -i posres.itp -ff oplsaa -water tip4p CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? ________________________________ test_MDRunner _________________________________ def test_MDRunner(): try: > mdrun = gromacs.run.MDrunner() tests/test_run.py:40: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = MDrunner(filename=None), dirname = '.', kwargs = {} def __init__(self, dirname=os.path.curdir, **kwargs): """Set up a simple run with ``mdrun``. :Keywords: *dirname* Change to this directory before launching the job. Input files must be supplied relative to this directory. *keywords* All other keword arguments are used to construct the :class:`~gromacs.tools.mdrun` commandline. Note that only keyword arguments are allowed. """ # run MD in this directory (input files must be relative to this dir!) self.dirname = dirname > self.driver, self.name = find_gromacs_command(self.mdrun) gromacs/run.py:138: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ commands = ('mdrun', 'gmx mdrun') def find_gromacs_command(commands): """Return *driver* and *name* of the first command that can be found on :envvar:`PATH`""" # We could try executing 'name' or 'driver name' but to keep things lean we # just check if the executables can be found and then hope for the best. commands = utilities.asiterable(commands) for command in commands: try: driver, name = command.split() except ValueError: driver, name = None, command executable = driver if driver else name if utilities.which(executable): break else: > raise OSError(errno.ENOENT, "No Gromacs executable found in", ", ".join(commands)) E FileNotFoundError: [Errno 2] No Gromacs executable found in: 'mdrun, gmx mdrun' gromacs/run.py:69: FileNotFoundError During handling of the above exception, another exception occurred: def test_MDRunner(): try: mdrun = gromacs.run.MDrunner() except OSError: > raise RuntimeError("This test requires a Gromacs environment.") E RuntimeError: This test requires a Gromacs environment. tests/test_run.py:42: RuntimeError _____________________ Test_find_gromacs_command.test_find ______________________ self = def test_find(self): > driver, name = gromacs.run.find_gromacs_command(self.commands) tests/test_run.py:52: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ commands = ['grompp', 'gmx grompp'] def find_gromacs_command(commands): """Return *driver* and *name* of the first command that can be found on :envvar:`PATH`""" # We could try executing 'name' or 'driver name' but to keep things lean we # just check if the executables can be found and then hope for the best. commands = utilities.asiterable(commands) for command in commands: try: driver, name = command.split() except ValueError: driver, name = None, command executable = driver if driver else name if utilities.which(executable): break else: > raise OSError(errno.ENOENT, "No Gromacs executable found in", ", ".join(commands)) E FileNotFoundError: [Errno 2] No Gromacs executable found in: 'grompp, gmx grompp' gromacs/run.py:69: FileNotFoundError ____________________________ test_trj_compact_main _____________________________ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] bufsize = -1, executable = None, stdin = None, stdout = None, stderr = None preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] executable = b'pdb2gmx', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1 errwrite = -1, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_trj_compact_main0') def test_trj_compact_main(tmpdir): pdb = datafile("1ake_A.pdb") top = tmpdir.mkdir("top") mdpfile = "simple.mdp" tprfile = "simple.tpr" outfile = "compact.pdb" with top.as_cwd(): > f = gromacs.setup.topology(struct=pdb, ff="oplsaa", water="tip4p") tests/test_setup.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/setup.py:215: in topology gromacs.pdb2gmx(**pdb2gmx_args) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'f': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', 'ff': 'oplsaa', 'i': 'posres.itp', 'o': 'protein.pdb', ...} stderr = None, stdout = None, stdin = None, input = None, use_shell = False cmd = ['pdb2gmx', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb', '-o', 'protein.pdb', '-p', ...] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 pdb2gmx -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/1ake_A.pdb -o protein.pdb -p system.top -i posres.itp -ff oplsaa -water tip4p CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[trjconv] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['trjconv', '-h'] errmsg = "Failed to find Gromacs command 'trjconv', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['trjconv', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['trjconv', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['trjconv', '-h'], executable = b'trjconv', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'trjconv': 'trjconv' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['trjconv', '-h'] errmsg = "Failed to find Gromacs command 'trjconv', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'trjconv', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 trjconv -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'trjconv', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[gmxcheck] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['gmxcheck', '-h'] errmsg = "Failed to find Gromacs command 'gmxcheck', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['gmxcheck', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['gmxcheck', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['gmxcheck', '-h'], executable = b'gmxcheck', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'gmxcheck': 'gmxcheck' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['gmxcheck', '-h'] errmsg = "Failed to find Gromacs command 'gmxcheck', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'gmxcheck', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 gmxcheck -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'gmxcheck', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[trjorder] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['trjorder', '-h'] errmsg = "Failed to find Gromacs command 'trjorder', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['trjorder', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['trjorder', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['trjorder', '-h'], executable = b'trjorder', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'trjorder': 'trjorder' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['trjorder', '-h'] errmsg = "Failed to find Gromacs command 'trjorder', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'trjorder', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 trjorder -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'trjorder', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[g_sgangle] __________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['g_sgangle', '-h'] errmsg = "Failed to find Gromacs command 'g_sgangle', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['g_sgangle', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['g_sgangle', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['g_sgangle', '-h'], executable = b'g_sgangle', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'g_sgangle': 'g_sgangle' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['g_sgangle', '-h'] errmsg = "Failed to find Gromacs command 'g_sgangle', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'g_sgangle', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 g_sgangle -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'g_sgangle', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[genconf] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genconf', '-h'] errmsg = "Failed to find Gromacs command 'genconf', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['genconf', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genconf', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genconf', '-h'], executable = b'genconf', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'genconf': 'genconf' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genconf', '-h'] errmsg = "Failed to find Gromacs command 'genconf', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'genconf', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 genconf -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'genconf', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[grompp] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['grompp', '-h'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-h'], executable = b'grompp', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['grompp', '-h'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[do_dssp] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['do_dssp', '-h'] errmsg = "Failed to find Gromacs command 'do_dssp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['do_dssp', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['do_dssp', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['do_dssp', '-h'], executable = b'do_dssp', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'do_dssp': 'do_dssp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['do_dssp', '-h'] errmsg = "Failed to find Gromacs command 'do_dssp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'do_dssp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 do_dssp -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'do_dssp', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[make_ndx] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['make_ndx', '-h'] errmsg = "Failed to find Gromacs command 'make_ndx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['make_ndx', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['make_ndx', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['make_ndx', '-h'], executable = b'make_ndx', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'make_ndx': 'make_ndx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['make_ndx', '-h'] errmsg = "Failed to find Gromacs command 'make_ndx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'make_ndx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 make_ndx -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'make_ndx', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[tpbconv] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['tpbconv', '-h'] errmsg = "Failed to find Gromacs command 'tpbconv', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['tpbconv', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['tpbconv', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['tpbconv', '-h'], executable = b'tpbconv', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'tpbconv': 'tpbconv' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['tpbconv', '-h'] errmsg = "Failed to find Gromacs command 'tpbconv', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'tpbconv', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 tpbconv -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'tpbconv', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[genion] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genion', '-h'] errmsg = "Failed to find Gromacs command 'genion', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['genion', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genion', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genion', '-h'], executable = b'genion', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'genion': 'genion' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genion', '-h'] errmsg = "Failed to find Gromacs command 'genion', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'genion', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 genion -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'genion', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[eneconv] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['eneconv', '-h'] errmsg = "Failed to find Gromacs command 'eneconv', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['eneconv', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['eneconv', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['eneconv', '-h'], executable = b'eneconv', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'eneconv': 'eneconv' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['eneconv', '-h'] errmsg = "Failed to find Gromacs command 'eneconv', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'eneconv', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 eneconv -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'eneconv', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[editconf] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['editconf', '-h'] errmsg = "Failed to find Gromacs command 'editconf', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['editconf', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['editconf', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['editconf', '-h'], executable = b'editconf', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'editconf': 'editconf' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['editconf', '-h'] errmsg = "Failed to find Gromacs command 'editconf', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'editconf', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 editconf -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'editconf', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[make_edi] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['make_edi', '-h'] errmsg = "Failed to find Gromacs command 'make_edi', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['make_edi', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['make_edi', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['make_edi', '-h'], executable = b'make_edi', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'make_edi': 'make_edi' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['make_edi', '-h'] errmsg = "Failed to find Gromacs command 'make_edi', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'make_edi', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 make_edi -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'make_edi', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[gmxdump] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['gmxdump', '-h'] errmsg = "Failed to find Gromacs command 'gmxdump', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['gmxdump', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['gmxdump', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['gmxdump', '-h'], executable = b'gmxdump', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'gmxdump': 'gmxdump' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['gmxdump', '-h'] errmsg = "Failed to find Gromacs command 'gmxdump', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'gmxdump', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 gmxdump -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'gmxdump', maybe its not on PATH or GMXRC must be sourced? ____________________________ test_tools_help[g_sas] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['g_sas', '-h'] errmsg = "Failed to find Gromacs command 'g_sas', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['g_sas', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['g_sas', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['g_sas', '-h'], executable = b'g_sas', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'g_sas': 'g_sas' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['g_sas', '-h'] errmsg = "Failed to find Gromacs command 'g_sas', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'g_sas', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 g_sas -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'g_sas', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[pdb2gmx] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['pdb2gmx', '-h'] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['pdb2gmx', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['pdb2gmx', '-h'], executable = b'pdb2gmx', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'pdb2gmx': 'pdb2gmx' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['pdb2gmx', '-h'] errmsg = "Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 pdb2gmx -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'pdb2gmx', maybe its not on PATH or GMXRC must be sourced? __________________________ test_tools_help[genrestr] ___________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genrestr', '-h'] errmsg = "Failed to find Gromacs command 'genrestr', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['genrestr', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genrestr', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genrestr', '-h'], executable = b'genrestr', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'genrestr': 'genrestr' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genrestr', '-h'] errmsg = "Failed to find Gromacs command 'genrestr', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'genrestr', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 genrestr -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'genrestr', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[g_dist] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['g_dist', '-h'] errmsg = "Failed to find Gromacs command 'g_dist', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['g_dist', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['g_dist', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['g_dist', '-h'], executable = b'g_dist', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'g_dist': 'g_dist' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/tools.py:202: in run return super(GromacsCommandMultiIndex, self).run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['g_dist', '-h'] errmsg = "Failed to find Gromacs command 'g_dist', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'g_dist', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 g_dist -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'g_dist', maybe its not on PATH or GMXRC must be sourced? ____________________________ test_tools_help[mdrun] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['mdrun', '-h'] errmsg = "Failed to find Gromacs command 'mdrun', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['mdrun', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['mdrun', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['mdrun', '-h'], executable = b'mdrun', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'mdrun': 'mdrun' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['mdrun', '-h'] errmsg = "Failed to find Gromacs command 'mdrun', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'mdrun', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 mdrun -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'mdrun', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[trjcat] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['trjcat', '-h'] errmsg = "Failed to find Gromacs command 'trjcat', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['trjcat', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['trjcat', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['trjcat', '-h'], executable = b'trjcat', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'trjcat': 'trjcat' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['trjcat', '-h'] errmsg = "Failed to find Gromacs command 'trjcat', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'trjcat', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 trjcat -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'trjcat', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[xpm2ps] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['xpm2ps', '-h'] errmsg = "Failed to find Gromacs command 'xpm2ps', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['xpm2ps', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['xpm2ps', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['xpm2ps', '-h'], executable = b'xpm2ps', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'xpm2ps': 'xpm2ps' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['xpm2ps', '-h'] errmsg = "Failed to find Gromacs command 'xpm2ps', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'xpm2ps', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 xpm2ps -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'xpm2ps', maybe its not on PATH or GMXRC must be sourced? ___________________________ test_tools_help[genbox] ____________________________ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genbox', '-h'] errmsg = "Failed to find Gromacs command 'genbox', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['genbox', '-h'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genbox', '-h'], bufsize = -1, executable = None, stdin = None stdout = -1, stderr = -1, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['genbox', '-h'], executable = b'genbox', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 11 c2pwrite = 12, errread = 13, errwrite = 14, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'genbox': 'genbox' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: gromacs_tool = def test_tools_help(gromacs_tool): > rc, out, err = gromacs_tool(h=True, stdout=False, stderr=False) tests/test_tools.py:24: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'h': True}, stderr = -1, stdout = -1, stdin = None, input = None use_shell = False, cmd = ['genbox', '-h'] errmsg = "Failed to find Gromacs command 'genbox', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'genbox', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 genbox -h CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'genbox', maybe its not on PATH or GMXRC must be sourced? _____________________________ test_failure_raises ______________________________ self = , args = () kwargs = {'y': True}, stderr = None, stdout = None, stdin = None, input = None use_shell = False, cmd = ['grompp', '-y'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-y'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-y'], bufsize = -1, executable = None, stdin = None stdout = None, stderr = None, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-y'], executable = b'grompp', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1 c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: def test_failure_raises(): # unknown option with pytest.raises(gromacs.GromacsError): > gromacs.grompp(y=True) tests/test_tools.py:33: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True}, stderr = None, stdout = None, stdin = None, input = None use_shell = False, cmd = ['grompp', '-y'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -y CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? ______________________________ test_failure_warns ______________________________ self = , args = () kwargs = {'y': True}, stderr = None, stdout = None, stdin = None, input = None use_shell = False, cmd = ['grompp', '-y'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-y'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-y'], bufsize = -1, executable = None, stdin = None stdout = None, stderr = None, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-y'], executable = b'grompp', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1 c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: def test_failure_warns(): # unknown option grompp_warn = gromacs.tools.Grompp(failure="warn") with pytest.warns(gromacs.GromacsFailureWarning): > grompp_warn(y=True) tests/test_tools.py:40: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True}, stderr = None, stdout = None, stdin = None, input = None use_shell = False, cmd = ['grompp', '-y'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -y CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? _____________________________ test_failure_ignore ______________________________ self = , args = () kwargs = {'y': True}, stderr = None, stdout = None, stdin = None, input = None use_shell = False, cmd = ['grompp', '-y'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-y'],) kwargs = {'close_fds': True, 'shell': False, 'stderr': None, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-y'], bufsize = -1, executable = None, stdin = None stdout = None, stderr = None, preexec_fn = None, close_fds = True, shell = False cwd = None, env = None, universal_newlines = True, startupinfo = None creationflags = 0, restore_signals = True, start_new_session = False pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-y'], executable = b'grompp', preexec_fn = None close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1 c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: def test_failure_ignore(): # unknown option grompp_ignore = gromacs.tools.Grompp(failure=None) try: > grompp_ignore(y=True) tests/test_tools.py:47: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True} def __call__(self,*args,**kwargs): """Run command with the given arguments:: rc,stdout,stderr = command(*args, input=None, **kwargs) All positional parameters *args* and all gromacs *kwargs* are passed on to the Gromacs command. input and output keywords allow communication with the process via the python subprocess module. :Arguments: *input* : string, sequence to be fed to the process' standard input; elements of a sequence are concatenated with newlines, including a trailing one [``None``] *stdin* ``None`` or automatically set to ``PIPE`` if input given [``None``] *stdout* how to handle the program's stdout stream [``None``] filehandle anything that behaves like a file object ``None`` or ``True`` to see output on screen ``False`` or ``PIPE`` returns the output as a string in the stdout parameter *stderr* how to handle the stderr stream [``None``] ``STDOUT`` merges standard error with the standard out stream ``False`` or ``PIPE`` returns the output as a string in the stderr return parameter ``None`` or ``True`` keeps it on stderr (and presumably on screen) Depending on the value of the GromacsWrapper flag :data:`gromacs.environment.flags```['capture_output']`` the above default behaviour can be different. All other kwargs are passed on to the Gromacs tool. :Returns: The shell return code rc of the command is always returned. Depending on the value of output, various strings are filled with output from the command. :Notes: In order to chain different commands via pipes one must use the special :class:`PopenWithInput` object (see :meth:`GromacsCommand.Popen` method) instead of the simple call described here and first construct the pipeline explicitly and then call the :meth:`PopenWithInput.communicate` method. ``STDOUT`` and ``PIPE`` are objects provided by the :mod:`subprocess` module. Any python stream can be provided and manipulated. This allows for chaining of commands. Use :: from subprocess import PIPE, STDOUT when requiring these special streams (and the special boolean switches ``True``/``False`` cannot do what you need.) (TODO: example for chaining commands) """ > return self.run(*args, **kwargs) gromacs/core.py:398: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True}, _args = (), _kwargs = {'y': True} def run(self, *args, **kwargs): """Run the command; args/kwargs are added or replace the ones given to the constructor.""" _args, _kwargs = self._combine_arglist(args, kwargs) > results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:178: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True} def _run_command(self,*args,**kwargs): """Execute the gromacs command; see the docs for __call__.""" > result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:610: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True}, use_input = True, capturefile = None def _run_command(self, *args, **kwargs): """Execute the command; see the docs for __call__. :Returns: a tuple of the *results* tuple ``(rc, stdout, stderr)`` and the :class:`Popen` instance. """ # hack to run command WITHOUT input (-h...) even though user defined # input (should have named it "ignore_input" with opposite values...) use_input = kwargs.pop('use_input', True) # logic for capturing output (see docs on I/O and the flags) capturefile = None if environment.flags['capture_output'] is True: # capture into Python vars (see subprocess.Popen.communicate()) kwargs.setdefault('stderr', PIPE) kwargs.setdefault('stdout', PIPE) elif environment.flags['capture_output'] == "file": if 'stdout' in kwargs and 'stderr' in kwargs: pass else: # XXX: not race or thread proof; potentially many commands write to the same file fn = environment.flags['capture_output_filename'] capturefile = file(fn, "w") # overwrite (clobber) capture file if 'stdout' in kwargs and 'stderr' not in kwargs: # special case of stdout used by code but stderr should be captured to file kwargs.setdefault('stderr', capturefile) else: # merge stderr with stdout and write stdout to file # (stderr comes *before* stdout in capture file, could split...) kwargs.setdefault('stderr', STDOUT) kwargs.setdefault('stdout', capturefile) try: > p = self.Popen(*args, **kwargs) gromacs/core.py:221: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'y': True}, stderr = None, stdout = None, stdin = None, input = None use_shell = False, cmd = ['grompp', '-y'] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError During handling of the above exception, another exception occurred: def test_failure_ignore(): # unknown option grompp_ignore = gromacs.tools.Grompp(failure=None) try: grompp_ignore(y=True) except Exception as err: > raise AssertionError("Should have ignored exception {}".format(err)) E AssertionError: Should have ignored exception Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? tests/test_tools.py:49: AssertionError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -y CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? ___________________________ TestRelease.test_release ___________________________ self = def test_release(self): > assert gromacs.release().startswith(('4', '2016', '2018', '2019')) E AttributeError: module 'gromacs' has no attribute 'release' tests/test_tools.py:53: AttributeError _____________________ TestRelease.test_release_startswith ______________________ self = def test_release_startswith(self): > assert gromacs.release.startswith(('4', '2016', '2018', '2019')) E AttributeError: module 'gromacs' has no attribute 'release' tests/test_tools.py:56: AttributeError _____________________________ TestRelease.test_str _____________________________ self = def test_str(self): > assert str(gromacs.release()) == gromacs.release() E AttributeError: module 'gromacs' has no attribute 'release' tests/test_tools.py:59: AttributeError ______________________________ TestXPM.test_to_pd ______________________________ self = xpm = XPM(filename='/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/ss.xpm') def test_to_pd(self, xpm): > df = xpm.to_df() tests/fileformats/test_xpm.py:40: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/fileformats/xpm.py:166: in to_df return df.convert_objects(convert_numeric='force') _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Time 1 2 3 ... 766 767 768 769 0 0 Coil A-Helix A-Helix ... A-Helix ...A-Helix A-Helix Coil 499 499 Coil A-Helix A-Helix ... A-Helix A-Helix A-Helix Coil [500 rows x 770 columns] name = 'convert_objects' def __getattr__(self, name: str): """ After regular attribute access, try looking up the name This allows simpler access to columns for interactive use. """ # Note: obj.x will always call obj.__getattribute__('x') prior to # calling obj.__getattr__('x'). if ( name in self._internal_names_set or name in self._metadata or name in self._accessors ): return object.__getattribute__(self, name) else: if self._info_axis._can_hold_identifiers_and_holds_name(name): return self[name] > return object.__getattribute__(self, name) E AttributeError: 'DataFrame' object has no attribute 'convert_objects' /usr/lib64/python3.6/site-packages/pandas/core/generic.py:5130: AttributeError ___________________________ TestXVG_array.test_plot ____________________________ self = xvg = XVG(filename=None) def test_plot(self, xvg): > ax = xvg.plot() tests/fileformats/test_xvg.py:74: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/fileformats/xvg.py:615: in plot cmap = matplotlib.cm.get_cmap(color) /usr/lib64/python3.6/site-packages/matplotlib/cm.py:204: in get_cmap cbook._check_in_list(sorted(_cmap_registry), name=name) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _values = ['Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', ...] kwargs = {'name': ['black', 'red', 'blue', 'orange', 'magenta', 'cyan', ...]} values = ['Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', ...] k = 'name', v = ['black', 'red', 'blue', 'orange', 'magenta', 'cyan', ...] def _check_in_list(_values, **kwargs): """ For each *key, value* pair in *kwargs*, check that *value* is in *_values*; if not, raise an appropriate ValueError. Examples -------- >>> cbook._check_in_list(["foo", "bar"], arg=arg, other_arg=other_arg) """ values = _values for k, v in kwargs.items(): if v not in values: raise ValueError( "{!r} is not a valid value for {}; supported values are {}" > .format(v, k, ', '.join(map(repr, values)))) E ValueError: ['black', 'red', 'blue', 'orange', 'magenta', 'cyan', 'yellow', 'brown', 'green'] is not a valid value for name; supported values are 'Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', 'BuGn', 'BuGn_r', 'BuPu', 'BuPu_r', 'CMRmap', 'CMRmap_r', 'Dark2', 'Dark2_r', 'GnBu', 'GnBu_r', 'Greens', 'Greens_r', 'Greys', 'Greys_r', 'OrRd', 'OrRd_r', 'Oranges', 'Oranges_r', 'PRGn', 'PRGn_r', 'Paired', 'Paired_r', 'Pastel1', 'Pastel1_r', 'Pastel2', 'Pastel2_r', 'PiYG', 'PiYG_r', 'PuBu', 'PuBuGn', 'PuBuGn_r', 'PuBu_r', 'PuOr', 'PuOr_r', 'PuRd', 'PuRd_r', 'Purples', 'Purples_r', 'RdBu', 'RdBu_r', 'RdGy', 'RdGy_r', 'RdPu', 'RdPu_r', 'RdYlBu', 'RdYlBu_r', 'RdYlGn', 'RdYlGn_r', 'Reds', 'Reds_r', 'Set1', 'Set1_r', 'Set2', 'Set2_r', 'Set3', 'Set3_r', 'Spectral', 'Spectral_r', 'Wistia', 'Wistia_r', 'YlGn', 'YlGnBu', 'YlGnBu_r', 'YlGn_r', 'YlOrBr', 'YlOrBr_r', 'YlOrRd', 'YlOrRd_r', 'afmhot', 'afmhot_r', 'autumn', 'autumn_r', 'binary', 'binary_r', 'bone', 'bone_r', 'brg', 'brg_r', 'bwr', 'bwr_r', 'cividis', 'cividis_r', 'cool', 'cool_r', 'coolwarm', 'coolwarm_r', 'copper', 'copper_r', 'cubehelix', 'cubehelix_r', 'flag', 'flag_r', 'gist_earth', 'gist_earth_r', 'gist_gray', 'gist_gray_r', 'gist_heat', 'gist_heat_r', 'gist_ncar', 'gist_ncar_r', 'gist_rainbow', 'gist_rainbow_r', 'gist_stern', 'gist_stern_r', 'gist_yarg', 'gist_yarg_r', 'gnuplot', 'gnuplot2', 'gnuplot2_r', 'gnuplot_r', 'gray', 'gray_r', 'hot', 'hot_r', 'hsv', 'hsv_r', 'inferno', 'inferno_r', 'jet', 'jet_r', 'magma', 'magma_r', 'nipy_spectral', 'nipy_spectral_r', 'ocean', 'ocean_r', 'pink', 'pink_r', 'plasma', 'plasma_r', 'prism', 'prism_r', 'rainbow', 'rainbow_r', 'seismic', 'seismic_r', 'spring', 'spring_r', 'summer', 'summer_r', 'tab10', 'tab10_r', 'tab20', 'tab20_r', 'tab20b', 'tab20b_r', 'tab20c', 'tab20c_r', 'terrain', 'terrain_r', 'turbo', 'turbo_r', 'twilight', 'twilight_r', 'twilight_shifted', 'twilight_shifted_r', 'viridis', 'viridis_r', 'winter', 'winter_r' /usr/lib64/python3.6/site-packages/matplotlib/cbook/__init__.py:2257: ValueError _______________________ TestXVG_array.test_plot_coarsend _______________________ self = xvg = XVG(filename=None) def test_plot_coarsend(self, xvg): > ax = xvg.plot_coarsened(maxpoints=100) tests/fileformats/test_xvg.py:82: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/fileformats/xvg.py:675: in plot_coarsened cmap = matplotlib.cm.get_cmap(color) /usr/lib64/python3.6/site-packages/matplotlib/cm.py:204: in get_cmap cbook._check_in_list(sorted(_cmap_registry), name=name) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _values = ['Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', ...] kwargs = {'name': ['black', 'red', 'blue', 'orange', 'magenta', 'cyan', ...]} values = ['Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', ...] k = 'name', v = ['black', 'red', 'blue', 'orange', 'magenta', 'cyan', ...] def _check_in_list(_values, **kwargs): """ For each *key, value* pair in *kwargs*, check that *value* is in *_values*; if not, raise an appropriate ValueError. Examples -------- >>> cbook._check_in_list(["foo", "bar"], arg=arg, other_arg=other_arg) """ values = _values for k, v in kwargs.items(): if v not in values: raise ValueError( "{!r} is not a valid value for {}; supported values are {}" > .format(v, k, ', '.join(map(repr, values)))) E ValueError: ['black', 'red', 'blue', 'orange', 'magenta', 'cyan', 'yellow', 'brown', 'green'] is not a valid value for name; supported values are 'Accent', 'Accent_r', 'Blues', 'Blues_r', 'BrBG', 'BrBG_r', 'BuGn', 'BuGn_r', 'BuPu', 'BuPu_r', 'CMRmap', 'CMRmap_r', 'Dark2', 'Dark2_r', 'GnBu', 'GnBu_r', 'Greens', 'Greens_r', 'Greys', 'Greys_r', 'OrRd', 'OrRd_r', 'Oranges', 'Oranges_r', 'PRGn', 'PRGn_r', 'Paired', 'Paired_r', 'Pastel1', 'Pastel1_r', 'Pastel2', 'Pastel2_r', 'PiYG', 'PiYG_r', 'PuBu', 'PuBuGn', 'PuBuGn_r', 'PuBu_r', 'PuOr', 'PuOr_r', 'PuRd', 'PuRd_r', 'Purples', 'Purples_r', 'RdBu', 'RdBu_r', 'RdGy', 'RdGy_r', 'RdPu', 'RdPu_r', 'RdYlBu', 'RdYlBu_r', 'RdYlGn', 'RdYlGn_r', 'Reds', 'Reds_r', 'Set1', 'Set1_r', 'Set2', 'Set2_r', 'Set3', 'Set3_r', 'Spectral', 'Spectral_r', 'Wistia', 'Wistia_r', 'YlGn', 'YlGnBu', 'YlGnBu_r', 'YlGn_r', 'YlOrBr', 'YlOrBr_r', 'YlOrRd', 'YlOrRd_r', 'afmhot', 'afmhot_r', 'autumn', 'autumn_r', 'binary', 'binary_r', 'bone', 'bone_r', 'brg', 'brg_r', 'bwr', 'bwr_r', 'cividis', 'cividis_r', 'cool', 'cool_r', 'coolwarm', 'coolwarm_r', 'copper', 'copper_r', 'cubehelix', 'cubehelix_r', 'flag', 'flag_r', 'gist_earth', 'gist_earth_r', 'gist_gray', 'gist_gray_r', 'gist_heat', 'gist_heat_r', 'gist_ncar', 'gist_ncar_r', 'gist_rainbow', 'gist_rainbow_r', 'gist_stern', 'gist_stern_r', 'gist_yarg', 'gist_yarg_r', 'gnuplot', 'gnuplot2', 'gnuplot2_r', 'gnuplot_r', 'gray', 'gray_r', 'hot', 'hot_r', 'hsv', 'hsv_r', 'inferno', 'inferno_r', 'jet', 'jet_r', 'magma', 'magma_r', 'nipy_spectral', 'nipy_spectral_r', 'ocean', 'ocean_r', 'pink', 'pink_r', 'plasma', 'plasma_r', 'prism', 'prism_r', 'rainbow', 'rainbow_r', 'seismic', 'seismic_r', 'spring', 'spring_r', 'summer', 'summer_r', 'tab10', 'tab10_r', 'tab20', 'tab20_r', 'tab20b', 'tab20b_r', 'tab20c', 'tab20c_r', 'terrain', 'terrain_r', 'turbo', 'turbo_r', 'twilight', 'twilight_r', 'twilight_shifted', 'twilight_shifted_r', 'viridis', 'viridis_r', 'winter', 'winter_r' /usr/lib64/python3.6/site-packages/matplotlib/cbook/__init__.py:2257: ValueError _________________________ TestAmber03star.test_grompp __________________________ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...ci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformat...stry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] bufsize = -1, executable = None, stdin = None, stdout = -1, stderr = -1 preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] executable = b'grompp', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = 11, c2pwrite = 12, errread = 13 errwrite = 14, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: self = tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_grompp0') def test_grompp(self, tmpdir): """Check if grompp can be run successfully at all""" f = self.mdp c = self.conf p = self.processed with tmpdir.as_cwd(): o = 'topol.tpr' po = 'mdout.mdp' > rc, output, junk = gromacs.grompp(f=f, p=p, c=c, o=o, po=po, stdout=False, stderr=False) tests/fileformats/top/top.py:150: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...ci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/grompp.mdp -p /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top -c /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/conf.gro -o topol.tpr -po mdout.mdp CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? __________________________ TestAmber03star.test_mdrun __________________________ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...ci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformat...stry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] bufsize = -1, executable = None, stdin = None, stdout = -1, stderr = -1 preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] executable = b'grompp', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = 11, c2pwrite = 12, errread = 13 errwrite = 14, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: self = tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_mdrun0') low_performance = False def test_mdrun(self, tmpdir, low_performance): """Check if grompp can be run successfully at all""" # set low_performance with # # pytest --low-performance gromacs/tests # f = self.mdp c = self.conf processed = self.processed nt = 2 if low_performance else 0 with tmpdir.as_cwd(): > tpr = grompp(f, c, processed, prefix="reference") tests/fileformats/top/top.py:166: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/fileformats/top/top.py:36: in grompp **kwargs) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...ci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...mistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/grompp.mdp -p /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/processed.top -c /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03star/conf.gro -o reference.tpr -po reference.mdp CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? ___________________________ TestAmber03w.test_grompp ___________________________ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...e/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformat...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] bufsize = -1, executable = None, stdin = None, stdout = -1, stderr = -1 preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] executable = b'grompp', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = 11, c2pwrite = 12, errread = 13 errwrite = 14, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: self = tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_grompp1') def test_grompp(self, tmpdir): """Check if grompp can be run successfully at all""" f = self.mdp c = self.conf p = self.processed with tmpdir.as_cwd(): o = 'topol.tpr' po = 'mdout.mdp' > rc, output, junk = gromacs.grompp(f=f, p=p, c=c, o=o, po=po, stdout=False, stderr=False) tests/fileformats/top/top.py:150: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...e/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/grompp.mdp -p /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top -c /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/conf.gro -o topol.tpr -po mdout.mdp CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? ___________________________ TestAmber03w.test_mdrun ____________________________ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...e/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformat...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] bufsize = -1, executable = None, stdin = None, stdout = -1, stderr = -1 preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] executable = b'grompp', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = 11, c2pwrite = 12, errread = 13 errwrite = 14, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: self = tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_mdrun1') low_performance = False def test_mdrun(self, tmpdir, low_performance): """Check if grompp can be run successfully at all""" # set low_performance with # # pytest --low-performance gromacs/tests # f = self.mdp c = self.conf processed = self.processed nt = 2 if low_performance else 0 with tmpdir.as_cwd(): > tpr = grompp(f, c, processed, prefix="reference") tests/fileformats/top/top.py:166: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/fileformats/top/top.py:36: in grompp **kwargs) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber0...e/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/grompp.mdp -p /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/processed.top -c /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/amber03w/conf.gro -o reference.tpr -po reference.mdp CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? __________________________ TestCharmm22st.test_grompp __________________________ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm...sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformat...istry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] bufsize = -1, executable = None, stdin = None, stdout = -1, stderr = -1 preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] executable = b'grompp', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = 11, c2pwrite = 12, errread = 13 errwrite = 14, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: self = tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_grompp2') def test_grompp(self, tmpdir): """Check if grompp can be run successfully at all""" f = self.mdp c = self.conf p = self.processed with tmpdir.as_cwd(): o = 'topol.tpr' po = 'mdout.mdp' > rc, output, junk = gromacs.grompp(f=f, p=p, c=c, o=o, po=po, stdout=False, stderr=False) tests/fileformats/top/top.py:150: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm...sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/grompp.mdp -p /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top -c /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/conf.gro -o topol.tpr -po mdout.mdp CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? __________________________ TestCharmm22st.test_mdrun ___________________________ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm...sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, > universal_newlines=True, input=input, shell=use_shell) gromacs/core.py:288: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = (['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformat...istry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...],) kwargs = {'close_fds': True, 'shell': False, 'stderr': -1, 'stdin': None, ...} input_string = '' def __init__(self, *args, **kwargs): """Initialize with the standard :class:`subprocess.Popen` arguments. :Keywords: *input* string that is piped into the command """ kwargs.setdefault('close_fds', True) # fixes 'Too many open fds' with 2.6 self.input = kwargs.pop('input', None) if six.PY2 and self.input is not None: # in Python 2, subprocess.Popen uses os.write(chunk) with default ASCII encoding self.input = self.input.encode('utf-8') self.command = args[0] try: input_string = 'printf "' + \ self.input.replace('\n','\\n') + '" | ' # display newlines except (TypeError, AttributeError): input_string = "" self.command_string = input_string + " ".join(self.command) > super(PopenWithInput,self).__init__(*args, **kwargs) gromacs/core.py:700: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] bufsize = -1, executable = None, stdin = None, stdout = -1, stderr = -1 preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None universal_newlines = True, startupinfo = None, creationflags = 0 restore_signals = True, start_new_session = False, pass_fds = () def __init__(self, args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=_PLATFORM_DEFAULT_CLOSE_FDS, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None): """Create new Popen instance.""" _cleanup() # Held while anything is calling waitpid before returncode has been # updated to prevent clobbering returncode if wait() or poll() are # called from multiple threads at once. After acquiring the lock, # code must re-check self.returncode to see if another thread just # finished a waitpid() call. self._waitpid_lock = threading.Lock() self._input = None self._communication_started = False if bufsize is None: bufsize = -1 # Restore default if not isinstance(bufsize, int): raise TypeError("bufsize must be an integer") if _mswindows: if preexec_fn is not None: raise ValueError("preexec_fn is not supported on Windows " "platforms") any_stdio_set = (stdin is not None or stdout is not None or stderr is not None) if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: if any_stdio_set: close_fds = False else: close_fds = True elif close_fds and any_stdio_set: raise ValueError( "close_fds is not supported on Windows platforms" " if you redirect stdin/stdout/stderr") else: # POSIX if close_fds is _PLATFORM_DEFAULT_CLOSE_FDS: close_fds = True if pass_fds and not close_fds: warnings.warn("pass_fds overriding close_fds.", RuntimeWarning) close_fds = True if startupinfo is not None: raise ValueError("startupinfo is only supported on Windows " "platforms") if creationflags != 0: raise ValueError("creationflags is only supported on Windows " "platforms") self.args = args self.stdin = None self.stdout = None self.stderr = None self.pid = None self.returncode = None self.universal_newlines = universal_newlines self.encoding = encoding self.errors = errors # Input and output objects. The general principle is like # this: # # Parent Child # ------ ----- # p2cwrite ---stdin---> p2cread # c2pread <--stdout--- c2pwrite # errread <--stderr--- errwrite # # On POSIX, the child objects are file descriptors. On # Windows, these are Windows file handles. The parent objects # are file descriptors on both platforms. The parent objects # are -1 when not using PIPEs. The child objects are -1 # when not redirecting. (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) # We wrap OS handles *before* launching the child, otherwise a # quickly terminating child could make our fds unwrappable # (see #8458). if _mswindows: if p2cwrite != -1: p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0) if c2pread != -1: c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0) if errread != -1: errread = msvcrt.open_osfhandle(errread.Detach(), 0) text_mode = encoding or errors or universal_newlines self._closed_child_pipe_fds = False try: if p2cwrite != -1: self.stdin = io.open(p2cwrite, 'wb', bufsize) if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors) if c2pread != -1: self.stdout = io.open(c2pread, 'rb', bufsize) if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) if errread != -1: self.stderr = io.open(errread, 'rb', bufsize) if text_mode: self.stderr = io.TextIOWrapper(self.stderr, encoding=encoding, errors=errors) self._execute_child(args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, > restore_signals, start_new_session) /usr/lib64/python3.6/subprocess.py:729: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = args = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] executable = b'grompp', preexec_fn = None, close_fds = True, pass_fds = () cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False p2cread = -1, p2cwrite = -1, c2pread = 11, c2pwrite = 12, errread = 13 errwrite = 14, restore_signals = True, start_new_session = False def _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session): """Execute program (POSIX version)""" if isinstance(args, (str, bytes)): args = [args] else: args = list(args) if shell: args = ["/bin/sh", "-c"] + args if executable: args[0] = executable if executable is None: executable = args[0] orig_executable = executable # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. errpipe_read, errpipe_write = os.pipe() # errpipe_write must not be in the standard io 0, 1, or 2 fd range. low_fds_to_close = [] while errpipe_write < 3: low_fds_to_close.append(errpipe_write) errpipe_write = os.dup(errpipe_write) for low_fd in low_fds_to_close: os.close(low_fd) try: try: # We must avoid complex work that could involve # malloc or free in the child process to avoid # potential deadlocks, thus we do all this here. # and pass it to fork_exec() if env is not None: env_list = [] for k, v in env.items(): k = os.fsencode(k) if b'=' in k: raise ValueError("illegal environment variable name") env_list.append(k + b'=' + os.fsencode(v)) else: env_list = None # Use execv instead of execve. executable = os.fsencode(executable) if os.path.dirname(executable): executable_list = (executable,) else: # This matches the behavior of os._execvpe(). executable_list = tuple( os.path.join(os.fsencode(dir), executable) for dir in os.get_exec_path(env)) fds_to_keep = set(pass_fds) fds_to_keep.add(errpipe_write) self.pid = _posixsubprocess.fork_exec( args, executable_list, close_fds, tuple(sorted(map(int, fds_to_keep))), cwd, env_list, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, errpipe_read, errpipe_write, restore_signals, start_new_session, preexec_fn) self._child_created = True finally: # be sure the FD is closed no matter what os.close(errpipe_write) # self._devnull is not always defined. devnull_fd = getattr(self, '_devnull', None) if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd: os.close(p2cread) if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd: os.close(c2pwrite) if errwrite != -1 and errread != -1 and errwrite != devnull_fd: os.close(errwrite) if devnull_fd is not None: os.close(devnull_fd) # Prevent a double close of these fds from __init__ on error. self._closed_child_pipe_fds = True # Wait for exec to fail or succeed; possibly raising an # exception (limited in size) errpipe_data = bytearray() while True: part = os.read(errpipe_read, 50000) errpipe_data += part if not part or len(errpipe_data) > 50000: break finally: # be sure the FD is closed no matter what os.close(errpipe_read) if errpipe_data: try: pid, sts = os.waitpid(self.pid, 0) if pid == self.pid: self._handle_exitstatus(sts) else: self.returncode = sys.maxsize except ChildProcessError: pass try: exception_name, hex_errno, err_msg = ( errpipe_data.split(b':', 2)) # The encoding here should match the encoding # written in by the subprocess implementations # like _posixsubprocess err_msg = err_msg.decode() except ValueError: exception_name = b'SubprocessError' hex_errno = b'0' err_msg = 'Bad exception data from child: {!r}'.format( bytes(errpipe_data)) child_exception_type = getattr( builtins, exception_name.decode('ascii'), SubprocessError) if issubclass(child_exception_type, OSError) and hex_errno: errno_num = int(hex_errno, 16) child_exec_never_called = (err_msg == "noexec") if child_exec_never_called: err_msg = "" # The error must be from chdir(cwd). err_filename = cwd else: err_filename = orig_executable if errno_num != 0: err_msg = os.strerror(errno_num) if errno_num == errno.ENOENT: err_msg += ': ' + repr(err_filename) > raise child_exception_type(errno_num, err_msg, err_filename) E FileNotFoundError: [Errno 2] No such file or directory: 'grompp': 'grompp' /usr/lib64/python3.6/subprocess.py:1364: FileNotFoundError During handling of the above exception, another exception occurred: self = tmpdir = local('/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/pytest-of-portage/pytest-0/test_mdrun2') low_performance = False def test_mdrun(self, tmpdir, low_performance): """Check if grompp can be run successfully at all""" # set low_performance with # # pytest --low-performance gromacs/tests # f = self.mdp c = self.conf processed = self.processed nt = 2 if low_performance else 0 with tmpdir.as_cwd(): > tpr = grompp(f, c, processed, prefix="reference") tests/fileformats/top/top.py:166: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/fileformats/top/top.py:36: in grompp **kwargs) gromacs/core.py:398: in __call__ return self.run(*args, **kwargs) gromacs/core.py:178: in run results, p = self._run_command(*_args, **_kwargs) gromacs/core.py:610: in _run_command result, p = super(GromacsCommand, self)._run_command(*args, **kwargs) gromacs/core.py:221: in _run_command p = self.Popen(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , args = () kwargs = {'c': '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm...sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', ...} stderr = -1, stdout = -1, stdin = None, input = None, use_shell = False cmd = ['grompp', '-f', '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats...emistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top', '-c', ...] errmsg = "Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced?" def Popen(self, *args, **kwargs): """Returns a special Popen instance (:class:`PopenWithInput`). The instance has its input pre-set so that calls to :meth:`~PopenWithInput.communicate` will not need to supply input. This is necessary if one wants to chain the output from one command to an input from another. :TODO: Write example. """ stderr = kwargs.pop('stderr', None) # default: print to stderr (if STDOUT then merge) if stderr is False: # False: capture it stderr = PIPE elif stderr is True: stderr = None # use stderr stdout = kwargs.pop('stdout', None) # either set to PIPE for capturing output if stdout is False: # ... or to False stdout = PIPE elif stdout is True: stdout = None # for consistency, make True write to screen stdin = kwargs.pop('stdin', None) input = kwargs.pop('input', None) use_shell = kwargs.pop('use_shell', False) if input: stdin = PIPE if isinstance(input, six.string_types) and not input.endswith('\n'): # make sure that input is a simple string with \n line endings input = six.text_type(input) + '\n' else: try: # make sure that input is a simple string with \n line endings input = '\n'.join(map(six.text_type, input)) + '\n' except TypeError: # so maybe we are a file or something ... and hope for the best pass cmd = self._commandline(*args, **kwargs) # lots of magic happening here # (cannot move out of method because filtering of stdin etc) try: p = PopenWithInput(cmd, stdin=stdin, stderr=stderr, stdout=stdout, universal_newlines=True, input=input, shell=use_shell) except OSError as err: logger.error(" ".join(cmd)) # log command line if err.errno == errno.ENOENT: errmsg = "Failed to find Gromacs command {0!r}, maybe its not on PATH or GMXRC must be sourced?".format(self.command_name) logger.fatal(errmsg) > raise OSError(errmsg) E OSError: Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? gromacs/core.py:294: OSError ------------------------------ Captured log call ------------------------------- ERROR gromacs.core:core.py:290 grompp -f /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/grompp.mdp -p /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/processed.top -c /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/data/fileformats/top/charmm22st/conf.gro -o reference.tpr -po reference.mdp CRITICAL gromacs.core:core.py:293 Failed to find Gromacs command 'grompp', maybe its not on PATH or GMXRC must be sourced? =============================== warnings summary =============================== gromacs/fileformats/mdp.py:59 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/fileformats/mdp.py:59: DeprecationWarning: invalid escape sequence \s COMMENT = re.compile("""\s*;\s*(?P.*)""") # eat initial ws gromacs/fileformats/mdp.py:65 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/fileformats/mdp.py:65: DeprecationWarning: invalid escape sequence \s """, re.VERBOSE) gromacs/fileformats/ndx.py:87 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/fileformats/ndx.py:87: DeprecationWarning: invalid escape sequence \s SECTION = re.compile("""\s*\[\s*(?P\S.*\S)\s*\]\s*""") gromacs/fileformats/xpm.py:130 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/fileformats/xpm.py:130: DeprecationWarning: invalid escape sequence \s """, re.VERBOSE) gromacs/tools.py:451 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/tools.py:451: DeprecationWarning: invalid escape sequence \s "\s*(VERSION)?\s*(?P.+)$") gromacs/core.py:425 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/core.py:425: DeprecationWarning: invalid escape sequence \s """ gromacs/__init__.py:272 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/__init__.py:272: GromacsImportWarning: Some Gromacs commands were NOT found; maybe source GMXRC first? The following are missing: ['release'] category=GromacsImportWarning) gromacs/cbook.py:613 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:613: DeprecationWarning: invalid escape sequence \d """ gromacs/cbook.py:614 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:614: DeprecationWarning: invalid escape sequence \d qtot_pattern = re.compile('System has non-zero total charge: *(?P[-+]?\d*\.\d+([eE][-+]\d+)?)') gromacs/cbook.py:835 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:835: DeprecationWarning: invalid escape sequence \s """ gromacs/cbook.py:857 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:857: DeprecationWarning: invalid escape sequence \s """.format(demangled(parameter)), re.VERBOSE) gromacs/cbook.py:1025 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:1025: DeprecationWarning: invalid escape sequence \s """ gromacs/cbook.py:1031 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:1031: DeprecationWarning: invalid escape sequence \s p_marker = re.compile("\s*{0!s}".format(marker)) gromacs/cbook.py:1032 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:1032: DeprecationWarning: invalid escape sequence \s p_molecule = re.compile("\s*[\w+_-]+\s+\d+\s*(;.*)?$") gromacs/cbook.py:1496 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/cbook.py:1496: DeprecationWarning: invalid escape sequence \d """, re.VERBOSE | re.IGNORECASE) gromacs/qsub.py:373 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/qsub.py:373: DeprecationWarning: invalid escape sequence \s ('^#.*(-J)', '((?<=-J\s))\s*\w+', jobname), gromacs/qsub.py:374 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/qsub.py:374: DeprecationWarning: invalid escape sequence \s ('^#.*(-A|account_no)', '((?<=-A\s)|(?<=account_no\s))\s*\w+', budget), gromacs/qsub.py:375 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/qsub.py:375: DeprecationWarning: invalid escape sequence \s ('^#.*(-t)', '(?<=-t\s)(\d+:\d+:\d+)', walltime), gromacs/qsub.py:387 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/qsub.py:387: DeprecationWarning: invalid escape sequence \s ('^#.*(-N|job_name)', '((?<=-N\s)|(?<=job_name\s))\s*\w+', jobname), gromacs/qsub.py:388 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/qsub.py:388: DeprecationWarning: invalid escape sequence \s ('^#.*(-A|account_no)', '((?<=-A\s)|(?<=account_no\s))\s*\w+', budget), gromacs/qsub.py:389 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/qsub.py:389: DeprecationWarning: invalid escape sequence \d ('^#.*(-l walltime|wall_clock_limit)', '(?<==)(\d+:\d+:\d+)', walltime), tests/fileformats/top/top.py:13 /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/fileformats/top/top.py:13: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead. from pandas.util.testing import assert_frame_equal tests/test_log.py::test_clear_handlers /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/test_log.py:25: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead logger.warn("Dodo") tests/test_log.py::test_NullHandler /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/tests/test_log.py:33: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead logger.warn("screaming in silence") tests/test_run.py::test_get_double_or_single_prec_mdrun /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/run.py:356: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead logger.warn(wmsg) tests/test_run.py::test_get_double_or_single_prec_mdrun /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/run.py:357: AutoCorrectionWarning: No 'mdrun_d' binary found so trying 'mdrun' instead. (Note that energy minimization runs better with mdrun_d.) warnings.warn(wmsg, category=AutoCorrectionWarning) tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[error] /var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0/gromacs/fileformats/xvg.py:1068: LowAccuracyWarning: Using undocumented decimate_error() is highly EXPERIMENTAL category=gromacs.exceptions.LowAccuracyWarning) tests/fileformats/test_xvg.py::TestXVG_array::test_decimate[error] /usr/lib64/python3.6/site-packages/numkit/timeseries.py:145: LowAccuracyWarning: tcorrel(): Only 100 datapoints for the chosen nstep=1; ACF will possibly not be accurate. warnings.warn(wmsg, category=LowAccuracyWarning) tests/fileformats/test_xvg.py: 200 warnings /usr/lib64/python3.6/site-packages/numkit/timeseries.py:146: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead logger.warn(wmsg) -- Docs: https://docs.pytest.org/en/stable/warnings.html =========================== short test summary info ============================ FAILED tests/test_cbook.py::test_grompp_qtot - OSError: Failed to find Gromac... FAILED tests/test_run.py::test_MDRunner - RuntimeError: This test requires a ... FAILED tests/test_run.py::Test_find_gromacs_command::test_find - FileNotFound... FAILED tests/test_setup.py::test_trj_compact_main - OSError: Failed to find G... FAILED tests/test_tools.py::test_tools_help[trjconv] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[gmxcheck] - OSError: Failed to fi... FAILED tests/test_tools.py::test_tools_help[trjorder] - OSError: Failed to fi... FAILED tests/test_tools.py::test_tools_help[g_sgangle] - OSError: Failed to f... FAILED tests/test_tools.py::test_tools_help[genconf] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[grompp] - OSError: Failed to find... FAILED tests/test_tools.py::test_tools_help[do_dssp] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[make_ndx] - OSError: Failed to fi... FAILED tests/test_tools.py::test_tools_help[tpbconv] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[genion] - OSError: Failed to find... FAILED tests/test_tools.py::test_tools_help[eneconv] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[editconf] - OSError: Failed to fi... FAILED tests/test_tools.py::test_tools_help[make_edi] - OSError: Failed to fi... FAILED tests/test_tools.py::test_tools_help[gmxdump] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[g_sas] - OSError: Failed to find ... FAILED tests/test_tools.py::test_tools_help[pdb2gmx] - OSError: Failed to fin... FAILED tests/test_tools.py::test_tools_help[genrestr] - OSError: Failed to fi... FAILED tests/test_tools.py::test_tools_help[g_dist] - OSError: Failed to find... FAILED tests/test_tools.py::test_tools_help[mdrun] - OSError: Failed to find ... FAILED tests/test_tools.py::test_tools_help[trjcat] - OSError: Failed to find... FAILED tests/test_tools.py::test_tools_help[xpm2ps] - OSError: Failed to find... FAILED tests/test_tools.py::test_tools_help[genbox] - OSError: Failed to find... FAILED tests/test_tools.py::test_failure_raises - OSError: Failed to find Gro... FAILED tests/test_tools.py::test_failure_warns - OSError: Failed to find Grom... FAILED tests/test_tools.py::test_failure_ignore - AssertionError: Should have... FAILED tests/test_tools.py::TestRelease::test_release - AttributeError: modul... FAILED tests/test_tools.py::TestRelease::test_release_startswith - AttributeE... FAILED tests/test_tools.py::TestRelease::test_str - AttributeError: module 'g... FAILED tests/fileformats/test_xpm.py::TestXPM::test_to_pd - AttributeError: '... FAILED tests/fileformats/test_xvg.py::TestXVG_array::test_plot - ValueError: ... FAILED tests/fileformats/test_xvg.py::TestXVG_array::test_plot_coarsend - Val... FAILED tests/fileformats/top/test_amber03star.py::TestAmber03star::test_grompp FAILED tests/fileformats/top/test_amber03star.py::TestAmber03star::test_mdrun FAILED tests/fileformats/top/test_amber03w.py::TestAmber03w::test_grompp - OS... FAILED tests/fileformats/top/test_amber03w.py::TestAmber03w::test_mdrun - OSE... FAILED tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_grompp - ... FAILED tests/fileformats/top/test_charmm22.py::TestCharmm22st::test_mdrun - O... ERROR tests/test_config.py::test_set_gmxrc_environment - RuntimeError: Cannot... ERROR tests/test_setup.py::test_topology - OSError: Failed to find Gromacs co... ERROR tests/test_setup.py::test_solvate - OSError: Failed to find Gromacs com... ERROR tests/test_setup.py::test_energy_minimize - OSError: Failed to find Gro... ERROR tests/test_setup.py::test_energy_minimize_custom_mdp - OSError: Failed ... ============ 41 failed, 206 passed, 228 warnings, 5 errors in 7.94s ============ * ERROR: sci-chemistry/GromacsWrapper-0.8.0::gentoo failed (test phase): * Tests fail with python3.6 * * Call stack: * ebuild.sh, line 125: Called src_test * environment, line 3138: Called distutils-r1_src_test * environment, line 1326: Called _distutils-r1_run_foreach_impl 'python_test' * environment, line 491: Called python_foreach_impl 'distutils-r1_run_phase' 'python_test' * environment, line 2748: Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_test' * environment, line 2170: Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_test' * environment, line 2168: Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_test' * environment, line 899: Called distutils-r1_run_phase 'python_test' * environment, line 1263: Called python_test * environment, line 3098: Called die * The specific snippet of code: * pytest -vv || die "Tests fail with ${EPYTHON}" * * If you need support, post the output of `emerge --info '=sci-chemistry/GromacsWrapper-0.8.0::gentoo'`, * the complete build log and the output of `emerge -pqv '=sci-chemistry/GromacsWrapper-0.8.0::gentoo'`. * The complete build log is located at '/var/log/emerge-log/build/sci-chemistry/GromacsWrapper-0.8.0:20200908-151705.log'. * For convenience, a symlink to the build log is located at '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/temp/environment'. * Working directory: '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0' * S: '/var/tmp/portage/sci-chemistry/GromacsWrapper-0.8.0/work/GromacsWrapper-0.8.0'