Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 722796 - dev-python/pip-20.1: test failures (test_entrypoints_work[entrypoint0,1,2])
Summary: dev-python/pip-20.1: test failures (test_entrypoints_work[entrypoint0,1,2])
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords: TESTFAILURE
Depends on:
Blocks:
 
Reported: 2020-05-13 03:11 UTC by Sam James
Modified: 2021-02-24 20:20 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (test phase) (file_722796.txt,36.21 KB, text/plain)
2020-05-13 03:11 UTC, Sam James
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sam James archtester Gentoo Infrastructure gentoo-dev Security 2020-05-13 03:11:37 UTC
Created attachment 638498 [details]
build.log (test phase)

FAILED tests/functional/test_cli.py::test_entrypoints_work[entrypoint0] - OSError: [Errno 2] No such file or directory
FAILED tests/functional/test_cli.py::test_entrypoints_work[entrypoint1] - OSError: [Errno 2] No such file or directory
FAILED tests/functional/test_cli.py::test_entrypoints_work[entrypoint2] - OSError: [Errno 2] No such file or directory
================================================ 3 failed, 1634 passed, 17 skipped, 259 deselected, 5 xfailed, 14 warnings in 3610.25 seconds 

----
Portage 2.3.99 (python 3.7.7-final-0, default/linux/arm64/17.0, gcc-9.3.0, glibc-2.31-r2, 4.9.0-4-arm64 aarch64)
=================================================================
System uname: Linux-4.9.0-4-arm64-aarch64-with-gentoo-2.7
KiB Mem:   131544964 total, 113479956 free
KiB Swap:    3321056 total,   3134972 free
Timestamp of repository gentoo: Tue, 12 May 2020 15:00:01 +0000
Head commit of repository gentoo: 95550adee868aabe0bbb23439ba961c3f764cd6e
sh bash 5.0_p17
ld GNU ld (Gentoo 2.34 p1) 2.34.0
ccache version 3.7.9 [disabled]
app-shells/bash:          5.0_p17::gentoo
dev-lang/perl:            5.30.2::gentoo
dev-lang/python:          2.7.18::gentoo, 3.6.10-r2::gentoo, 3.7.7-r2::gentoo, 3.8.2-r2::gentoo
dev-util/ccache:          3.7.9::gentoo
dev-util/cmake:           3.17.2::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.18::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:       1.11.6-r3::gentoo, 1.16.2::gentoo
sys-devel/binutils:       2.34::gentoo
sys-devel/gcc:            9.3.0::gentoo, 10.1.0::gentoo
sys-devel/gcc-config:     2.2.1::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.6::gentoo (virtual/os-headers)
sys-libs/glibc:           2.31-r2::gentoo
Repositories:

gentoo
    location: /usr/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts:
    sync-rsync-verify-max-age: 24

ACCEPT_KEYWORDS="arm64 ~arm64"
ACCEPT_LICENSE="@FREE"
CBUILD="aarch64-unknown-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="aarch64-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/portage/package.keywords/zz-autounmask /etc/portage/package.use/archtest /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=2"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -march=native"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=native"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j80"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl arm64 berkdb bzip2 cli crypt dri fortran gdbm iconv ipv6 libtirpc ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl tcpd unicode xattr zlib" ADA_TARGET="gnat_2018" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp thumb thumb2 v4 v5 v6 v7 v8 vfp vfp-d32 vfpv3 vfpv4" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7" RUBY_TARGETS="ruby24 ruby25" USERLAND="GNU" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-05-28 07:36:45 UTC
I've run pip's test suite a dozen times in these last days, and can't reproduce.
Comment 2 Renata Nagtalon 2020-08-08 08:58:15 UTC
Maybe I can help. The reason these tests fail is that the shebang of the generated test file is too long (current limit 127).

The test in question (test_entrypoints_work) creates a stub project inside a temp directory and creates a stub executable which contains a shebang "#!/path/to/python. The stub project is encapsulated in a virtual env, thus the /path/to/python is something like /var/tmp/portage/..../temp/.../virtualenv/bin/python.

For the default value PORTAGE_TMPDIR, namely /var/tmp/portage, that generated shepang line is 129 characters long, 2 over the limit of 127 (see man execve(2)). This means that the last chars are simply droped and the system tries to execute something like "/var/.../bin/pyth" which obviously does not exists, thus the OS Error.

I guess Sam is using the default path "/var/tmp/portage" and Michał something like "/tmp".

The workaround for this is to set PORTAGE_TMPDIR to something short, like "/t".

If you want to reproduce this you can do
mkdir /very/long/path/
ln -s /usr/bin/python  /very/long/path/
echo "#!/very/long/path/python" > /tmp/test.py
/tmp/test.py


Some references:
https://github.com/pypa/pip/issues/1773
https://github.com/pypa/virtualenv/issues/596
https://lwn.net/Articles/779997/


Ideas/workarounds I can think of (in random order):

* If 'test' is enabled, abort merge (and print a workaround suggestions) if the base path is detected to be longer than 128-chars-needed-by characters. Note that the path contains the version number of pip, i.e. pip-20.10 takes a lot more.

* Same as above but instead of aborting print a warning + workaround suggestions.

* Modify the build process, such that the generated directory structure is not so deeply nested and/or uses shorter names.

* Use something like https://github.com/shlevy/long-shebang or the snippets suggested in the pip issue to circumvent the long shebang line.
(probably needs patching of the test sources)

* Do use /usr/bin/python instead of the virtualenv python interpreter in the tests. (probably needs modifying of the test sources)

The proper solution would be to enlarge the 128 character array of the kernel to handle longer paths, which occur more and more often as time and technology advances.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2021-02-24 20:20:42 UTC
Closing bugs reported for old versions, no longer in ::gentoo.  Please reopen if you can still reproduce.