Bug 285669 – net-misc/curl: multiple test failures

• Status: RESOLVED OBSOLETE
• Product: Gentoo Linux

[Description Patrick Lauer 2009-09-20 11:08:07 UTC]

test 2003...OK (531 out of 532, remaining: 00:00)
TESTDONE: 454 tests out of 455 reported OK: 99%
TESTFAIL: These test cases failed: 513
TESTDONE: 534 tests were considered during 222 seconds.
TESTINFO: 79 tests were skipped due to these restraints:
TESTINFO: "rlimit problem: fds needed 1050 > system limit 1024" 1 times (518)
TESTINFO: "curl lacks scp support" 10 times (601, 603, 605, 607, 617, 619, 621, 623, 629, 631)
TESTINFO: "curl lacks netrc_debug support" 6 times (130, 131, 132, 133, 134, 257)
TESTINFO: "no stunnel" 20 times (300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 400, 401, 403, 404, 406, 407, 408, 409, 560, 1097)
TESTINFO: "failed starting SSH server" 6 times (700, 701, 702, 703, 706, 707)
TESTINFO: "configured as DISABLED" 2 times (563, 564)
TESTINFO: "Resolving IPv6 'ip6-localhost' didn't work" 2 times (241, 1083)
TESTINFO: "curl lacks idn support" 3 times (165, 1034, 1035)
TESTINFO: "curl lacks sftp support" 29 times (600, 602, 604, 606, 608, 609, 610, 611, 612, 613, 614, 615, 616, 618, 620, 622, 624, 625, 626, 627, 628, 630, 632, 633, 634, 635, 636, 637, 2004)
make[1]: *** [quiet-test] Error 1
make[1]: Leaving directory `/var/tmp/portage/net-misc/curl-7.19.6/work/curl-7.19.6/tests'
make: *** [test] Error 2
 *
 * ERROR: net-misc/curl-7.19.6 failed.

Portage 2.2_rc41 (default/linux/amd64/2008.0, gcc-4.4.1, glibc-2.10.1-r0, 2.6.30-gentoo-r1 x86_64)
=================================================================
System uname: Linux-2.6.30-gentoo-r1-x86_64-AMD_Phenom-tm-_9950_Quad-Core_Processor-with-gentoo-2.0.1
Timestamp of tree: Unknown
app-shells/bash:     4.0_p33
dev-lang/python:     2.6.2-r1, 3.1.1
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.4.3-r3
sys-apps/sandbox:    2.1
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.10.2, 1.11
sys-devel/binutils:  2.19.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/4.3/env /usr/kde/4.3/share/config /usr/kde/4.3/shutdown /usr/kde/4.4/env /usr/kde/4.4/share/config /usr/kde/4.4/shutdown /usr/kde/live/env /usr/kde/live/share/config /usr/kde/live/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests buildpkg distlocks fixpackages parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://localhost/gentoo-portage"
USE="X acl amd64 berkdb bzip2 cli cracklib crypt cups dri fortran gcj gdbm gpm iconv ipv6 isdnlog mmx mudflap multilib ncurses nls nptl nptlonly objc openmp pam pcre perl pppd python readline reflection session spl sse sse2 ssl sysfs tcpd unicode xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="vga"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

[Comment 1 Daniel Black (RETIRED) 2009-11-05 04:36:23 UTC]

I got the same when I ran this specific test on curl-7.19.7 individually in the group it passed. Individually it didnt' start the http server so I'm assuming it was a test error. Let me know if this is still a problem.

[Comment 2 Dustin Polke 2009-12-02 20:42:28 UTC]

I get this on my amd64 box as well as on sparc:
TESTDONE: 442 tests out of 444 reported OK: 99%
TESTFAIL: These test cases failed: 20 507
TESTDONE: 534 tests were considered during 224 seconds.
TESTINFO: 90 tests were skipped due to these restraints:
TESTINFO: "no stunnel" 20 times (300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 400, 401, 403, 404, 406, 407, 408, 409, 560, 1097)
TESTINFO: "failed starting SSH server" 6 times (700, 701, 702, 703, 706, 707)
TESTINFO: "rlimit problem: fds needed 1050 > system limit 1024" 1 times (518)
TESTINFO: "curl lacks ipv6 support" 16 times (116, 212, 240, 241, 242, 252, 253, 254, 255, 263, 1046, 1048, 1050, 1056, 1083, 1085)
TESTINFO: "curl lacks scp support" 10 times (601, 603, 605, 607, 617, 619, 621, 623, 629, 631)
TESTINFO: "configured as DISABLED" 2 times (563, 564)
TESTINFO: "curl lacks netrc_debug support" 6 times (130, 131, 132, 133, 134, 257)
TESTINFO: "curl lacks sftp support" 29 times (600, 602, 604, 606, 608, 609, 610, 611, 612, 613, 614, 615, 616, 618, 620, 622, 624, 625, 626, 627, 628, 630, 632, 633, 634, 635, 636, 637, 2004)
make[1]: *** [quiet-test] Error 1
make[1]: Leaving directory `/var/tmp/portage/net-misc/curl-7.19.6/work/curl-7.19.6/tests'
make: *** [test] Error 2
 ^[[31;01m*^[[0m
 ^[[31;01m*^[[0m ERROR: net-misc/curl-7.19.6 failed.

sparc: emerge --info
Portage 2.1.6.13 (default/linux/sparc/experimental/multilib/server, gcc-4.3.4, glibc-2.9_p20081201-r2, 2.6.30-gentoo-r8 sparc64)
=================================================================
System uname: Linux-2.6.30-gentoo-r8-sparc64-sun4u-with-gentoo-1.12.13
Timestamp of tree: Wed, 02 Dec 2009 18:45:02 +0000
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p28
dev-lang/python:     2.6.2-r1
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.7.9-r1, 1.10.2
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="sparc"
CBUILD="sparc-unknown-linux-gnu"
CFLAGS="-O2 -mcpu=ultrasparc -pipe -ggdb"
CHOST="sparc-unknown-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O2 -mcpu=ultrasparc -pipe -ggdb"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache collision-protect distlocks fixpackages multilib-strict parallel-fetch protect-owned sandbox sfperms splitdebug strict test unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="ftp://mirror.netcologne.de/gentoo/     ftp://gentoo.tiscali.nl/pub/mirror/gentoo/     ftp://mirror.cambrium.nl/pub/os/linux/gentoo/     ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo     ftp://ftp6.uni-muenster.de/pub/linux/distributions/gentoo"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en de"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_COMPRESS=""
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--timeout=500"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/sping /usr/local/portage/modified /usr/local/portage/own"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="acl admin apache2 bash-completion berkdb bzip2 cgi cli courier coverpage cracklib crypt cups dovecot-sasl dri fortran gd gdbm git gpm iconv iproute2 javascript jpeg kerberos keyscrub logrotate loop-aes mime modules mudflap multilib ncurses nls nptl nptlonly openmp pam pcre perl php png posix pppd python readline reflection sasl session snmp sparc spl ssl suhosin sysfs syslog tcpd threads tiff truetype unicode userlocales xml xorg zlib" ALSA_CARDS="cs4231" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias asis auth_basic auth_digest authn_anon authn_dbm     authn_file authz_host autoindex cache cern_meta charset_lite dav dav_fs     deflate dir disk_cache env expires ext_filter file_cache filter headers     include info log_config logio mem_cache mime mime_magic negotiation proxy     proxy_connect proxy_ftp proxy_http rewrite setenvif speling status unique_id     userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en de" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="mach64"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS_FLAGS

amd64: emerge --info
Portage 2.1.6.13 (default/linux/amd64/10.0, gcc-4.3.4, glibc-2.9_p20081201-r2, 2.6.31-gentoo-r6-stable x86_64)
=================================================================
System uname: Linux-2.6.31-gentoo-r6-stable-x86_64-Intel-R-_Core-TM-2_Duo_CPU_T8100_@_2.10GHz-with-gentoo-1.12.13
Timestamp of tree: Wed, 02 Dec 2009 18:15:01 +0000
app-shells/bash:     4.0_p28
dev-java/java-config: 2.1.9-r1
dev-lang/python:     2.6.2-r1
dev-python/pycrypto: 2.0.1-r8
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe -msse4.1"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/bin/mygenkernel /usr/sbin/run-crons /usr/share/X11/xkb /usr/share/cairo-dock/plug-ins/wifi/wifi /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=core2 -O2 -pipe -msse4.1"
DISTDIR="/usr/portage/distfiles"
FEATURES="collision-protect distlocks fixpackages multilib-strict parallel-fetch protect-owned sandbox sfperms strict test unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo/     ftp://de-mirror.org/distro/gentoo/     ftp://mirror.netcologne.de/gentoo/     ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo     ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo"
LANG="C"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,relro -Wl,-z,now"
LINGUAS="en de"
MAKEOPTS="-j5"
PKGDIR="/home/ftp/binpkg/"
PORTAGE_COMPRESS=""
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--timeout=500"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/alon-barlev /usr/local/portage/layman/sping /usr/local/portage/layman/science /usr/local/portage/layman/desktop-effects /usr/local/portage/layman/sunrise /usr/local/portage/g-ctan /usr/local/portage/modified /usr/local/portage/own"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="X X509 Xaw3d a52 aac aalib accessibility acl acpi alsa amd64 amr amrnb amrr amrwb apm async audacious audiofile bash-completion battery berkdb bindist bl bluetooth branding bzip2 cairo cardbus cdaudio cdda cdinstall cjk cli consolekit cracklib crypt css ctype cups curl dbus dga directfb dri dts dv dvd dvdr eds encode evo exchange exif fam fame fbcon fbcondecor fbsplash ffmpeg flac fontforge foomaticdb force-cgi-redirect fortran ftp gd gdbm gif gimp git glade glitz gmp gphoto2 gs gsm gstreamer gtk gtkhtml guile hal hddtemp hdf5 iconv icq icu idn ieee1394 imagemagick imap imlib iproute2 jack java javascript jpeg jpeg2k kerberos keyscrub kino kpathsea lame lapack laptop latex lcms libcaca libnotify libsamplerate libsexy libv4l2 libwww live lm_sensors logrotate loop-aes lzma lzo mad memlimit mikmod mime mjpeg mmx mmxext mng modules mono motif mp2 mp3 mp4 mpeg mplayer mudflap multilib musepack ncurses nls nntp nowin nptl nptlonly nsplugin ntp nuv nvidia ogg opengl openmp opensslcrypt pam pch pcmcia pcre pda pdf perl plotutils png posix ppds pppd print python qt3support quicktime readline reflection resolvconf rtsp ruby rubytests samba sasl schroedinger sdl server session simplexml slang sndfile soap sockets sou speex spell spl sse sse2 sse3 sse4.1 ssl ssse3 startup-notification stream subversion svg swat sysfs syslog tcpd tga theora threads tiff truetype unicode usb userlocales v4l v4l2 vcd vdpau vim-syntax vim-with-x vorbis wifi winbind wma wmf wxwindows x264 xanim xcomposite xfce xinetd xml xmp xorg xpm xsl xulrunner xv xvid xvmc zlib" ALSA_CARDS="hda-intel mpu401" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAMERAS="fuji ptp2" ELIBC="glibc" INPUT_DEVICES="keyboard evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en de" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS_FLAGS

[Comment 3 Dustin Polke 2009-12-02 21:20:57 UTC]

test 20 and 507 fail because my ISP hijacks Page Not Found Errors

Is it possible to make these non-fatal?

[Comment 4 Daniel Black (RETIRED) 2009-12-02 21:39:00 UTC]

(In reply to comment #3)
> test 20 and 507 fail because my ISP hijacks Page Not Found Errors
> 
> Is it possible to make these non-fatal?
yes. Also possible is getting a non-crappy ISP.

My suggestion however it to put an upstream feature request that DNS replies like this are faked using LD_PRELOAD/LD_LIBRARY_PATH wrapping the network functions so this doesn't occur.

thanks for the further diagnosis.

[Comment 5 n.reyrolle 2010-01-15 13:50:27 UTC]

For information, I also ran into this one with x86. And i tried to upgrade to 7.19.7, but ran into bug #234281.

[Comment 6 Thomas Kahle (RETIRED) 2010-05-07 09:08:06 UTC]

Still present in the current (x86) stable. Updating summary.

[Comment 7 Dan Fandrich 2010-08-31 19:12:12 UTC]

You should be able to skip tests affected by such rogue ISPs by running the test suite with 'make test TEST_Q="-a -s !non-existing\ host"'

[Comment 8 Pacho Ramos 2011-07-20 15:33:45 UTC]

*** Bug 335370 has been marked as a duplicate of this bug. ***

[Comment 9 Pacho Ramos 2011-07-20 15:33:49 UTC]

*** Bug 370415 has been marked as a duplicate of this bug. ***

[Comment 10 Pacho Ramos 2011-07-20 15:33:53 UTC]

*** Bug 373285 has been marked as a duplicate of this bug. ***

[Comment 11 Xake 2011-07-21 20:09:35 UTC]

(In reply to comment #10)
> *** Bug 373285 has been marked as a duplicate of this bug. ***

Is this a bug report catch-all for all test-failures?

I ask because this bug seems to have two different reports with two completely DIFFERENT sets of failures.
My bug report had four failed testcases, but only one testcase in common with one of the two different reports in this bug and no build.log in this report to even come close to know if we have the same symptoms (mine seems to be something triggering valgrind).
If my problem was due to rogue DNS-provider (which this reports seems to be the current topic eve if the OP has not confirmed of denied weather his problem is the same) should I not fail testcase 20 as well?

Maybe close this one, and instead make a new clean tracker bug for curl test failures since this bug currently feels like a mess?

[Comment 12 SpanKY 2012-01-26 19:26:17 UTC]

no, it shouldn't be a catch all bug report for test failure.  if there are known test failures which are unrelated, i'd re-open and improve the summary.

[Comment 13 Anthony Basile 2012-05-26 13:38:22 UTC]

Starting with 7.25.0 and now 7.26.0 which I'll add to the tree in a bit, most tests are failing.  This is the case even when one simply works with the upstream source directly, without any patches and outside of the portage environment.  There seem to be at least two issues going on here, related to 1) network connectivity, and 2) valgrind errors.

Despite this, curl seems to be working fine.  So until we get this sorted out, I'm adding curl-7.26.0.ebuild to the tree with RESTRICT="test".

[Comment 14 Agostino Sarubbo 2012-08-25 15:15:51 UTC]

@blueness, 7.26.0 passes tests for me. 

Can someone confirm?

[Comment 15 Anthony Basile 2012-08-25 18:18:31 UTC]

(In reply to comment #14)
> @blueness, 7.26.0 passes tests for me. 
> 
> Can someone confirm?

ago, I still do not understand what's going on with these tests.  Here's what I get:

********* System characteristics ******** 
* curl 7.26.0 (x86_64-pc-linux-gnu) 
* libcurl/7.26.0 OpenSSL/1.0.0j zlib/1.2.5.1
* Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz 
* Host: yellow
* System: Linux yellow 3.2.22 #1 SMP PREEMPT Tue Jul 10 07:07:56 EDT 2012 x86_64 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz GenuineIntel GNU/Linux
* Server SSL:        OFF  libcurl SSL:  ON 
* debug build:       OFF  track memory: OFF
* valgrind:          ON   HTTP IPv6     ON 
* FTP IPv6           ON   Libtool lib:  OFF
* Shared build:      yes
* SSL library:       OpenSSL
* Ports:
*   HTTP/8990 FTP/8992 FTP2/8995 RTSP/9007 
*   TFTP/8997 HTTP-IPv6/8994 RTSP-IPv6/9008 FTP-IPv6/8996 
*   GOPHER/9009 GOPHER-IPv6/9009
*   SSH/8999 SOCKS/9000 POP3/9001 IMAP/9003 SMTP/9005
*   POP3-IPv6/9002 IMAP-IPv6/9004 SMTP-IPv6/9006
*   HTTPTLS/9011 HTTPTLS-IPv6/9012 
***************************************** 
test 001...FAILED
test 002...FAILED
test 003...FAILED
test 004...FAILED
test 005...FAILED
test 006...FAILED
test 007...FAILED
...


Practically every test fails for me.  Can you give me more details and I'll see if I can figure out why tests are failing for me, but succeeding for you.

[Comment 16 Markus Walter 2012-09-01 12:27:36 UTC]

For me all tests passed too. I used hardened gcc-4.7.1 on ~amd64.

[Comment 17 Anthony Basile 2012-09-01 12:40:32 UTC]

(In reply to comment #16)
> For me all tests passed too. I used hardened gcc-4.7.1 on ~amd64.

Sounds like my environment then.  Let me build a fresh system and test.

[Comment 18 Dan Fandrich 2012-09-01 19:15:30 UTC]

If you run the tests with 'make TEST_Q="-a -p -n -v -d"' you'll get lots of verbose information about what the test failure are.

[Comment 19 Roman Žilka 2013-03-17 20:34:15 UTC]

FYI: when I remove RESTRICT="test" (curl-7.29.0-r1), the test phase runs and completes without an error on my x86 system now. I only get this:

(...)
test 562...OK (388 out of 784, remaining: 02:34)
test 563...OK (389 out of 784, remaining: 02:33)
Will not run ssh server as root to mitigate security risks
RUN: failed to start the SSH server
test 565...OK (391 out of 784, remaining: 02:49)
test 566...OK (392 out of 784, remaining: 02:48)
(...)

But I suppose that's all right. Try emerging on your system.

[Comment 20 Anthony Basile 2013-03-17 21:01:33 UTC]

(In reply to comment #19)
> FYI: when I remove RESTRICT="test" (curl-7.29.0-r1), the test phase runs and
> completes without an error on my x86 system now. I only get this:
> 
> (...)
> test 562...OK (388 out of 784, remaining: 02:34)
> test 563...OK (389 out of 784, remaining: 02:33)
> Will not run ssh server as root to mitigate security risks
> RUN: failed to start the SSH server
> test 565...OK (391 out of 784, remaining: 02:49)
> test 566...OK (392 out of 784, remaining: 02:48)
> (...)
> 
> But I suppose that's all right. Try emerging on your system.

There are varioius reasons why the test for curl fail.  Upstream has cleaned them up and people are saying they work, but I'll keep them restricted just in case.  That doesn't stop someone bold, like yourself, from manually testing.

[Comment 21 Markus Walter 2013-12-22 21:25:44 UTC]

(In reply to Anthony Basile from comment #20)
> There are varioius reasons why the test for curl fail.  Upstream has cleaned
> them up and people are saying they work, but I'll keep them restricted just
> in case.  That doesn't stop someone bold, like yourself, from manually
> testing.

I just tried curl-7.34.0 on ~amd64 and there are still some test failures, so restricting is probably appropriate. I see the following:

TESTFAIL: These test cases failed: 9 39 41 44 64 65 70 71 72 88 153 154 158 163 166 167 168 170 173 186 206 245 246 258 259 273 277 540 551 552 554 565 579 587 1001 1002 1030 1053 1060 1061 1071 1072 1079 1095 1133 1229 1315 1404 1412

[Comment 22 Anthony Basile 2013-12-22 22:28:47 UTC]

(In reply to Markus Oehme from comment #21)
> (In reply to Anthony Basile from comment #20)
> > There are varioius reasons why the test for curl fail.  Upstream has cleaned
> > them up and people are saying they work, but I'll keep them restricted just
> > in case.  That doesn't stop someone bold, like yourself, from manually
> > testing.
> 
> I just tried curl-7.34.0 on ~amd64 and there are still some test failures,
> so restricting is probably appropriate. I see the following:
> 
> TESTFAIL: These test cases failed: 9 39 41 44 64 65 70 71 72 88 153 154 158
> 163 166 167 168 170 173 186 206 245 246 258 259 273 277 540 551 552 554 565
> 579 587 1001 1002 1030 1053 1060 1061 1071 1072 1079 1095 1133 1229 1315
> 1404 1412

Thanks for testing this.

[Comment 23 Julian Ospald 2014-03-17 20:09:22 UTC]

It seems python is used for some of the tests, so the ebuild should use python-any-r1, otherwise they will fail with syntax error if python3 is used.

also, 172 fails because of
http://curl.haxx.se/mail/tracker-2014-02/0000.html
fix is here: https://github.com/bagder/curl/commit/ffb8a21d85bde8

After those 2 fixes, I get 100% on amd64 for curl-7.35.0 with USE flags: adns ipv6 ssl curl_ssl_polarssl

[Comment 24 Julian Ospald 2014-03-30 23:07:31 UTC]

Created attachment 373940 [details, diff]
curl-7.35.0.ebuild.diff

[Comment 25 Julian Ospald 2014-03-30 23:08:01 UTC]

Created attachment 373942 [details, diff]
curl-7.35.0-tests.patch

[Comment 26 Anthony Basile 2014-03-31 10:49:00 UTC]

(In reply to Julian Ospald (hasufell) from comment #25)
> Created attachment 373942 [details, diff] [details, diff]
> curl-7.35.0-tests.patch

Thanks!  I'd like to apply these to 7.36.0 since it will be the next stabilized.

[Comment 27 Anthony Basile 2014-03-31 13:44:57 UTC]

(In reply to Anthony Basile from comment #26)
> (In reply to Julian Ospald (hasufell) from comment #25)
> > Created attachment 373942 [details, diff] [details, diff] [details, diff]
> > curl-7.35.0-tests.patch
> 
> Thanks!  I'd like to apply these to 7.36.0 since it will be the next
> stabilized.

I added this to 7.36.0.  I didn't need the patch becuase its already in 7.36.0.  Do you mind testing and letting me know if this is okay at your end.

[Comment 28 Julian Ospald 2014-03-31 21:10:24 UTC]

(In reply to Anthony Basile from comment #27)
> (In reply to Anthony Basile from comment #26)
> > (In reply to Julian Ospald (hasufell) from comment #25)
> > > Created attachment 373942 [details, diff] [details, diff] [details, diff] [details, diff]
> > > curl-7.35.0-tests.patch
> > 
> > Thanks!  I'd like to apply these to 7.36.0 since it will be the next
> > stabilized.
> 
> I added this to 7.36.0.  I didn't need the patch becuase its already in
> 7.36.0.  Do you mind testing and letting me know if this is okay at your end.

No, 815 fails... I think backporting is the right approach here. The same goes for the polarssl bug.

[Comment 29 Anthony Basile 2014-03-31 21:17:27 UTC]

(In reply to Julian Ospald (hasufell) from comment #28)
> (In reply to Anthony Basile from comment #27)
> > (In reply to Anthony Basile from comment #26)
> > > (In reply to Julian Ospald (hasufell) from comment #25)
> > > > Created attachment 373942 [details, diff] [details, diff] [details, diff] [details, diff] [details, diff]
> > > > curl-7.35.0-tests.patch
> > > 
> > > Thanks!  I'd like to apply these to 7.36.0 since it will be the next
> > > stabilized.
> > 
> > I added this to 7.36.0.  I didn't need the patch becuase its already in
> > 7.36.0.  Do you mind testing and letting me know if this is okay at your end.
> 
> No, 815 fails... I think backporting is the right approach here. The same
> goes for the polarssl bug.

Okay.  I was hoping 7.36.0 would take care of this and the polarssl problem, but if I'm backporting the above, I guess I'm backport the polarssl commits too.  (For reference, that's bug #503558.)

[Comment 30 Julian Ospald 2014-03-31 21:28:11 UTC]

http://curl.haxx.se/mail/archive-2014-03/0061.html

[Comment 31 Anthony Basile 2014-04-01 23:34:47 UTC]

(In reply to Anthony Basile from comment #29)
> (In reply to Julian Ospald (hasufell) from comment #28)
> > (In reply to Anthony Basile from comment #27)
> > > (In reply to Anthony Basile from comment #26)
> > > > (In reply to Julian Ospald (hasufell) from comment #25)
> > > > > Created attachment 373942 [details, diff] [details, diff] [details, diff] [details, diff] [details, diff] [details, diff]
> > > > > curl-7.35.0-tests.patch
> > > > 
> > > > Thanks!  I'd like to apply these to 7.36.0 since it will be the next
> > > > stabilized.
> > > 
> > > I added this to 7.36.0.  I didn't need the patch becuase its already in
> > > 7.36.0.  Do you mind testing and letting me know if this is okay at your end.
> > 
> > No, 815 fails... I think backporting is the right approach here. The same
> > goes for the polarssl bug.
> 
> Okay.  I was hoping 7.36.0 would take care of this and the polarssl problem,
> but if I'm backporting the above, I guess I'm backport the polarssl commits
> too.  (For reference, that's bug #503558.)

I just tested these patches on 7.35.0 and I'm getting all the same faiures as on 7.36.0.  Did they work for you?

[Comment 32 Julian Ospald 2014-04-02 15:09:37 UTC]

(In reply to Anthony Basile from comment #31)
> (In reply to Anthony Basile from comment #29356)
> > (In reply to Julian Ospald (hasufell) from comment #28357)
> > > (In reply to Anthony Basile from comment #27358)
> > > > (In reply to Anthony Basile from comment #26359)
> > > > > (In reply to Julian Ospald (hasufell) from comment #25360)
> > > > > > Created attachment 373942361 [details362, diff363] [details, diff] [details, diff] [details, diff] [details, diff] [details, diff]
> > > > > > curl-7.35.0-tests.patch
> > > > > 
> > > > > Thanks!  I'd like to apply these to 7.36.0 since it will be the next
> > > > > stabilized.
> > > > 
> > > > I added this to 7.36.0.  I didn't need the patch becuase its already in
> > > > 7.36.0.  Do you mind testing and letting me know if this is okay at your end.
> > > 
> > > No, 815 fails... I think backporting is the right approach here. The same
> > > goes for the polarssl bug.
> > 
> > Okay.  I was hoping 7.36.0 would take care of this and the polarssl problem,
> > but if I'm backporting the above, I guess I'm backport the polarssl commits
> > too.  (For reference, that's bug #503558364.)
> 
> I just tested these patches on 7.35.0 and I'm getting all the same faiures
> as on 7.36.0.  Did they work for you?

Did you only use the tarball? curl-7.35.0-tests.patch is not in the tarball

[Comment 33 Anthony Basile 2014-04-02 15:42:12 UTC]

> > 
> > I just tested these patches on 7.35.0 and I'm getting all the same faiures
> > as on 7.36.0.  Did they work for you?
> 
> Did you only use the tarball? curl-7.35.0-tests.patch is not in the tarball

No.  I think I did it right.  We've seen this happen before where ago said tests were working but others (me included) could not reproduce.

Okay, I committed this to the tree:

1) 7.35.0 is the current tree stable and is as it was, without tests enabled.

2) I added 7.35.0-r1 with your changes and dropped to ~arch.

3)  I reverted 7.36.0 to what it was before without the tests were enabled.  Its currently ~arch but we need to stabilzie 7.36.0 soon because of bug #505864.

Since this is only tests (the polarssl bug is fixed in 7.36.0), I'm going to start stabilization of 7.36.0 as is.  Let's work with 7.35.0-r1 to figure out what's going on with these tests --- feel free to make commits to it.  As I said, ago did report once that all were working for him, but I could not reproduce and neither could some other people (a while back so I forget who).  I wonder if it has anything to do with our network env or hardening?

[Comment 34 Julian Ospald 2014-04-02 16:19:03 UTC]

(In reply to Anthony Basile from comment #33)

It seems that this is indeed dependant on the ssl provider. A quick test with openssl shows a LOT of failed tests for me.

TESTFAIL: These test cases failed: 9 39 41 44 64 65 70 71 72 88 153 154 158 163 166 167 168 170 173 186 206 245 246 258 259 273 277 540 551 552 554 565 579 587 1001 1002 1030 1053 1060 1061 1071 1072 1079 1095 1133 1229 1315 1404 1412 

For polarssl, I still get 100%. Which actually is a pretty good argument for making curl_ssl_polarssl the default, and not curl_ssl_openssl.

[Comment 35 Anthony Basile 2014-04-02 16:38:51 UTC]

(In reply to Julian Ospald (hasufell) from comment #34)
> (In reply to Anthony Basile from comment #33)
> 
> It seems that this is indeed dependant on the ssl provider. A quick test
> with openssl shows a LOT of failed tests for me.
> 
> TESTFAIL: These test cases failed: 9 39 41 44 64 65 70 71 72 88 153 154 158
> 163 166 167 168 170 173 186 206 245 246 258 259 273 277 540 551 552 554 565
> 579 587 1001 1002 1030 1053 1060 1061 1071 1072 1079 1095 1133 1229 1315
> 1404 1412 
> 
> For polarssl, I still get 100%. Which actually is a pretty good argument for
> making curl_ssl_polarssl the default, and not curl_ssl_openssl.

*sigh* no idea what's going on here.  I get lots of failures with either openssl or polarssl.  I used your flags from comment #23.

[Comment 36 Julian Ospald 2014-04-02 16:49:48 UTC]

(In reply to Anthony Basile from comment #35)

what a mess... and no one exactly knows if the tests are screwed up or if those failures tell us something important

[Comment 37 Anthony Basile 2021-02-09 16:26:48 UTC]

(In reply to Julian Ospald from comment #36)
> (In reply to Anthony Basile from comment #35)
> 
> what a mess... and no one exactly knows if the tests are screwed up or if
> those failures tell us something important

This keeps coming up in different contexts.  I'm going to close this bug because it provides no new information over other more recent bugs.