Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 635996 - net-misc/youtube-dl-2017.10.29 FEATURES=network-sandbox - tests fail
Summary: net-misc/youtube-dl-2017.10.29 FEATURES=network-sandbox - tests fail
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Lowest minor (vote)
Assignee: Jeroen Roovers (RETIRED)
URL:
Whiteboard:
Keywords: TESTFAILURE
Depends on:
Blocks:
 
Reported: 2017-10-31 10:12 UTC by Toralf Förster
Modified: 2019-01-28 18:30 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge-info.txt (emerge-info.txt,15.06 KB, text/plain)
2017-10-31 10:15 UTC, Toralf Förster
Details
emerge-history.txt (emerge-history.txt,43.01 KB, text/plain)
2017-10-31 10:15 UTC, Toralf Förster
Details
environment (environment,108.51 KB, text/plain)
2017-10-31 10:15 UTC, Toralf Förster
Details
etc.portage.tbz2 (etc.portage.tbz2,12.47 KB, application/x-bzip)
2017-10-31 10:15 UTC, Toralf Förster
Details
net-misc:youtube-dl-2017.10.29:20171031-000018.log (net-misc:youtube-dl-2017.10.29:20171031-000018.log,532.41 KB, text/plain)
2017-10-31 10:15 UTC, Toralf Förster
Details
temp.tbz2 (temp.tbz2,72.10 KB, application/x-bzip)
2017-10-31 10:15 UTC, Toralf Förster
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Toralf Förster gentoo-dev 2017-10-31 10:12:56 UTC
Ran 2239 tests in 74.674s

FAILED (errors=50)
make: *** [Makefile:34: test] Error 1
 * ERROR: net-misc/youtube-dl-2017.10.29::gentoo failed (test phase):
 *   emake failed

  -------------------------------------------------------------------

  This is an unstable amd64 chroot image at a tinderbox (==build bot)
  name: 13.0-desktop_abi32+64-test_20171029-135004

  -------------------------------------------------------------------

gcc-config -l:
 [1] x86_64-pc-linux-gnu-6.4.0 *

Available Python interpreters, in order of preference:
  [1]   python3.4
  [2]   python2.7 (fallback)




emerge -qpv net-misc/youtube-dl
[ebuild  N    ] net-misc/youtube-dl-2017.10.29  USE="offensive {-test}" PYTHON_TARGETS="python2_7 python3_4 -python3_5 -python3_6"
Comment 1 Toralf Förster gentoo-dev 2017-10-31 10:15:08 UTC
Created attachment 501200 [details]
emerge-info.txt
Comment 2 Toralf Förster gentoo-dev 2017-10-31 10:15:11 UTC
Created attachment 501202 [details]
emerge-history.txt
Comment 3 Toralf Förster gentoo-dev 2017-10-31 10:15:14 UTC
Created attachment 501204 [details]
environment
Comment 4 Toralf Förster gentoo-dev 2017-10-31 10:15:17 UTC
Created attachment 501206 [details]
etc.portage.tbz2
Comment 5 Toralf Förster gentoo-dev 2017-10-31 10:15:21 UTC
Created attachment 501208 [details]
net-misc:youtube-dl-2017.10.29:20171031-000018.log
Comment 6 Toralf Förster gentoo-dev 2017-10-31 10:15:24 UTC
Created attachment 501210 [details]
temp.tbz2
Comment 7 Jeroen Roovers (RETIRED) gentoo-dev 2017-10-31 11:54:07 UTC
How do I tell whether FEATURES=network-sandbox?
Comment 8 Toralf Förster gentoo-dev 2017-10-31 12:02:41 UTC
(In reply to Jeroen Roovers from comment #7)

The attached  etc.portage.tbz2  contains the tar'ed /etc/portage which contains the make.conf which has the FEATURES=network-sandbox line in it
Comment 9 gentoo_usr 2017-11-28 19:40:17 UTC
Seems like you are not the only one stumbling about this. I have too and apparently someone else as well since there have been lately some fixes to the test suite by upstream.

Some tests fail due to missing test data (e.g. test_parse_m3u8_formats (test.test_InfoExtractor.TestInfoExtractor)). This is known upstream issue [0] and fixed in [1] (included in youtube-dl-2017.11.26).

[0] https://github.com/rg3/youtube-dl/issues/14854
[1] https://github.com/rg3/youtube-dl/commit/e94d1adc360d0fbfadced82c72a7f20375a1ec87

Other test fail because -- well -- the tests had some typos/wrong markings, e.g. test_ORFTVthek_2 (test.test_download.TestDownload). This is known by upstream as [2] and is fixed in [3] (also included in youtube-dl-2017.11.26).

[2] https://github.com/rg3/youtube-dl/issues/14855
[3] https://github.com/rg3/youtube-dl/commit/7512aa986f517c301fb3272584d21c8d5cec720f

There have also been some other fixes [4,5,6] (*not* included in youtube-dl-2017.11.26) which might or might not affect us.

[4] https://github.com/rg3/youtube-dl/issues/14856
[5] https://github.com/rg3/youtube-dl/issues/14857
[6] https://github.com/rg3/youtube-dl/issues/14858

[6] is especially interesting as it mentions an 'offlinetest' make target. I've run it and indeed it does not fail with FEATURE network-sandbox enabled.

I therefore suggest to run either 'test' or 'offlinetest' depending on whether we have the FEATURE network-sandbox enabled. A snippet for src_test() could look like:

        # bug #635996
        if has network-sandbox ${FEATURES}; then
                emake offlinetest
        else
                emake test
        fi


BTW: Is there any policy about whether a merge of an ebuild allowed to contact servers in the internet? In this case the test suite heavily contacts sites to test its extractors. This in particular includes some NSFW sites (e.g. the extractors for those sites and the test_age_restriction.py test), which not everyone might appreciate.
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-11-15 19:49:21 UTC
(In reply to gentoo_usr from comment #9)
> BTW: Is there any policy about whether a merge of an ebuild allowed to
> contact servers in the internet? In this case the test suite heavily
> contacts sites to test its extractors. This in particular includes some NSFW
> sites (e.g. the extractors for those sites and the test_age_restriction.py
> test), which not everyone might appreciate.

Yes, there is a policy and it's explicitly forbidden [1].  In case of NSFW sites, this is certainly something that shouldn't be happening by default.

[1]:https://devmanual.gentoo.org/ebuild-writing/functions/src_test/index.html#tests-that-require-network-or-service-access
Comment 11 Larry the Git Cow gentoo-dev 2019-01-28 18:30:05 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=122ab201e342645a4d3aac722fdae303230812c9

commit 122ab201e342645a4d3aac722fdae303230812c9
Author:     Jeroen Roovers <jer@gentoo.org>
AuthorDate: 2019-01-28 18:26:07 +0000
Commit:     Jeroen Roovers <jer@gentoo.org>
CommitDate: 2019-01-28 18:30:00 +0000

    net-misc/youtube-dl: Remove USE=offensive, add RESTRICT=test
    
    Bug: https://bugs.gentoo.org/show_bug.cgi?id=599928
    Closes: https://bugs.gentoo.org/show_bug.cgi?id=605010
    Closes: https://bugs.gentoo.org/show_bug.cgi?id=635996
    Closes: https://bugs.gentoo.org/show_bug.cgi?id=643200
    Closes: https://bugs.gentoo.org/show_bug.cgi?id=676532
    Package-Manager: Portage-2.3.58, Repoman-2.3.12
    Signed-off-by: Jeroen Roovers <jer@gentoo.org>

 net-misc/youtube-dl/youtube-dl-2019.01.27.ebuild | 59 ++---------------------
 net-misc/youtube-dl/youtube-dl-99999999.ebuild   | 60 ++----------------------
 2 files changed, 7 insertions(+), 112 deletions(-)