Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 561032 - [science overlay] please keyword jupyter ebuilds ~x86
Summary: [science overlay] please keyword jupyter ebuilds ~x86
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Marius Brehler
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-21 14:45 UTC by Andrey Grozin
Modified: 2015-09-22 07:19 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Grozin gentoo-dev 2015-09-21 14:45:37 UTC
I've successfully installed jupyter and ipython-4 on my ~amd64 box, many thanks to those who brought them to gentoo. I tried to do the same on a ~x86 box: added the needed lines to /etc/portage/package.keyword and tried to emerge (I added -doc to notebook and nbconvert, because I don't want haskell and hence pandoc). Several packages emerged successfully, but for ipywidgets I got

>>> Source compiled.
 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-9683.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: mkdir
S: deny
P: /usr/etc
A: /usr/etc
R: /usr/etc
C: npm                                
<this is repeated 11 times>

From where this /usr/etc could appear?
Comment 1 Marius Brehler 2015-09-21 14:57:55 UTC
I currently have neither a x86 box nor a schroot setup, thus I can't test and therefore only ~amd64 keywords were added to the ebuilds. Maybe, I can setup a chroot during the next week. It seems that the ebuild calls 'npm' which tries to install some js packages. Can you provide a more detailed extract of the build log? If so, I see a lot of trouble coming up, since Gentoo currently doesn't provides an npm.eclass or similar.

Is the same happening with ipywidgets-4.0.2-r1?

Further, I would appreciate any pull request adding ~x86 keywords to the jupyter packages in the science overlay, which were successfully tested with 'ebuild test'.
Comment 2 Marius Brehler 2015-09-21 15:03:23 UTC
Okay, actually we have the same trouble on ~amd64:


>>> Compiling source in /var/tmp/portage/dev-python/ipywidgets-9999/work/ipywidgets-9999 ...
 * python3_3: running distutils-r1_run_phase distutils-r1_python_compile
/usr/bin/python3.3 setup.py build
running build
running build_py
running jsdeps
 * ACCESS DENIED:  mkdir:        /usr/etc
npm WARN package.json ipython-widget-deps@4.0.0 license should be a valid SPDX license expression

> spawn-sync@1.0.13 postinstall /var/tmp/portage/dev-python/ipywidgets-9999/work/ipywidgets-9999/node_modules/spawn-sync
> node postinstall


> ipython-widget-deps@4.0.0 postinstall /var/tmp/portage/dev-python/ipywidgets-9999/work/ipywidgets-9999
> npm run bower

 * ACCESS DENIED:  mkdir:        /usr/etc

> ipython-widget-deps@4.0.0 bower /var/tmp/portage/dev-python/ipywidgets-9999/work/ipywidgets-9999
> bower install

bower bootstrap#~3.3        not-cached git://github.com/components/bootstrap.git#~3.3
bower bootstrap#~3.3           resolve git://github.com/components/bootstrap.git#~3.3



npm downloads durring src_compile(), which is prohibited and further tries to install packages. There is no quick solution for this problem, as we stared to discussed in https://github.com/gentoo-science/sci/pull/475.
Comment 3 Andrey Grozin gentoo-dev 2015-09-21 16:24:40 UTC
Sorry, I did a wrong thing. On my ~amd64 box I have ipywidgets-4.0.2-r1, not 9999. I've corrected /etc/portage/package.accept_keywords to contain

dev-python/ipywidgets ~*
dev-python/nbconvert ~*
dev-python/nbformat ~*
dev-python/ipyparallel ~*
dev-python/qtconsole ~*
dev-python/jupyter_core ~*
dev-python/ipykernel ~*
dev-python/notebook ~*
dev-python/jupyter_client ~*
dev-python/traitlets ~*
dev-python/ipython_genutils ~*
dev-python/ipython ~*
dev-python/pickleshare ~*

and now all of them have successfully installed on my ~x86 box. What shouls I do to test them - re-emerge with FEATURES=test?
Comment 4 Andrey Grozin gentoo-dev 2015-09-21 17:27:36 UTC
OK, I've emerged with FEATURES=test on ~x86 (python-2.7, 3.3, 3.4):

dev-python/ipython_genutils-0.1.0
dev-python/jupyter_core-4.0.6
dev-python/nbformat-4.0.0
dev-python/ipython-4.0.0-r2
dev-python/ipykernel-4.0.3
dev-python/jupyter_client-4.0.0
dev-python/nbconvert-4.0.0-r1
dev-python/ipywidgets-4.0.2-r1
dev-python/ipyparallel-4.0.2
dev-python/qtconsole-4.0.1-r1
dev-python/notebook-4.0.4-r3

For ipython-4.0.0-r2 I got 1 test failure:

Test group: lib
.............................E...........................................
======================================================================
ERROR: Test that latex_to_png_dvipng just runs without error.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/var/tmp/portage/dev-python/ipython-4.0.0-r2/work/ipython-4.0.0-python3_3/build/test/lib/IPython/lib/latextools.py", line 128, in latex_to_png_dvipng
    cwd=workdir, stdout=devnull, stderr=devnull)
  File "/usr/lib/python3.3/subprocess.py", line 547, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['latex', '-halt-on-error', '-interaction', 'batchmode', '/var/tmp/portage/dev-python/ipython-4.0.0-r2/temp/tmpoypqys/tmpelkp32/tmp.tex']' returned non-zero exit status 1

----------------------------------------------------------------------
Ran 73 tests in 5.468s

FAILED (errors=1)

For qtconsole-4.0.1-r1 I got a test failure:

 * python3_3: running distutils-r1_run_phase python_test
..........FQXcbConnection: Could not connect to display localhost:11.0
/var/tmp/portage/dev-python/qtconsole-4.0.1-r1/temp/environment: line 3423: 29324 Aborted                 nosetests --with-coverage --cover-package qtconsole qtconsole

but his is because I did it remotely via ssh, so, this does not count. All other tests were OK (on all 3 python versions). Tomorrow I'll have a look what went wrong with latexing tmp.tex. But I think it is already possible to keyword ~x86 all these 11 ebuilds. I have no write access to the science overlay, and I don't know how to prepare pull requests (I am new to git). So, could you please keyword them ~x86 yourself?
Comment 5 Marius Brehler 2015-09-21 18:14:21 UTC
Thanks a lot for your support Andrey. That realy speeds up getting jupyter for ~x86. I am currently preparing to push ebuilds with the added ~x86 KEYWORDS.

To get back to the bug. This is finally "solved", since only ipywidgets-9999 fails, but ipywidgets-4.0.2-r1 can be installed successfully?
Comment 6 Andrey Grozin gentoo-dev 2015-09-22 04:13:17 UTC
Yes, the original bug is solved: now I have a working jupyter on my ~x86 box. I have a student who will work on a REDUCE kernel for jupyter.

The test failure in qtconsole persists even when I work locally:

 * python3_3: running distutils-r1_run_phase python_test
..........FNo protocol specified
QXcbConnection: Could not connect to display :0
/var/tmp/portage/dev-python/qtconsole-4.0.1-r1/temp/environment: line 3423: 25907 Aborted                 nosetests --with-coverage --cover-package qtconsole qtconsole

It's not a good idea to have tests which require connection to X. I heard that some kind of virtual X is used in such cases, but, unfortunately, I don't know details. Will check it on ~amd64.

Unfortunately, I cannot debug the latex_to_png test: the temporary .tex file which gives latex error(s) is not preserved in /var/tmp/portage/dev-python/ipython-4.0.0-r2/work/... It would be interesting to check if this test succeeds on ~amd64; I'll check this in the evening when I'll be at home.
Comment 7 Marius Brehler 2015-09-22 07:19:32 UTC
I added ~x86 to the KEYWORDS for the requested packages:

dev-python/ipython_genutils
dev-python/traitlets
dev-python/jupyter_core
dev-python/nbformat
dev-python/pickleshare
dev-python/ipykernel
dev-python/ipyparallel
dev-python/ipython
dev-python/ipywidgets
dev-python/jupyter_client
dev-python/notebook
dev-python/nbconvert
dev-python/qtconsole

Further, I added ~x86 to:

dev-python/testpath
dev-python/jupyter_console



I agree that tests requiring an X connection might not be desirable, but these are the tests upstream provides. On my machine (amd64), 'ebuild qtconsole-4.0.1-r1.ebuild test' passes.

I am closing this bug, since ~x86 was added to all jupyter packages. If the problems remain, please open new bug reports, preferable as issue on github [1]. Thanks again for your support in testing.

[1] https://github.com/gentoo-science/sci/issues