Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 589994 - www-client/phantomjs: mask for removal
Summary: www-client/phantomjs: mask for removal
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo TreeCleaner Project
URL:
Whiteboard: Pending removal: 2017-09-05
Keywords: PMASKED
Depends on: 596536 596538
Blocks:
  Show dependency tree
 
Reported: 2016-07-29 13:40 UTC by Michael Orlitzky
Modified: 2018-07-20 20:36 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
patch for building 2.5.0 (file_589994.txt,1000 bytes, patch)
2017-06-12 14:40 UTC, Daniel Maslowski
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Orlitzky gentoo-dev 2016-07-29 13:40:00 UTC
I just looked into doing a version bump for this package, and I'm horrified. It bundles all of qt-base, qtwebkit, qcommandline, mongoose, linenoise, breakpad...
 old versions, with known security issues (particularly qtwebkit).

The build-system is hand-rolled. It has a number of bugs open against it, and no one wants to maintain it. In my opinion this has no business being in the tree in its current form, and probably can't be fixed to meet basic QA standards.

Unfortunately there are two packages depending upon it. There's dev-ruby/poltergeist, which would probably have to be masked as well, since phantomjs is its raison d'etre. The other is dev-python/bokeh-0.7.1; fortunately, newer versions have dropped the dependency on phantomjs, so we would only need to stabilize something newer than v0.7.1 and drop v0.7.1.

Maintainers of those packages, please comment if this all sounds reasonable.
Comment 1 Hans de Graaff gentoo-dev 2016-07-31 06:19:13 UTC
I agree the whole thing is a mess, but it's also a useful tool. I see upstream is currently trying to fix at least some of the issues: https://github.com/ariya/phantomjs/pull/14386

Masking dev-ruby/poltergeist would also affect dev-ruby/best_in_place, but we can probably limit tests there (or mask it as well as it does not have reverse dependencies).
Comment 2 Tomáš Mózes 2016-10-13 11:42:16 UTC
(In reply to Michael Orlitzky from comment #0)
> The build-system is hand-rolled. It has a number of bugs open against it,
> and no one wants to maintain it. In my opinion this has no business being in
> the tree in its current form, and probably can't be fixed to meet basic QA
> standards.

How about we introduce a bin version (that never gets stabilized)?
Comment 3 Michael Orlitzky gentoo-dev 2016-10-13 11:58:03 UTC
(In reply to Tomáš Mózes from comment #2)
> 
> How about we introduce a bin version (that never gets stabilized)?

That would eliminate the build issues, but the biggest problem it has is the bundled qtwekbit, which has a number of known security vulnerabilities already fixed in our dev-qt/qtwebkit. If you don't unbundle qtwebkit, the -bin version would have the same problem. 

The pull request that Hans pointed out is now closed. Maybe it's possible to build phantomjs against the system libs now? Even then, it has no maintainer, so no one is likely to do the work. If no one cares enough to fix it, all that we have in the tree now are vulnerable versions.

I was going to attempt a version bump back when I filed this bug, but that was before the semester started...
Comment 4 Tomáš Mózes 2016-10-13 12:31:02 UTC
It's a handy tool for automatic tests, so probably it won't be installed on your critical servers but rather on some CI/test server. We for example use it to test our web applications, but we just start it when we need it and turn it off afterwards.

In other words, it's not like a web server that you are facing to the Internet, but more like something running locally.

It would be a pity to remove it completely from Portage I suppose.
Comment 5 Hans de Graaff gentoo-dev 2016-10-15 08:27:42 UTC
(In reply to Michael Orlitzky from comment #3)

> The pull request that Hans pointed out is now closed. Maybe it's possible to
> build phantomjs against the system libs now? Even then, it has no
> maintainer, so no one is likely to do the work. If no one cares enough to
> fix it, all that we have in the tree now are vulnerable versions.

Yes, looks like that code has been merged and it should now be possible to build against system Qt and more importantly QtWebkit. I looked at adding a current snapshot for this since there is no released version yet. Building phantomjs works fine and builds against system QtWebkit, but almost all tests fail with segmentation faults. This may be bad luck in picking the latest upstream commit, or something in the way phantomjs gets built, not sure yet.
Comment 6 Hans de Graaff gentoo-dev 2016-11-17 06:44:36 UTC
I have an ebuild for current master of the project which no longer bundles anything but links to qtwebkit. It builds fine and actually works for my use case but it also segfaults a lot for most of the test suite cases.

I'll check again next week to see if additional changes are available upstream, and in any case I can add a masked version for further testing.
Comment 7 Pacho Ramos gentoo-dev 2017-06-04 11:30:36 UTC
The situation is still really ugly, we cannot treeclean this due to bug 596538 , current stable is not compiling with gcc5 (now stable), and 2.0.0-r1 doesn't even compile for me :/

Traceback (most recent call last):
  File "/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/disassembler/udis86/itab.py", line 359, in <module>
    main()
  File "/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/disassembler/udis86/itab.py", line 354, in main
    optableXmlParser.parse( args[ 0 ], generator.addInsnDef )
  File "/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/disassembler/udis86/ud_optable.py", line 83, in parse
    fn( prefixes, mnemonic, opcodes, operands, vendor )
  File "/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/disassembler/udis86/ud_opcode.py", line 217, in addInsnDef
    vendor=vendor)
  File "/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/disassembler/udis86/ud_opcode.py", line 157, in __init__
    self.opcext[arg] = self.OpcExtMap[arg](val)
  File "/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/disassembler/udis86/ud_opcode.py", line 120, in <lambda>
    '/m'     : lambda v: "%02x" % (int(v) / 32),
TypeError: %x format: an integer is required, not float
make[2]: *** [Makefile.JavaScriptCore.DerivedSources:2344: generated/udis86_itab.c] Error 1
make[2]: *** Waiting for unfinished jobs....
offlineasm: Including file /var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm
offlineasm: Including file /var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm
offlineasm: Assembly file generated/LLIntAssembly.h successfully generated.
make[2]: Leaving directory '/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore'
make[1]: *** [Makefile.JavaScriptCore:66: sub-DerivedSources-pri-make_first-ordered] Error 2
make[1]: Leaving directory '/var/tmp/portage/www-client/phantomjs-2.0.0-r1/work/phantomjs-2.0.0/src/qt/qtwebkit/Source/JavaScriptCore'
make: *** [Makefile:88: sub-Source-JavaScriptCore-JavaScriptCore-pro-make_first-ordered] Error 2
 * ERROR: www-client/phantomjs-2.0.0-r1::gentoo failed (compile phase):

If this is only needed by dev-ruby/best_in_place and that is not needed, I would treeclean the tree packages
Comment 8 Hans de Graaff gentoo-dev 2017-06-05 07:10:19 UTC
# Hans de Graaff <graaff@gentoo.org> (05 Jun 2017)
# Bundles obsolete and vulnerable webkit version.
# Upstream has stopped development and recommends using
# headless mode in >=www-client/chromium-59.
# Masked for removal in 30 days. Bug #589994.
www-client/phantomjs
dev-ruby/poltergeist
Comment 9 Pacho Ramos gentoo-dev 2017-06-05 08:51:44 UTC
Thanks :)

Please keep this opened until this is finally removed from the tree ;)
Comment 10 Daniel Maslowski 2017-06-12 14:40:37 UTC
Created attachment 476138 [details, diff]
patch for building 2.5.0

I've successfully built 2.5.0 once from the master branch (a few months ago, on Gentoo and Arch Linux, respectively) using a more recent Qt WebKit as shared libs (see the patch I attached). I could try fixing this ebuild here, if there is still demand and that version of Qt WebKit is not already patched against the vulnerabilities mentioned here. Could you please provide some more information on the version etc.?
Comment 11 Hans de Graaff gentoo-dev 2017-06-13 04:38:45 UTC
(In reply to verchiel from comment #10)
> Created attachment 476138 [details, diff] [details, diff]
> patch for building 2.5.0
> 
> I've successfully built 2.5.0 once from the master branch (a few months ago,
> on Gentoo and Arch Linux, respectively) using a more recent Qt WebKit as
> shared libs (see the patch I attached). I could try fixing this ebuild here,
> if there is still demand and that version of Qt WebKit is not already
> patched against the vulnerabilities mentioned here. Could you please provide
> some more information on the version etc.?

Do all tests pass? In my 2.5.0 build most tests caused segmentation faults even when building was successful. In any case I don't think we would add/unmask phantomjs without an active upstream.
Comment 12 Daniel Maslowski 2017-06-13 06:07:01 UTC
(In reply to Hans de Graaff from comment #11)
> 
> Do all tests pass? In my 2.5.0 build most tests caused segmentation faults
> even when building was successful. In any case I don't think we would
> add/unmask phantomjs without an active upstream.

Yes, I agree with that. A few facts for conclusion:
- the most recent commit was 4 months ago (as of now)
- the tests do indeed not pass
- I used the most recent Qt WebKit I could find
- Qt is dropping WebKit in favor of the Blink-based QWebPage

Other options:
- use Chromium >=58 in headless mode directly
- NW.js (formerly node-webkit, https://nwjs.io/), based on Node 8.0.0 and Chromium 59 (currently)
- Electron (https://electron.atom.io/), with the current 1.7.x beta based on Node 7.9.0, v8 5.8.283.38 and Chrome 58 or 1.6.x stable based on Node 7.4.0, v8 5.6.326.50 and Chrome 56

We have ebuilds with slot and PaX support for Electron, and more specifically, v1.3.13 is the latest version. It's not that easy to bump though, so big kudos to elprans for the efforts to have it in the first place. In my company, we switched to using Electron. :) I'm using a binary release currently.
Comment 13 Herb Miller Jr. 2017-08-16 01:59:08 UTC
Please don't remove this just yet!

I can't comment on the build system or the bundling, but QtWebKit is no longer deprecated/outdated per bug #624404.

A dev who goes by the username Vitallium has been working closely with the main QtWebKit dev getting phantomjs up to speed, so it's not stagnant. I would be happy to take over phantomjs ebuilds as a proxy maintainer once QtWebKit 5.212 is pulled in. I've been working with upstream for about a year now since I use the new QtWebKit in a product for my employer.
Comment 14 Tony Vroon gentoo-dev 2017-08-17 14:17:48 UTC
(In reply to Herb Miller Jr. from comment #13)
> I would be happy to take over phantomjs ebuilds as a proxy maintainer

Do you have overlay ebuilds that build now? Or are you actively working with "verchiel" on the patches already produced?
Comment 15 Tony Vroon gentoo-dev 2017-09-19 15:22:51 UTC
(In reply to Herb Miller Jr. from comment #13)
> Please don't remove this just yet!

Okay, I have committed a 2.1.1 ebuild to the tree.
Comment 16 Tony Vroon gentoo-dev 2017-09-19 15:24:24 UTC
(In reply to Hans de Graaff from comment #8)
> # Hans de Graaff <graaff@gentoo.org> (05 Jun 2017)
> # Bundles obsolete and vulnerable webkit version.

Counter-offer, keep masked due to webkit vulnerabilities (I got it working with 5.7.1 but I doubt that's fully secure) but do not fully remove. Is that acceptable?
I will keep working on this 2.1.1 for now, until an actual 2.5.0 comes out.
Comment 17 Pacho Ramos gentoo-dev 2017-09-19 15:42:56 UTC
I think it's ok from my point of view, also, it seems you took the package, thanks! :)