Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 626550

Summary: dev-python/scrapy: removal
Product: Gentoo Linux Reporter: Hannan Sadar <hannan.sadar>
Component: Current packagesAssignee: Maxim Koltsov (RETIRED) <maksbotan>
Status: RESOLVED FIXED    
Severity: normal CC: jstein, mgorny, python, qa, vdupras
Priority: Normal Keywords: PMASKED
Version: unspecified   
Hardware: All   
OS: Linux   
URL: https://scrapy.org
Whiteboard: Pending removal: 2018-09-27
Package list:
Runtime testing required: ---
Bug Depends on: 626612, 626610, 626680    
Bug Blocks:    
Attachments: ebuild for scrapy
ebuild for parsel
ebuild for w3lib
ebuild for dev-python/scrapy-1.4.0
fix of syntax and dependency versions
ebuild for dev-python/parsel-1.2.0

Description Hannan Sadar 2017-07-29 19:35:49 UTC
A new version of scrapy is existed, 1.4.0.

Reproducible: Always
Comment 1 Hannan Sadar 2017-07-29 19:37:09 UTC
Created attachment 487228 [details]
ebuild for scrapy
Comment 2 Hannan Sadar 2017-07-29 19:37:34 UTC
Created attachment 487230 [details]
ebuild for parsel
Comment 3 Hannan Sadar 2017-07-29 19:37:59 UTC
Created attachment 487232 [details]
ebuild for w3lib
Comment 4 Hannan Sadar 2017-07-29 19:39:38 UTC
with those 3 ebuilds I was able to build scrapy-1.4.0

Remark:
I have not tested all uses in the ebuilds
Comment 5 Jonas Stein gentoo-dev 2017-07-29 19:57:50 UTC
Thank you for your contribution. I had a short look on the ebuild. 
Here a few ideas:

* please create one ticket per package, have a look on the title/summary, how we write it. There are some scripts helping us. They need this syntax.

* Please test all ebuilds with repoman https://wiki.gentoo.org/wiki/Repoman
Check, that headers are up to date https://devmanual.gentoo.org/ebuild-writing/eapi/

* please fix the indentation
Comment 6 Hannan Sadar 2017-07-30 14:22:11 UTC
Created attachment 487374 [details]
ebuild for dev-python/scrapy-1.4.0
Comment 7 Hannan Sadar 2017-07-30 14:53:50 UTC
Created attachment 487376 [details]
fix of syntax and dependency versions

Nether the less, it is unfortunately not "repoman -dx full" error free.
Comment 8 Hannan Sadar 2017-07-30 18:23:30 UTC
Created attachment 487414 [details]
ebuild for dev-python/parsel-1.2.0

This ebuild failed repoman -dx full.

Once the support of python 3.5 and 3.6 is removed the ebuild pass the test.
Comment 9 tonemgub 2017-08-23 14:57:12 UTC
Needed the newer version of scrapy and found this bug. Also curious if this can be built against pycryptodome instead of pycrypto, as the latter is extremely outdated and vulnerable.
Comment 10 Hannan Sadar 2017-08-23 19:16:53 UTC
(In reply to tonemgub from comment #9)
> Needed the newer version of scrapy and found this bug. Also curious if this
> can be built against pycryptodome instead of pycrypto, as the latter is
> extremely outdated and vulnerable.

unfortunately this ebuild in Bug is still waiting to be accepted...
(I use it and it works just fine)

Concerning you question about dev-python/pycrypto, this ebuild was not build on my computer and I was still able to build scrapy v1.4.0.

Why do you think it is needed?
Comment 11 tonemgub 2017-10-11 15:57:11 UTC
(In reply to Hannan Sadar from comment #10)
>
> Concerning you question about dev-python/pycrypto, this ebuild was not build
> on my computer and I was still able to build scrapy v1.4.0.
> 
> Why do you think it is needed?

Got pulled in as a depend. 

...
Calculating dependencies... done!
[ebuild  N    ~] dev-python/w3lib-1.17.0  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_5 -python3_6" 
[ebuild  N     ] dev-python/queuelib-1.1.1  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 
[ebuild  N     ] dev-python/cssselect-0.9.1  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 
[ebuild  N     ] dev-python/twisted-names-13.2.0  PYTHON_TARGETS="python2_7" 
[ebuild  N     ] dev-python/pyasn1-modules-0.0.8  PYTHON_TARGETS="python2_7 python3_4 (-pypy) (-pypy3) (-python3_5) (-python3_6)" 
[ebuild  N     ] dev-python/attrs-16.3.0-r1  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 (-pypy) (-python3_5) (-python3_6)" 
[ebuild  N     ] dev-python/service_identity-16.0.0  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 (-pypy) (-python3_5) (-python3_6)" 
[ebuild  N     ] dev-python/twisted-mail-13.2.0  PYTHON_TARGETS="python2_7" 
[ebuild  N     ] dev-python/pycrypto-2.6.1-r2  USE="gmp -doc {-test}" PYTHON_TARGETS="python2_7 python3_4 (-python3_5) (-python3_6)" 
[ebuild  N     ] dev-python/twisted-conch-13.2.0  PYTHON_TARGETS="python2_7" 
[ebuild  N    ~] dev-python/scrapy-1.0.3  USE="ssl -boto -doc -ibl {-test}" PYTHON_TARGETS="python2_7" 
[blocks B      ] dev-python/pycrypto ("dev-python/pycrypto" is blocking dev-python/pycryptodome-3.4.6)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-python/pycryptodome-3.4.6:0/0::gentoo, installed) pulled in by
    dev-python/pycryptodome required by @selected

  (dev-python/pycrypto-2.6.1-r2:0/0::gentoo, ebuild scheduled for merge) pulled in by
    dev-python/pycrypto[python_targets_python2_7(-)?,-python_single_target_python2_7(-)] (dev-python/pycrypto[python_targets_python2_7(-),-python_single_target_python2_7(-)]) required by (dev-python/twisted-conch-13.2.0:0/0::gentoo, ebuild scheduled for merge)
....

Appears to actually be bug #611618 for dev-python/twisted-conch.
Comment 12 Larry the Git Cow gentoo-dev 2017-10-28 13:37:03 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=67b3229a676c2d240a8027f84bf91b9f6e29bbe5

commit 67b3229a676c2d240a8027f84bf91b9f6e29bbe5
Author:     Bob Brooks <gitbugged@cool.fr.nf>
AuthorDate: 2017-10-23 16:55:22 +0000
Commit:     Patrice Clement <monsieurp@gentoo.org>
CommitDate: 2017-10-28 13:36:57 +0000

    dev-python/w3lib: version bump to 1.18.0.
    
    Closes: https://bugs.gentoo.org/626550
    Closes: https://github.com/gentoo/gentoo/pull/6028

 dev-python/w3lib/Manifest            |  1 +
 dev-python/w3lib/w3lib-1.18.0.ebuild | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+)
Comment 13 Larry the Git Cow gentoo-dev 2017-10-28 13:41:17 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d1268374755277dfe0323d5cdef9e2b4f6234fd6

commit d1268374755277dfe0323d5cdef9e2b4f6234fd6
Author:     Bob Brooks <gitbugged@cool.fr.nf>
AuthorDate: 2017-10-23 17:31:24 +0000
Commit:     Patrice Clement <monsieurp@gentoo.org>
CommitDate: 2017-10-28 13:40:44 +0000

    dev-python/scrapy: version bump to 1.4.0.
    
    Bug: https://bugs.gentoo.org/626550
    Closes: https://github.com/gentoo/gentoo/pull/6029

 dev-python/scrapy/Manifest            |  1 +
 dev-python/scrapy/scrapy-1.4.0.ebuild | 80 +++++++++++++++++++++++++++++++++++
 2 files changed, 81 insertions(+)}
Comment 14 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-08-26 09:25:07 UTC
This is certainly not fixed since the ebuild has been merged with missing deps, and masked afterwards with no action to fix it since.
Comment 15 Larry the Git Cow gentoo-dev 2018-08-26 09:39:42 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bc7d04a381c2f5b3195f0317e694c288ce72b986

commit bc7d04a381c2f5b3195f0317e694c288ce72b986
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2018-08-26 09:25:51 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2018-08-26 09:25:51 +0000

    dev-python/scrapy: Revert "version bump to 1.4.0."
    
    The version bump had missing dependencies which were not fixed since it
    was merged.
    
    Bug: https://bugs.gentoo.org/626550

 dev-python/scrapy/Manifest            |  1 -
 dev-python/scrapy/scrapy-1.4.0.ebuild | 81 -----------------------------------
 2 files changed, 82 deletions(-)
Comment 16 Virgil Dupras (RETIRED) gentoo-dev 2018-08-26 12:52:51 UTC
Maxim wrote in bug 589620 that he doesn't want to maintain it, the python team seems to be unable to maintain it too, badly outdated, no revdeps. Anyone objects to last rites?
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-08-26 17:28:45 UTC
Please do if you believe so.  I think the usual 30 days should be enough for any proxied maintainer to stand up.
Comment 18 Virgil Dupras (RETIRED) gentoo-dev 2018-08-27 12:22:18 UTC
Last riting scrapy as well as two scrapy-related projects that have no revdeps other than scrapy:

dev-python/w3lib
dev-python/queuelib
Comment 19 Larry the Git Cow gentoo-dev 2018-08-27 12:31:39 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9d39b3430e097100f727b8b64e0ba3c1ff7e2846

commit 9d39b3430e097100f727b8b64e0ba3c1ff7e2846
Author:     Virgil Dupras <vdupras@gentoo.org>
AuthorDate: 2018-08-27 12:29:14 +0000
Commit:     Virgil Dupras <vdupras@gentoo.org>
CommitDate: 2018-08-27 12:29:14 +0000

    profiles: mask dev-python/scrapy for removal
    
    (as well as 2 scrapy-specific deps).
    
    Bug: https://bugs.gentoo.org/626550

 profiles/package.mask | 7 +++++++
 1 file changed, 7 insertions(+)
Comment 20 Larry the Git Cow gentoo-dev 2018-09-27 13:07:55 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4896a6290d3a38f137530d07f05dc058c6321074

commit 4896a6290d3a38f137530d07f05dc058c6321074
Author:     Virgil Dupras <vdupras@gentoo.org>
AuthorDate: 2018-09-27 13:06:48 +0000
Commit:     Virgil Dupras <vdupras@gentoo.org>
CommitDate: 2018-09-27 13:06:48 +0000

    profiles: remove dev-python/scrapy obsolete last-rites mask
    
    Closes: https://bugs.gentoo.org/626550
    Signed-off-by: Virgil Dupras <vdupras@gentoo.org>

 profiles/package.mask | 7 -------
 1 file changed, 7 deletions(-)

Additionally, it has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2f5bf2f6fdc27560be6fc0fb1119717ee9a2c68f

commit 2f5bf2f6fdc27560be6fc0fb1119717ee9a2c68f
Author:     Virgil Dupras <vdupras@gentoo.org>
AuthorDate: 2018-09-27 13:06:03 +0000
Commit:     Virgil Dupras <vdupras@gentoo.org>
CommitDate: 2018-09-27 13:06:03 +0000

    dev-python/scrapy: remove last-rited package
    
    Bug: https://bugs.gentoo.org/626550
    Signed-off-by: Virgil Dupras <vdupras@gentoo.org>

 dev-python/scrapy/Manifest            |  1 -
 dev-python/scrapy/metadata.xml        | 18 --------
 dev-python/scrapy/scrapy-1.0.3.ebuild | 79 -----------------------------------
 3 files changed, 98 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9fb89deb17260beb80c2d06de6f6a9f02226c294

commit 9fb89deb17260beb80c2d06de6f6a9f02226c294
Author:     Virgil Dupras <vdupras@gentoo.org>
AuthorDate: 2018-09-27 13:05:29 +0000
Commit:     Virgil Dupras <vdupras@gentoo.org>
CommitDate: 2018-09-27 13:05:29 +0000

    dev-python/w3lib: remove last-rited package
    
    Bug: https://bugs.gentoo.org/626550
    Signed-off-by: Virgil Dupras <vdupras@gentoo.org>

 dev-python/w3lib/Manifest            |  2 --
 dev-python/w3lib/metadata.xml        | 15 ---------------
 dev-python/w3lib/w3lib-1.18.0.ebuild | 20 --------------------
 dev-python/w3lib/w3lib-1.19.0.ebuild | 20 --------------------
 4 files changed, 57 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ee6fb6310e16226019b7ab07295d6d59f84c3ac4

commit ee6fb6310e16226019b7ab07295d6d59f84c3ac4
Author:     Virgil Dupras <vdupras@gentoo.org>
AuthorDate: 2018-09-27 13:04:31 +0000
Commit:     Virgil Dupras <vdupras@gentoo.org>
CommitDate: 2018-09-27 13:04:31 +0000

    dev-python/queuelib: remove last-rited package
    
    Bug: https://bugs.gentoo.org/626550
    Signed-off-by: Virgil Dupras <vdupras@gentoo.org>

 dev-python/queuelib/Manifest              |  1 -
 dev-python/queuelib/metadata.xml          | 11 -----------
 dev-python/queuelib/queuelib-1.1.1.ebuild | 29 -----------------------------
 3 files changed, 41 deletions(-)