Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 643692 - dev-python/pyzor: depends on old version of dev-python/redis-py and dev-python/gevent
Summary: dev-python/pyzor: depends on old version of dev-python/redis-py and dev-pytho...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Michael Orlitzky
URL:
Whiteboard:
Keywords:
Depends on: 658546
Blocks:
  Show dependency tree
 
Reported: 2018-01-06 13:09 UTC by Michał Górny
Modified: 2018-06-29 12:11 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-01-06 13:09:33 UTC
Blocks removals of old versions:

 ~dev-python/redis-py-2.9.1
 ~dev-python/gevent-1.0.1

Also has horrible partial py3 support. No revdeps. Upstream doesn't seem to have made a release since 2014, though git master seems alive.
Comment 1 Tomáš Mózes 2018-01-06 17:59:38 UTC
It works fine,been using it for ages. Since the real work is done on their servers,this does not change too much.
Comment 2 Michael Orlitzky gentoo-dev 2018-03-18 14:22:54 UTC
I'll save this... should I take over from python@ in metadata.xml?

Most people only use the client, which has no dependencies at all. And it looks like upstream is tested with the newer redis-py, so that shouldn't be a problem.

I'm willing to try the newer gevent, too, and see what happens. Their use of it is minimal, and only in the server daemon. If it really requires the older gevent, I could drop that flag and I bet no one would notice.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-03-18 17:18:39 UTC
Your choice. Python us fine as backup maint for some minor/obvious fixes, as long as we don't end up the only maint.
Comment 4 Nico Baggus 2018-03-20 13:21:16 UTC
hm. removing the redis & gevent use flags seems a viable option as well.
in my case i need only the client.
Comment 5 Tomáš Mózes 2018-03-20 15:26:49 UTC
(In reply to Nico Baggus from comment #4)
> hm. removing the redis & gevent use flags seems a viable option as well.
> in my case i need only the client.

Same here.
Comment 6 Larry the Git Cow gentoo-dev 2018-03-23 23:35:22 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4d3fe7a396ce39cfe22c324b03d4b48fd359da77

commit 4d3fe7a396ce39cfe22c324b03d4b48fd359da77
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2018-03-18 15:13:33 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2018-03-23 23:34:26 +0000

    dev-python/pyzor: new revision to fix some open bugs.
    
    Lots of changes all at once:
    
    * Update to EAPI=6.
    * Remove an old pkg_postinst() notice about a relocated executable.
    * Add IUSE=gdbm to make the gdbm daemon backend optional.
    * Only require a (gdbm, mysql, redis) backend with USE=pyzord.
    * Add a patch submitted upstream (by me) to fix a unicode crash.
    * Require sphinx with USE=doc (bug 636752).
    * Remove version constraints on redis-py and gevent (bug 643692).
    * Use HOMEPAGE to make SRC_URI fit on one line.
    * Add die() to "mv" and "rm" commands in src_install.
    * Removed seemingly-unnecessary DISTUTILS_IN_SOURCE_BUILD=1.
    * Added myself to metadata.xml as a maintainer.
    * Improved the back-end database engine USE flag descriptions.
    
    Bug: https://bugs.gentoo.org/636752
    Bug: https://bugs.gentoo.org/643692
    Package-Manager: Portage-2.3.19, Repoman-2.3.6

 .../read-stdin-as-binary-in-get_input_msg.patch    | 45 ++++++++++++++
 dev-python/pyzor/metadata.xml                      | 22 +++++--
 dev-python/pyzor/pyzor-1.0.0-r1.ebuild             | 70 ++++++++++++++++++++++
 3 files changed, 133 insertions(+), 4 deletions(-)}
Comment 7 Michael Orlitzky gentoo-dev 2018-03-23 23:36:30 UTC
It'd be nice if someone from python@ could review my changes w.r.t. the eclass, but otherwise just whoever please test it out.
Comment 8 Tomáš Mózes 2018-03-24 14:02:40 UTC
Thank you Michael for taking care of this. However the patch seems to be causing trouble on my testing machine, both on py2.7/3.5 (using default USE flags). Googling for get_binary_stdin does not find anything, any idea what's wrong please?

# spamassassin -D -t < gtube.txt
Mar 24 14:54:18.872 [23033] dbg: util: executable for pyzor was found at /usr/bin/pyzor
Mar 24 14:54:18.872 [23033] dbg: pyzor: pyzor is available: /usr/bin/pyzor
Mar 24 14:54:18.872 [23033] dbg: dns: entering helper-app run mode
Mar 24 14:54:18.873 [23033] dbg: pyzor: opening pipe: /usr/bin/pyzor check < /tmp/.spamassassin230339uUlIZtmp
Mar 24 14:54:18.875 [23042] dbg: util: setuid: ruid=0 euid=0
Mar 24 14:54:18.975 [23033] dbg: pyzor: [23042] finished: exit 1
Mar 24 14:54:18.975 [23033] dbg: pyzor: got response: Traceback (most recent call last):\n File "/usr/lib/python-exec/python2.7/pyzor", line 411, in <module>\n main()\n File "/usr/lib/python-exec/python2.7/pyzor", line 152, in main\n if not dispatch(client, servers, config):\n File "/usr/lib/python-exec/python2.7/pyzor", line 240, in check\n for digested in get_input_handler(style):\n File "/usr/lib/python-exec/python2.7/pyzor", line 177, in _get_input_msg\n msg = email.message_from_bytes(get_binary_stdin().read())\nNameError: global name 'get_binary_stdin' is not defined
Mar 24 14:54:18.975 [23033] dbg: dns: leaving helper-app run mode
Mar 24 14:54:18.975 [23033] warn: pyzor: check failed: internal error, python traceback seen in response
Comment 9 Larry the Git Cow gentoo-dev 2018-03-24 16:13:51 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=300f46c1c52cc79238e88dba28241a6e78525966

commit 300f46c1c52cc79238e88dba28241a6e78525966
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2018-03-24 16:12:23 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2018-03-24 16:12:23 +0000

    dev-python/pyzor: fix the binary stdin patch to work with v1.0.0.
    
    In my previous commit (adding -r1), I applied a new patch that I've
    submitted upstream to address a unicode crash with python-3.x. That
    patch applies cleanly against v1.0.0, but won't actually work: the
    get_binary_stdin() function it uses exists only in upstream's git
    master branch.
    
    To make the patch work (and to fix some other small issues), I've
    included the rest of the client changes between v1.0.0 and git
    master. There are very few of them -- all python-3.x fixes -- so this
    should not be too objectionable.
    
    Bug: https://bugs.gentoo.org/643692
    Package-Manager: Portage-2.3.24, Repoman-2.3.6

 .../read-stdin-as-binary-in-get_input_msg.patch    | 94 +++++++++++++++-------
 1 file changed, 67 insertions(+), 27 deletions(-)}
Comment 10 Michael Orlitzky gentoo-dev 2018-03-24 16:17:03 UTC
(In reply to Tomáš Mózes from comment #8)
> Googling for get_binary_stdin does not find anything, any idea
> what's wrong please?

Oooh sorry. I took the patch from the upstream PR and not my own /etc/portage/patches where it's working. The get_binary_stdin() function was added for python-3.x compatibility in

  https://github.com/SpamExperts/pyzor/commit/67b471dd

which I've now included in the patch.
Comment 11 Tomáš Mózes 2018-03-25 08:10:06 UTC
Thanks, silly me I didn't check the pyzor source code.

The updated ebuild now works with py2.7, but on py3.5 this error is returned:
Mar 25 09:58:02.246 [19135] dbg: util: executable for pyzor was found at /usr/bin/pyzor
Mar 25 09:58:02.246 [19135] dbg: pyzor: pyzor is available: /usr/bin/pyzor
Mar 25 09:58:02.246 [19135] dbg: dns: entering helper-app run mode
Mar 25 09:58:02.247 [19135] dbg: pyzor: opening pipe: /usr/bin/pyzor check < /tmp/.spamassassin191351yPMfRtmp
Mar 25 09:58:02.249 [19145] dbg: util: setuid: ruid=0 euid=0
Mar 25 09:58:02.439 [19135] dbg: pyzor: [19145] finished: exit 1
Mar 25 09:58:02.439 [19135] dbg: pyzor: got response: Traceback (most recent call last):\n File "/usr/lib/python-exec/python3.5/pyzor", line 429, in <module>\n main()\n File "/usr/lib/python-exec/python3.5/pyzor", line 131, in main\n config, options, args = load_configuration()\n File "/usr/lib/python-exec/python3.5/pyzor", line 110, in load_configuration\n config = configparser.ConfigParser()\nNameError: name 'configparser' is not defined
Mar 25 09:58:02.439 [19135] dbg: dns: leaving helper-app run mode
Mar 25 09:58:02.440 [19135] warn: pyzor: check failed: internal error, python traceback seen in response
Comment 12 Michael Orlitzky gentoo-dev 2018-03-25 13:37:37 UTC
> name 'configparser' is not defined

I'm so confused. If I look in /usr/lib/python-exec/python3.5/pyzor, I see...

  try:
      import configparser as ConfigParser
  except ImportError:
      import configparser

with the lowercase name (that doesn't work). But the first hunk of the patch is supposed to change those lines...

  --- a/scripts/pyzor                                                             
  +++ b/scripts/pyzor                                                             
  @@ -17,9 +17,9 @@ import tempfile
   import threading                                                               
                                                                                
   try:                                                                           
  -    import ConfigParser                                                        
  -except ImportError:                                                            
       import configparser as ConfigParser                                        
  +except ImportError:                                                            
  +    import ConfigParser                                                        
                                                                                
   import pyzor.digest                                                            
   import pyzor.client

The v1.0.0 source didn't have that line, and the patch doesn't have it. So how is the lowercase name winding up in there? I don't see the all-lowercase "import configparser" anywhere in the history or in the patch...
Comment 13 Michael Orlitzky gentoo-dev 2018-03-25 13:47:58 UTC
Is 2to3 somehow running during installation? It looks like all instances of "ConfigParser" got lowercased, and I know 2to3 will do that...
Comment 14 Michael Orlitzky gentoo-dev 2018-03-25 14:05:42 UTC
(In reply to Michael Orlitzky from comment #13)
> Is 2to3 somehow running during installation?

FFS, that's exactly what it's doing. And if I don't run 2to3 with the v1.0.0 source, then pyzord will be dead out of the box. I'll need to figure out a way to avoid running 2to3, but only for scripts/pyzor.
Comment 15 Larry the Git Cow gentoo-dev 2018-03-25 14:30:24 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4d1fb37992ef3d6cbe0cd4ed63d4b796428ccbd4

commit 4d1fb37992ef3d6cbe0cd4ed63d4b796428ccbd4
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2018-03-25 14:27:57 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2018-03-25 14:30:13 +0000

    dev-python/pyzor: add another patch to keep 2to3 from breaking things.
    
    The pyzor-1.0.0 build system still runs 2to3 and that's breaking the
    (now-patched) client, because it was patched to support python-3.x
    already. This commit adds another patch that should prevent 2to3 from
    breaking the client by giving it something to fix.
    
    Bug: https://bugs.gentoo.org/643692
    Package-Manager: Portage-2.3.24, Repoman-2.3.6

 .../files/unfix-configparser-compat-for-2to3.patch | 41 ++++++++++++++++++++++
 dev-python/pyzor/pyzor-1.0.0-r1.ebuild             |  5 ++-
 2 files changed, 45 insertions(+), 1 deletion(-)}
Comment 16 Tomáš Mózes 2018-03-26 07:22:44 UTC
Thanks, this works fine now. It also works on py3.6, so maybe we could add support for it too?
Comment 17 Larry the Git Cow gentoo-dev 2018-03-27 14:53:36 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eed09be935956cbd5f9baa073be646c26c8005d0

commit eed09be935956cbd5f9baa073be646c26c8005d0
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2018-03-27 14:47:27 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2018-03-27 14:47:27 +0000

    profiles: unmask dev-python/pyzor.
    
    I took over as the maintainer of dev-python/pyzor, and am in the
    process of resolving the open issues that caused it to be masked in
    the first place.
    
    Bug: https://bugs.gentoo.org/643692

 profiles/package.mask | 4 ----
 1 file changed, 4 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2ef966d35c16b5d0d0fe51f36f17193f6cafe492

commit 2ef966d35c16b5d0d0fe51f36f17193f6cafe492
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2018-03-27 14:46:41 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2018-03-27 14:47:06 +0000

    dev-python/pyzor: add python-3.6 support.
    
    Bug: https://bugs.gentoo.org/643692
    Package-Manager: Portage-2.3.24, Repoman-2.3.6

 dev-python/pyzor/pyzor-1.0.0-r1.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}
Comment 18 Michael Orlitzky gentoo-dev 2018-03-27 14:57:06 UTC
We'll need to stabilize the new revision before you can drop the old dependencies. Somebody remind me in a month if I forget. Assigning to myself to encourage remembrance.
Comment 19 Pacho Ramos gentoo-dev 2018-06-17 11:21:13 UTC
(In reply to Michael Orlitzky from comment #18)
> We'll need to stabilize the new revision before you can drop the old
> dependencies. Somebody remind me in a month if I forget. Assigning to myself
> to encourage remembrance.

Any news? :)
Comment 20 Larry the Git Cow gentoo-dev 2018-06-29 12:11:27 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d77406d8be24779b98b1eed7f1aab9e8e0585c91

commit d77406d8be24779b98b1eed7f1aab9e8e0585c91
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2018-06-29 12:09:10 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2018-06-29 12:09:10 +0000

    dev-python/pyzor: remove old pyzor-1.0.0.ebuild.
    
    Closes: https://bugs.gentoo.org/643692
    Package-Manager: Portage-2.3.40, Repoman-2.3.9

 dev-python/pyzor/pyzor-1.0.0.ebuild | 66 -------------------------------------
 1 file changed, 66 deletions(-)