Bug 634936 – dev-python/pyblake2: keywordreq

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Michał Górny 2017-10-21 08:32:50 UTC]

This one is needed by sys-apps/portage to support BLAKE2 on Python < 3.6, so we also need it keyworded everywhere.

[Comment 1 Sergei Trofimovich (RETIRED) 2017-10-21 09:00:11 UTC]

ia64 keyworded

[Comment 2 Fabian Groffen 2017-10-21 11:47:16 UTC]

works on Solaris, segfaults test_digest (__main__.BLAKE2bKeyedTest) on Darwin.

[Comment 3 Sergei Trofimovich (RETIRED) 2017-10-21 12:51:17 UTC]

~ppc/~ppc64 keyworded

[Comment 4 Fabian Groffen 2017-10-22 09:34:38 UTC]

Fixed Darwin by reducing optimisation levels.

[Comment 5 Michał Górny 2017-10-22 11:14:45 UTC]

(In reply to Fabian Groffen from comment #4)
> Fixed Darwin by reducing optimisation levels.

Is that a compiler or package problem?

[Comment 6 Fabian Groffen 2017-10-22 11:38:48 UTC]

It's clang-3.9.1 and the first time I ever see something like this happen.  I didn't bother to look into the code to see what happens.  The thing is, -march=native is used, which translates to -march=haswell, I suspect it enables some instructions it shouldn't, using -O1 it doesn't trigger.

[Comment 7 Mike Gilbert 2017-10-22 14:14:03 UTC]

grobian's build failure with -march=haswell is probably the same issue as bug 608586.

We could probably fix it using the same patch.

https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/python/files/3.6-blake2.patch?id=61958b88555c08f099c70d0b1989b86bcd3f2465

[Comment 8 Mike Gilbert 2017-10-22 14:21:01 UTC]

Err, maybe not. It looks like grobian put in some weird code to work around a segfault by messing with CFLAGS? Yuck.

[Comment 9 Michał Górny 2017-10-22 17:33:47 UTC]

Yes, it is awful. Furthermore, I'm pretty sure toolchain-funcs has something sane to replace arch. However, I'd like to know what the problem is and if it really Darwin-specific here.

[Comment 10 Fabian Groffen 2017-10-23 08:42:46 UTC]

the reason I do it like that is because:
-O1 -march=haswell
-O2 -march=ivybridge
both work, but the optimisation level benefits more than the -march.  A little complication is the use of -march=native.  A knee-jerk fix would be to replace -O[23456789] with -O1, to avoid the entire complexity, but since this is a hashing thing we're going to use frequently (with Portage) I figured it was warranted to do a little fuglyness to make it as fast as can be (in the little time I had to deal with it).

[Comment 11 Matt Turner 2017-11-10 01:09:08 UTC]

added ~mips

[Comment 12 Larry the Git Cow 2017-11-13 22:17:11 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fafd9f00168e11964b46ed0542879c8fbbad1a42

commit fafd9f00168e11964b46ed0542879c8fbbad1a42
Author:     Robin H. Johnson <[e-mail]>
AuthorDate: 2017-11-13 22:15:18 +0000
Commit:     Robin H. Johnson <[e-mail]>
CommitDate: 2017-11-13 22:17:02 +0000

    dev-python/pyblake2: keyword ~hppa
    
    Passes src_test on hake.hppa.dev.gentoo.org, for py27 & py34.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.8, Repoman-2.3.3

 dev-python/pyblake2/pyblake2-1.0.0.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}

[Comment 13 Larry the Git Cow 2017-11-14 03:41:52 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5c653218772342acc4c43a833c42c571f32bba2f

commit 5c653218772342acc4c43a833c42c571f32bba2f
Author:     Robin H. Johnson <[e-mail]>
AuthorDate: 2017-11-13 22:15:18 +0000
Commit:     Robin H. Johnson <[e-mail]>
CommitDate: 2017-11-14 03:41:44 +0000

    sys-apps/portage: keyword ~hppa
    
    Passes src_test on hake.hppa.dev.gentoo.org, for py27 & py34.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.8, Repoman-2.3.3

 sys-apps/portage/portage-2.3.13-r1.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}

[Comment 14 Robin Johnson 2017-11-14 03:42:40 UTC]

hppa done using hake.hppa.dev.gentoo.org.

[Comment 15 Zac Medico 2017-11-15 01:24:44 UTC]

Please also keyword app-portage/repoman-2.3.6 since I had to drop the same keywords there.

[Comment 16 Zac Medico 2017-11-15 01:29:49 UTC]

Also sys-apps/portage-2.3.14 needs ~arch keywords.

[Comment 17 Sergei Trofimovich (RETIRED) 2017-11-15 07:36:48 UTC]

~sparc keyworded

[Comment 18 Matt Turner 2017-11-15 19:22:14 UTC]

added ~alpha

[Comment 19 Richard Gray 2017-11-27 11:12:31 UTC]

I think I've found another problem with pyblake2.

The current version of portage now has pyblake2 package as a dependency, and although that package installs OK, it is impossible to do anything further with an emerge command or an ebuild command because they terminate with an "Illegal Instruction". If you do a ulimit -c unlimited, a core dump is generated, but it's not much help.

After much delving, I've found that the problem lies in pyblake2.so, and here's an example of what ends-up in the syslog...

Nov 27 10:36:01 newton3 kernel: traps: emerge[9800] trap invalid opcode ip:7f93b276a797 sp:7ffd904ad068 error:0 in pyblake2.so[7f93b2767000+8000]

I strongly suspect that the problem is that I'm using a rather old (but perfectly adequate) core2 [Pentium D] machine which does not support an opcode that is available on a more recent CPU. I'm uncertain of whether this is a compiler issue or it's simply because an incorrect assumption has been made somewhere.

For the less determined user, this will create much misery in that after updating portage to the latest version, it's impossible to install or update anything else.

I manually reinstalled the earlier portage and things are working again, but this will be a real crisis for some users.

My compiler options are (from make.conf)

CFLAGS="-O2 -pipe -march=core2"

Do let me know if I can be of further assistance.

[Comment 20 Michał Górny 2017-11-27 12:00:47 UTC]

Please file a separate bug for that since it's going to be lost in all the comments here. Then state the exact version of pyblake2 (and try newest ~arch). Possibly try -march=native to rule out wrong flags, and generic -march=x86_64. Enable FEATURES=test to have the package fail before being installed, and 'emerge -C pyblake2' if you need to unbreak your system.

[Comment 21 Larry the Git Cow 2017-11-27 22:19:16 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2e99bfee112c58ef087924696f6ec976fc8eb588

commit 2e99bfee112c58ef087924696f6ec976fc8eb588
Author:     James Le Cuirot <[e-mail]>
AuthorDate: 2017-11-27 22:14:41 +0000
Commit:     James Le Cuirot <[e-mail]>
CommitDate: 2017-11-27 22:19:05 +0000

    app-portage/repoman: Add ~arm keyword
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.16, Repoman-2.3.6

 app-portage/repoman/Manifest             | 8 ++++----
 app-portage/repoman/repoman-2.3.6.ebuild | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bea2ee45aae810f35129700f220aaf47c0d46f39

commit bea2ee45aae810f35129700f220aaf47c0d46f39
Author:     James Le Cuirot <[e-mail]>
AuthorDate: 2017-11-27 22:11:07 +0000
Commit:     James Le Cuirot <[e-mail]>
CommitDate: 2017-11-27 22:18:52 +0000

    sys-apps/portage: Add ~arm keyword
    
    Passes src_test with py27 and py35 on Utilite Pro.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.16, Repoman-2.3.6

 sys-apps/portage/Manifest                 | 4 ++--
 sys-apps/portage/portage-2.3.13-r1.ebuild | 2 +-
 sys-apps/portage/portage-2.3.14.ebuild    | 2 +-
 sys-apps/portage/portage-2.3.16.ebuild    | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ec16537e9987abf0af1bc34074c363f5bef60f63

commit ec16537e9987abf0af1bc34074c363f5bef60f63
Author:     James Le Cuirot <[e-mail]>
AuthorDate: 2017-11-27 22:06:05 +0000
Commit:     James Le Cuirot <[e-mail]>
CommitDate: 2017-11-27 22:18:39 +0000

    dev-python/pyblake2: Add ~arm keyword
    
    Passes src_test with py27 and py35 on Utilite Pro.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.16, Repoman-2.3.6

 dev-python/pyblake2/Manifest              | 2 +-
 dev-python/pyblake2/pyblake2-1.1.0.ebuild | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)}

[Comment 22 Richard Gray 2017-12-04 04:31:24 UTC]

(In reply to Michał Górny from comment #20)
> Please file a separate bug for that since it's going to be lost in all the
> comments here. Then state the exact version of pyblake2 (and try newest
> ~arch). Possibly try -march=native to rule out wrong flags, and generic
> -march=x86_64. Enable FEATURES=test to have the package fail before being
> installed, and 'emerge -C pyblake2' if you need to unbreak your system.

Thanks for the guidance, Michał. I'll create another bug.

[Comment 23 Larry the Git Cow 2017-12-21 16:03:27 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1df4219acef5a76e0bb5bbb664a2ce9a843afdea

commit 1df4219acef5a76e0bb5bbb664a2ce9a843afdea
Author:     Michael Haubenwallner <[e-mail]>
AuthorDate: 2017-12-21 16:02:33 +0000
Commit:     Michael Haubenwallner <[e-mail]>
CommitDate: 2017-12-21 16:02:58 +0000

    dev-python/pyblake2: keyword ~x64-cygwin
    
    Passes src_test for py27.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.13, Repoman-2.3.3

 dev-python/pyblake2/pyblake2-1.1.0.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}

[Comment 24 Markus Meier 2017-12-22 05:23:33 UTC]

~arm64 added

[Comment 25 Larry the Git Cow 2017-12-27 15:26:23 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4b47038cdaa5322c85e3ab11b9c2f60342c1d288

commit 4b47038cdaa5322c85e3ab11b9c2f60342c1d288
Author:     Yuta Satoh <[e-mail]>
AuthorDate: 2017-12-27 13:59:24 +0000
Commit:     Mike Gilbert <[e-mail]>
CommitDate: 2017-12-27 15:25:58 +0000

    dev-python/pyblake2: add ~amd64-fbsd, ~x86-fbsd KEYWORDS.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.13, Repoman-2.3.3
    Closes: https://github.com/gentoo/gentoo/pull/6660

 dev-python/pyblake2/pyblake2-1.1.0.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}

[Comment 26 Larry the Git Cow 2018-01-22 16:08:03 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4273a169a77eb87ef833c6d99a81c997938fecf2

commit 4273a169a77eb87ef833c6d99a81c997938fecf2
Author:     Michael Haubenwallner <[e-mail]>
AuthorDate: 2018-01-22 16:04:25 +0000
Commit:     Michael Haubenwallner <[e-mail]>
CommitDate: 2018-01-22 16:07:48 +0000

    dev-python/pyblake2: add ~x86-linux, ~amd64-linux keywords
    
    These are the non-RAP keywords for linux prefix.
    
    Bug: https://bugs.gentoo.org/634936
    Package-Manager: Portage-2.3.19, Repoman-2.3.6

 dev-python/pyblake2/pyblake2-1.1.0.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)}

[Comment 27 Mikle Kolyada (RETIRED) 2018-06-03 17:03:30 UTC]

s390 was done

[Comment 28 Mikle Kolyada (RETIRED) 2018-09-16 08:13:49 UTC]

sh and m68k handled straught to stable

[Comment 29 NATTkA bot 2020-04-06 15:25:34 UTC]

Unable to check for sanity:

> no match for package: sys-apps/portage-2.3.24-r1

[Comment 30 Zac Medico 2020-04-06 16:38:26 UTC]

This bug is obsolete since latest portage releases have dropped Python < 3.6 from PYTHON_COMPAT:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0f0f20554b618bec9f337dadacc3c9e8ca8c92d5