Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 478124

Summary: app-crypt/johntheripper-1.7.9-r8: openmpi is not working / multiple threads is not working
Product: Gentoo Linux Reporter: Steffen <tomatenesser>
Component: Current packagesAssignee: Rick Farina (Zero_Chaos) <zerochaos>
Status: UNCONFIRMED ---    
Severity: normal CC: alonbl, crypto+disabled, dschridde+gentoobugs
Priority: Normal    
Version: unspecified   
Hardware: AMD64   
OS: Linux   
URL: https://forums.gentoo.org/viewtopic-t-964052.html
Whiteboard:
Package list:
Runtime testing required: ---

Description Steffen 2013-07-25 12:17:55 UTC
I expect openmpi to increase the speed of johntheripper instead of opening single threads.



Reproducible: Always

Steps to Reproduce:
1. Compile johntheripper with openmpi support
2. mpiexec -np 3 john --test --format=raw-md5

Actual Results:  
mpiexec -np 3 john --test --format=raw-md5
Benchmarking: Raw MD5 [128/128 SSE2 intrinsics 12x]... Benchmarking: Raw MD5 [128/128 SSE2 intrinsics 12x]... Benchmarking: Raw MD5 [128/128 SSE2 intrinsics 12x]... DONE
Raw:   10626K c/s real, 10626K c/s virtual
DONE
Raw:   10579K c/s real, 10579K c/s virtual
DONE
Raw:   10459K c/s real, 10459K c/s virtual 



Expected Results:  
There should be only one line with about 30000K c/s real, ... 

emerge --info
Portage 2.1.12.13 (default/linux/amd64/13.0, gcc-4.7.3, glibc-2.17, 3.9.4-gentoo x86_64)
=================================================================
System uname: Linux-3.9.4-gentoo-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q9550_@_2.83GHz-with-gentoo-2.2
KiB Mem:     8179592 total,   3156468 free
KiB Swap:   16394328 total,  16394328 free
Timestamp of tree: Sun, 21 Jul 2013 15:15:01 +0000
ld GNU ld (GNU Binutils) 2.23.1
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.5-r1, 3.2.5-r1, 3.3.2-r1
dev-util/cmake:           2.8.11.1
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.11.8
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.14
sys-devel/binutils:       2.23.1
sys-devel/gcc:            4.7.3
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.9 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo x-portage
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA PUEL AdobeFlash-11.x Oracle-BCLA-JavaSE dlj-1.1"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/"
LANG="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X acl alsa amd64 apache2 aspell berkdb bzip2 cairo cli consolekit cracklib crypt cups curl cxx dbus dri emacs fftw fortran gd gdbm gif gpm gtk gtk2 gtk3 iconv imap ipv6 jpeg jpeg2k kerberos ldap lock maildir mmx mng modules mudflap multilib ncurses nls nptl nsplugin opencl opengl openldap openmp pam pcre perl png policykit python qt qt3support qt4 readline sasl session spell sse sse2 sse3 ssl startup-notification svg tcpd thunar tiff truetype udev unicode webkit zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="krita" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en de" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-4" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 Rick Farina (Zero_Chaos) gentoo-dev 2013-07-25 15:26:45 UTC
Does it work when you build jtr yourself?
Comment 2 Steffen 2013-07-28 09:03:59 UTC
If I download johntheripper from
http://www.openwall.com/john/g/john-1.7.9-jumbo-7.tar.gz
and remove the comment marks in the makefile
CC = mpicc -DHAVE_MPI
MPIOBJ = john-mpi.o
it works as expected.

mpirun -np 3 ./john --test --format=raw-md5
Benchmarking: Raw MD5 [128/128 AVX intrinsics 12x]... (3xMPI) DONE
Raw:	91419K c/s real, 91419K c/s virtual
Comment 3 Steffen 2017-07-17 12:23:31 UTC
Not that I expect a fix for this problem anytime soon, but it is still there. Obviously the bug is somewhere in the ebuild, as the manual compilation works.

mpiexec --allow-run-as-root -np 3 john --test --format=raw-md5
Benchmarking: Raw MD5 [128/128 AVX intrinsics 12x]... Benchmarking: Raw MD5 [128/128 AVX intrinsics 12x]... Benchmarking: Raw MD5 [128/128 AVX intrinsics 12x]... DONE
Raw:	13274K c/s real, 13274K c/s virtual

DONE
Raw:	13199K c/s real, 13333K c/s virtual

DONE
Raw:	13139K c/s real, 13139K c/s virtual


emerge -pv johntheripper

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] app-crypt/johntheripper-1.7.9-r11::gentoo  USE="mpi openmp -cuda -custom-cflags -libressl -minimal -mozilla -opencl" CPU_FLAGS_X86="sse2 (-mmx)" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

emerge --info
Portage 2.3.6 (python 2.7.13-final-0, default/linux/amd64/13.0, gcc-5.4.0, glibc-2.24-r3, 4.9.37-gentoo x86_64)
=================================================================
System uname: Linux-4.9.37-gentoo-x86_64-Intel-R-_Core-TM-_i5-2500K_CPU_@_3.30GHz-with-gentoo-2.4.1
KiB Mem:    16402092 total,  12693152 free
KiB Swap:   33554428 total,  33554428 free
Timestamp of repository gentoo: Fri, 14 Jul 2017 07:15:01 +0000
sh bash 4.4_p12
ld GNU ld (Gentoo 2.28 p1.2) 2.28
app-shells/bash:          4.4_p12::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.24.1-r2::gentoo
dev-lang/python:          2.7.13::gentoo, 3.4.6::gentoo, 3.5.3::gentoo, 3.6.1-r1::gentoo
dev-util/cmake:           3.8.2::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1::gentoo
sys-apps/openrc:          0.28::gentoo
sys-apps/sandbox:         2.10-r4::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r3::gentoo
sys-devel/automake:       1.11.6-r2::gentoo, 1.13.4-r1::gentoo, 1.15.1::gentoo
sys-devel/binutils:       2.28-r2::gentoo
sys-devel/gcc:            5.4.0-r3::gentoo, 6.3.0::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r4::gentoo
sys-devel/make:           4.2.1-r1::gentoo
sys-kernel/linux-headers: 4.10::gentoo (virtual/os-headers)
sys-libs/glibc:           2.24-r3::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.europe.gentoo.org/gentoo-portage
    priority: -1000

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

bobwya
    location: /var/lib/layman/bobwya
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA PUEL AdobeFlash-11.x Oracle-BCLA-JavaSE dlj-1.1"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/"
LANG="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X acl aes alsa amd64 apache2 aspell avx berkdb bluetooth bzip2 cairo cleartype cli consolekit corefonts cracklib crypt cups curl cxx dbus dri emacs ffmpeg fftw fortran gd gdbm gif gtk gtk2 gtk3 iconv imap ipv6 jpeg jpeg2k kerberos ldap lock maildir mmx mmxext mng modules multilib ncurses nls nptl nsplugin nsscache nvidia opencl opengl openldap openmp pam pcre perl png policykit popcnt python qt qt3support qt4 readline sasl seccomp session spell sse sse2 sse3 sse4_1 sse4_2 ssl ssse3 startup-notification svg tcpd thunar tiff truetype type1 udev unicode webkit xattr xvmc zlib" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="krita" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" L10N="de en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21 ruby22" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON