Bugzilla Bug 197801

If test is set in /etc/make.conf arpack-96-r1 test programs execution fails.

Reproducible: Always

Steps to Reproduce:
1.Enable the "test" function of portage
2.emerge -1 arpack
3.

Actual Results:  
After succesfully building the test programs against arpack the execution
hangs. I have waited several minutes with the CPU at 100%, but nothing happens
(no output on the stdout nor nothing). It look like that the computer has
entered a never ending loop inside one of the test routines:

>>> Source compiled.
gfortran   -c -o sssimp.o sssimp.f
gfortran   -c -o dssimp.o dssimp.f
gfortran   -c -o snsimp.o snsimp.f
gfortran   -c -o dnsimp.o dnsimp.f
gfortran   -c -o cnsimp.o cnsimp.f
gfortran   -c -o znsimp.o znsimp.f
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs
-L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/LAPACK/.libs sssimp.o
-larpack -llapack_arpack -lblas -o sssimp
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs
-L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/LAPACK/.libs dssimp.o
-larpack -llapack_arpack -lblas -o dssimp
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs
-L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/LAPACK/.libs snsimp.o
-larpack -llapack_arpack -lblas -o snsimp
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs
-L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/LAPACK/.libs dnsimp.o
-larpack -llapack_arpack -lblas -o dnsimp
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs
-L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/LAPACK/.libs cnsimp.o
-larpack -llapack_arpack -lblas -o cnsimp 
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs
-L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/LAPACK/.libs znsimp.o
-larpack -llapack_arpack -lblas -o znsimp  


Exiting on signal 2

(I have waited a lot of time before killing)

Expected Results:  
Test programs should be executed correctly...

If I build the test programs against LAPACK everithing works and the test phase
is carried out succesfully in a few seconds. I have modified the ebuild this
way:

I changed this:

    sed -i \
        -e '/^include/d' \
        -e "s/\$(ALIBS)/-larpack ${BLAS_LIBS}/g" \
        -e 's/$(FC)/$(F77)/g' \
        -e 's/$(FFLAGS)/$(FFLAGS) $(LDFLAGS)/g' \
        EXAMPLES/*/makefile || die "sed failed"

into this:

    sed -i \
        -e '/^include/d' \
        -e "s/\$(ALIBS)/-llapack -larpack ${BLAS_LIBS}/g" \
        -e 's/$(FC)/$(F77)/g' \
        -e 's/$(FFLAGS)/$(FFLAGS) $(LDFLAGS)/g' \
        EXAMPLES/*/makefile || die "sed failed"

(please notice that lapack is before arpack, otherwise it does not work). This
way everything works, but here I am using the lapack-atlas package which is not
a dependency of arpack. I have tried using the temporary lapack_arpack library
which is build by the package but this was useless...

As for my environment: 

davide@gentoo ~ $ emerge --info
Portage 2.1.3.9 (default-linux/x86/2007.0/desktop, gcc-4.1.2, glibc-2.6.1-r0,
2.6.22-gentoo-r8 i686)
=================================================================
System uname: 2.6.22-gentoo-r8 i686 Genuine Intel(R) CPU T2400 @ 1.83GHz
Timestamp of tree: Fri, 26 Oct 2007 19:04:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p17
dev-lang/python:     2.4.4-r6
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 1.12.9-r2
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.22-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=prescott -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /etc/init.d/ipw3945d /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/init.d /etc/revdep-rebuild
/etc/splash /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -march=prescott -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks metadata-transfer parallel-fetch sandbox sfperms
strict test unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://mirror.muntinternet.net/pub/gentoo/
ftp://mirror.switch.ch/mirror/gentoo
http://www.die.unipd.it/pub/Linux/distributions/gentoo-sources/
ftp://ftp.unina.it/pub/linux/distributions/gentoo
http://mirror.ing.unibo.it/gentoo/ ftp://mirror.ing.unibo.it/gentoo/ "
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages
--filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/sunrise /usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X acl acpi alsa berkdb bitmap-fonts blas cairo cdr cli cracklib crypt cups
dbus dri dvd dvdr dvdread eds emboss encode esd evo fam ffmpeg firefox fortran
ftp gb gdbm gif glitz gphoto2 gpm gstreamer gtk hal iconv imagemagick ipv6
isdnlog jabber javascript jpeg kerberos lapack ldap mad midi mikmod mime mmx
mozilla mp3 mpeg msn mudflap mysql ncurses nls nptl nptlonly ogg opengl openmp
oss pam pcre pdf perl png pppd python quicktime readline reflection samba sdl
session spell spl sse sse2 ssl svg tcpd test tetex theora tiff truetype
truetype-fonts type1-fonts unicode usb vim-syntax vorbis win32codecs x86
xcomposite xml xorg xscreensaver xv xvid zlib" ALSA_CARDS="ali5451 als4000
atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938
es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx
via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop
empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi
null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard
mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk
hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="nvidia
nv"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS,
LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

I can't reproduce it on my x86 and amd64. Also, some users have reported bugs
using external lapack, which is what changed in the -r1.
It could be that you are using blas-atlas which needs atlas. I commited a small
fix that allows arpack to use any blas. Please resync later on and try again.

Created an attachment (id=135013) [details]
build.log

Thanks you for the quick help.

It does not build the examples any more. I get an error when emerge is running
the autotools:

 * You need one of these Fortran Compilers: gfortran ifc g77
 * Installed are:  gfortran
>>> Unpacking source...
>>> Unpacking arpack96.tar.gz to /var/tmp/portage/sci-libs/arpack-96-r2/work
>>> Unpacking patch.tar.gz to /var/tmp/portage/sci-libs/arpack-96-r2/work
>>> Unpacking parpack96.tar.gz to /var/tmp/portage/sci-libs/arpack-96-r2/work
>>> Unpacking ppatch.tar.gz to /var/tmp/portage/sci-libs/arpack-96-r2/work
 * Applying arpack-autotools.patch ...                                    [ ok
]
 * Applying arpack-arscnd.patch ...                                       [ ok
]
Package blas was not found in the pkg-config search path.
Perhaps you should add the directory containing `blas.pc'
to the PKG_CONFIG_PATH environment variable
No package 'blas' found
 * Running eautoreconf in '/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK'
...
 * Running aclocal ...                                                    [ ok
]
 * Running libtoolize --copy --force --automake ...                       [ ok
]
 * Running aclocal ...                                                    [ ok
]
 * Running autoconf ...                                                   [ ok
]
 * Running automake --add-missing --copy --foreign ...                    [ ok
]
 * Running elibtoolize in: ARPACK
 *   Applying sed-1.5.6.patch ...
>>> Source unpacked.

And then while building the tests: (it seems that BLAS symbols are missing)

>>> Source compiled.
gfortran   -c -o sssimp.o sssimp.f
gfortran   -c -o dssimp.o dssimp.f
gfortran   -c -o snsimp.o snsimp.f
gfortran   -c -o dnsimp.o dnsimp.f
gfortran   -c -o cnsimp.o cnsimp.f
gfortran   -c -o znsimp.o znsimp.f
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs sssimp.o
-larpack  -o sssimp
gfortran  -L/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs dssimp.o
-larpack  -o dssimp
dssimp.o: In function `MAIN__':
dssimp.f:(.text+0x833): undefined reference to `daxpy_'
dssimp.f:(.text+0x856): undefined reference to `dnrm2_'
dssimp.o: In function `av_':
dssimp.f:(.text+0x11f8): undefined reference to `daxpy_'
dssimp.f:(.text+0x1290): undefined reference to `daxpy_'
dssimp.f:(.text+0x12d4): undefined reference to `daxpy_'
dssimp.f:(.text+0x136c): undefined reference to `daxpy_'
dssimp.f:(.text+0x13c5): undefined reference to `dscal_'
/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs/libarpack.so:
undefined reference to `cgemv_'
/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs/libarpack.so:
undefined reference to `cscal_'

[Many many more of them]

/var/tmp/portage/sci-libs/arpack-96-r2/work/ARPACK/.libs/libarpack.so:
undefined reference to `drot_'
collect2: ld returned 1 exit status
make: *** [sssimp] Error 1
make: *** [snsimp] Error 1
/usr/portage/sci-libs/arpack/arpack-96-r2.ebuild: line 71: ./sssimp: No such
file or directory

This is happening while using BLAS-ATLAS. (pkg-config seems to be unable to
find it).

If I use the reference BLAS implementation I get the same problem as before...
the computer hangs while running the first test program and I have to kill the
process...

I am attaching the build log.

If you want to use blas-atlas, you need to bump to the tobe stable 3.8.0.
Otherwise, stick to blas-reference.

Ok, with reference-BLAS it does build, but it hangs while running the tests. It
does work if I link with -llapack -larpack (lapack before arpack). I think that
this way the LAPACK subroutines from within Arpack are not used, and instead
the one from lapack-atlas are used. So it must be that some Lapack subroutine
within Arpack Lib is broken in my system... This is not really a big issue as I
can always link this way against the working lapack...  

David,

Could you provide the result of "eselect blas show" and "eselect lapack show"?

You should not need to link with the external lapack. Altough the documentation
is old, check http://www.caam.rice.edu/software/ARPACK/UG/node13.html

Sébastien

Thank you very much for the help. 

Here is the eselect output for blas:
davide@gentoo ~ $ eselect blas show
lib: reference

I don't have the lapack eselect module installed... I have:

davide@gentoo ~ $ lapack-config -p

Current profile:
F77 LAPACK: /usr/lib/lapack/f77-ATLAS



davide@gentoo ~ $ eix -Ic blas
[I] app-admin/eselect-blas (0.1@08/05/07): BLAS module for eselect
[I] app-admin/eselect-cblas (0.1@08/05/07): C-language BLAS module for eselect
[I] sci-libs/blas-atlas (3.7.11-r1@08/05/07): Automatically Tuned Linear
Algebra Software BLAS implementation
[I] sci-libs/blas-reference (20070226@11/01/07): Basic Linear Algebra
Subprograms F77 reference implementations
[I] sci-libs/cblas-reference (20030223-r4@10/26/07): C wrapper interface to the
F77 reference BLAS implementation
[I] virtual/blas (1.0@10/26/07): Virtual for FORTRAN 77 BLAS implementation
[I] virtual/cblas (1.0@10/26/07): Virtual for BLAS C implementation
Found 7 matches.
davide@gentoo ~ $ eix -Ic lapack
[I] sci-libs/lapack-atlas (3.7.11@08/05/07): F77 and C LAPACK implementations
using available ATLAS routines
[I] sci-libs/lapack-config (1.0.1@08/05/07): Utility to change the default
LAPACK library
Found 2 matches.

AFAIK I should be in sync with the stable release...

I have read the documentation and I knew that -lapack shouldn't be needed
before opening this bug, but this is the only way to make the examples work.
Actually I was trying to build freefem++ with arpack and found that the only
way to make it work was patching the arpack with the "rename second patch" and
linking against lapack in a way such that the lapack subset inside arpack is
overwritten. If I use the default lapack routines inside arpack nothing works
and every program goes in a never-ending loop as the first routine of arpack is
called...  

> [I] sci-libs/lapack-atlas (3.7.11@08/05/07): F77 and C LAPACK implementations
> using available ATLAS routines
> [I] sci-libs/lapack-config (1.0.1@08/05/07): Utility to change the default
> LAPACK library
> Found 2 matches.
> 
> AFAIK I should be in sync with the stable release...

Yes you are, but we are a bit behind with stabilization. You probably want
lapack-reference-3.1.1-r1. Check our updated guide at
http://www.gentoo.org/proj/en/science/blas-lapack.xml for more on lapack.

Anyway it should not matter which lapack you have installed if it's using the
arpack internal one.

> I have read the documentation and I knew that -lapack shouldn't be needed
> before opening this bug, but this is the only way to make the examples work.
> Actually I was trying to build freefem++ with arpack and found that the only
> way to make it work was patching the arpack with the "rename second patch" and
> linking against lapack in a way such that the lapack subset inside arpack is
> overwritten. If I use the default lapack routines inside arpack nothing works
> and every program goes in a never-ending loop as the first routine of arpack is
> called...  

This would be annoying if it's confirmed. Some users confirm the opposite: they
need the internal lapack, that's the main reason for the -r1 bump. I can't
reproduce it on my boxes, everything builds and tests fine. Anyone could? 
I will try freefem++ when I have more time.

Created an attachment (id=135483) [details]
Test case: building and testing freefem++ on eigenvalues problems

I have updated to atlas-3.8.0, but nothing changes (it builds also with atlas
now, but it goes to the neverending loop too).

The issue with freefem++ should be the same of the testing libs, so you should
not need to test it, but, if you want to test, it I am attaching an ebuild I
have been writing for freefem++ for the sunrise project (it's my first ebuild),
maybe it could save you some time... (I will not "release" this until I
understand how to fix this issue...)

Thanks to your build log, I think I spotted the problem. It seems it needs the
-pipe flags added to fortran flags FFLAGS. By the way, you probably want to set
your fortran flags in your /etc/make.conf, such as FFLAGS="${CFLAGS}".
Now I would like to know if there is any other issue with the internal lapack
libs. Could you set your flags as said and test your freefem++ lib?

Created an attachment (id=135515) [details]
Complete Build Log

I have modified my make.conf and added the "FFLAGS" variable, but nothing
really changed: it is still hanging. I have tried playing with the eselect blas
and lapack values, but it was useless... I am attaching the full build log. 

I found that building with -pipe only does it: everything works this way. But
if I build with -O (or -O{2,3}) the lapack "sublib" is not built well, but
appears to be broken. For example:

-O -pipe 

does not work while:

-march=prescott -pipe 

works, but I don't know what -march does without any optimization (almost
nothing I think).

I can't say that this really fixes the problem as building this library without
any optimization is not really good: eigenvalues problems can require a lot of
calculations... 

David,

I made a new autotools patch that doesn't optimize a few internal lapack
routines, just as we do with our lapack-reference. It worked on my x86 with any
FFLAGS I set. Please let me know if it works for you after a re-sync.

It works perfectly here now, even with -O3 FFLAGS, the test routine seems to be
executed correctly now.

> It works perfectly here now, even with -O3 FFLAGS, the test routine seems to be
> executed correctly now.


Thanks! Closing, then.