Bug 114722 - build failures triggered by packages with mixed time resolutions during build process (fast machines, ccache, tmpfs)
|
Bug#:
114722
(tmpfs)
|
Product: Gentoo Linux
|
Version: 2005.1
|
Platform: AMD64
|
|
OS/Version: Linux
|
Status: RESOLVED
|
Severity: normal
|
Priority: P2
|
|
Resolution: FIXED
|
Assigned To: dsd@gentoo.org
|
Reported By: herbs@gentoo.org
|
|
Component: Ebuilds
|
|
|
URL:
|
|
Summary: build failures triggered by packages with mixed time resolutions during build process (fast machines, ccache, tmpfs)
|
|
Keywords:
|
|
Status Whiteboard:
|
|
Opened: 2005-12-07 03:13 0000
|
To quote from the ebuild:
#we also need to force -fPIC throughout on amd64
if [ "${ARCH}" = "amd64" ] && ! is-flag -fPIC; then append-flags -fPIC; fi
I can't see any reason why this is neccessary, the shared libs already get
compiled with -fPIC. Why was this added?
Because it was failing to link without it..
I mentioned this in a few bugs (#79278, #87790) and here is the specific
failure message:
/usr/bin/ld -o libpari.so.2.1.7 -shared -soname libpari.so.1 -lc -lm kernel.o
mp.o alglin1.o alglin2.o arith1.o arith2.o base1.o base2.o base3.o base4.o
base5.o bibli1.o bibli2.o buch1.o buch2.o buch3.o buch4.o galconj.o gen1.o
gen2.o gen3.o ifactor1.o polarit1.o polarit2.o polarit3.o rootpol.o subgroup.o
trans1.o trans2.o trans3.o elliptic.o galois.o kummer.o mpqs.o nffactor.o
stark.o subfield.o thue.o anal.o compat.o errmsg.o es.o helpmsg.o init.o
sumiter.o mpinl.o
/usr/bin/ld: kernel.o: relocation R_X86_64_32S against `a local symbol' can not
be used when making a shared object; recompile with -fPIC
kernel.o: could not read symbols: Bad value
make: *** [libpari.so.2.1.7] Error 1
This is during the "building executables" phase.
Shows after disabling the -fPIC line for amd64 ;) (and takes just 2 minutes to
check).
George
Pari can be reasonably easliy compiled with only -fPIC on the shared libraries,
I will attach a patch to the ebuild that does it.
(In reply to comment #1)
> This is during the "building executables" phase.
> Shows after disabling the -fPIC line for amd64 ;) (and takes just 2 minutes to
> check).
Unable to reproduce this on either of my amd64 boxes with or without the
append-flags -fPIC line. Even if I were, adding an append-flags -fPIC is not an
acceptable solution, see:
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=3&chap=1
and
http://www.gentoo.org/proj/en/hardened/pic-internals.xml
for reasons why is gentoo policy to compile _only_ the shared libraries with -fPIC.
(In reply to comment #2)
> Pari can be reasonably easliy compiled with only -fPIC on the shared libraries,
My point is that we already have the shared libs compiled with -fPIC and all the
quoted line does is force compiling of everything with -fPIC on amd64.
Pari does compile some shared libraries, so some parts of it do need -fPIC,
hence the patch to make the libraries compile with fPIC
(In reply to comment #5)
> Pari does compile some shared libraries, so some parts of it do need -fPIC,
> hence the patch to make the libraries compile with fPIC
Yes it does, yes they need to be and yes the ebuild already does this without
your patch. In fact your patch has no effect on the build process as the
Makefile CFLAGS are overriden on the command line:
emake ${mymake} CFLAGS="${CFLAGS} -DGCC_INLINE -fPIC" lib-dyn || die "Building
shared library failed!"
Well, it does not compile on my laptop here, so there is a problem.
And the addition of that DLCFLAGS="-fPIC" does not do anything..
Herbie: what gcc/toolchain are your systems using? Below is my emerge info.
Do you by chance have LDFLAGS set to anything?
George
Portage 2.0.53 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.5-r3,
2.6.13-gentoo-r5 x86_64)
=================================================================
System uname: 2.6.13-gentoo-r5 x86_64 AMD Athlon(tm) 64 Processor 3200+
Gentoo Base System version 1.12.0_pre11
dev-lang/python: 2.3.5, 2.4.2
sys-apps/sandbox: 1.2.17
sys-devel/autoconf: 2.13, 2.59-r7
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils: 2.16.1-r1
sys-devel/libtool: 1.5.20-r1
virtual/os-headers: 2.6.11-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig collision-protect cvs digest distlocks noauto sandbox
sfperms strict userpriv usersandbox"
GENTOO_MIRRORS="ftp://mirror.switch.ch/mirror/gentoo/
http://mirror.datapipe.net/gentoo http://pandemonium.tiscali.de/pub/gentoo/"
LANG="ru_RU.UTF-8"
LC_ALL="C"
LINGUAS="en de fr ru uk"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="amd64 16bit X a52 aac aalib acl activefilter ada aim alsa amd apache2 arts
async audiofile avi bash-completion berkdb bigger-fonts bitmap-fonts bluetooth
bmp bootsplash bzip2 calendar caps cdparanoia cdr cdrom chroot crypt cscope css
cups curl dga dhcp directfb dmx dpms dts dvb dvd dvdr dvdread dxr3 edl eds
emboss encode escreen esd exif expat fam fame fbcon festival ffmpeg fftw flac
foomaticdb fortran ftp gd gd-external gdbm ggi gif gimp gimpprint ginac glut
gmail gmp gnome gnuplot gphoto2 gpm graphviz gs gsl gstreamer gtk gtk2 hal hdf
hdf5 icq idn ieee1394 imagemagick imlib insecure-drivers ipv6 irc java
javascript jpeg jpeg2k kcal kde kdeenablefinal kdepim latex lcms libcaca libwww
lm_sensors logitech-mouse lzo lzw lzw-tiff mad mhash mikmod mime mjpeg mng
motif mp3 mpeg mpeg4 mplayer msn musicbrainz mysql mysqli ncurses nls nptl
nptlonly nsplugin numeric nvidia oav octave ofx ogg oggvorbis openal opengl pam
pcmcia pcre pda pdf pdflib perl php physfs pic plotutils png pnp povray ppds
python qt quicktime quotas readline recode rpc rss samba scanner sdl serial
slang sms sndfile sockets sox speex spell sqlite ssl subversion svg svgz szip
tcltk tcpd tetex tga theora threads tidy tiff transcode truetype truetype-fonts
type1 type1-fonts udev unicode usb userlocales utf8 uudeview v4l v4l2 vcd vim
vim-pager vnc vorbis wifi winbind wmf xanim xine xinerama xml xml2 xmms xosd
xpm xprint xscreensaver xsl xv xvid yahoo yv12 zlib linguas_en linguas_de
linguas_fr linguas_ru linguas_uk userland_GNU kernel_linux elibc_glibc"
Unset: ASFLAGS, CTARGET, LDFLAGS
Oh, on those references:
I did read both (quite some time ago actually, that short blurb in ebuild
policy was added rather recently I must say), and the second specifically
states (just looked it up again):
Only the AMD64 seems to support some kind of "emulation" mode where it does not
seem to make a difference if PIC or normal addressing is used for referring
code functions and data to access.
Which relates to some discussions (probably a year or more ago) on -dev mailing
list about how it was a good idea to use -fPIC on amd64.. So, is it really that
bad to force -fPIC on this arch?
Well, of course if we can fix this compilation issue, we better fix it :).
George
No, nothing out of the ordinary in my toolchain, see emerge info below. What's
confusing me is that looking at your error message the build is dying during the
linking of the shared lib:
"make: *** [libpari.so.2.1.7] Error 1".
From looking at the makefile this should take place during the "make lib-dyn"
stage where we have -fPIC specifically added and indeed does appear to be what
happens here. But you say you get this error during building executables phase
so I can only assume that for some reason "make lib-dyn" is failing to build the
shared library on your system.
Portage 2.0.53 (default-linux/amd64/2006.0/no-symlinks/no-lib32, gcc-3.4.4,
glibc-2.3.5-r3, 2.6.14-gentoo x86_64)
=================================================================
System uname: 2.6.14-gentoo x86_64 AMD Athlon(tm) 64 Processor 3200+
Gentoo Base System version 1.12.0_pre11
ccache version 2.4 [enabled]
dev-lang/python: 2.4.2
sys-apps/sandbox: 1.2.17
sys-devel/autoconf: 2.13, 2.59-r7
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils: 2.16.1-r1
sys-devel/libtool: 1.5.20-r1
virtual/os-headers: 2.6.11-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.5/env
/usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config
/usr/lib64/mozilla/defaults/pref /usr/share/X11/xkb /usr/share/config
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/init.d /etc/pam.d /etc/splash /etc/terminfo
/etc/texmf/web2c /etc/env.d"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/mnt/nfs/home/portage/distfiles"
FEATURES="autoconfig ccache cvs distlocks multilib-strict sandbox sfperms sign"
GENTOO_MIRRORS="ftp://194.117.143.69/mirrors/gentoo
ftp://212.219.56.146/sites/www.ibiblio.org/gentoo/"
PKGDIR="/home/herbie/Gentoo/packages"
PORTAGE_TMPDIR="/home/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/home/herbie/Gentoo/misc-overlays/main"
SYNC="rsync://trantor/gentoo-portage"
USE="amd64 S3TC X a52 aac aalib acpi alsa apache2 asf audiofile avi
bash-completion berkdb bitmap-fonts browserplugin bzip2 cdr crypt cups curl dbus
dga divx4linux dri dvd dvdr dvdread dvi eds emboss encode escreen ethereal etwin
evo exif expat faad fam fbcon fbsplash ffmpeg firefox flac flash foomaticdb
fortran gdbm gif gimpprint glitz glut glx gmp gnome gphoto2 gpm gstreamer gtk
gtk2 gtkhtml hal idn imagemagick imap imlib ipv6 ithreads java javascript jpeg
latex lcms ldap libwww logrotate lzw lzw-tiff mad maildir matroska mikmod mjpeg
mng mono motif mozilla mp3 mpeg mysql ncurses nls nptl nptlonly nsplugin nvidia
offensive ogg oggvorbis openal opengl pam pcre pdf pdflib perl plotutils png
ppds python quicktime readline reiserfs ruby samba sdl slang sox speex spell
sqlite ssl svg tcltk tcpd tetex theora threads tiff truetype truetype-fonts
type1-fonts udev unicode usb userlocales utf8 vorbis xface xinerama xml2 xmms
xpm xv xvid xvmc zlib video_cards_nvidia userland_GNU kernel_linux elibc_glibc"
Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS
Pari-2.1.7 compiles for me w/o the mentioned -fPIC line. I'm all for
removing it, as it's violating at least 2 ebuild policies (no general -fPIC
addition, no conditional -fPIC changes).
George, please recheck that this line is necessary on your system. If not, tell me
to remove it or remove it yourself please.
Yes it is, please see posting #1, this I checked right away (again), it really
fails here without that -fPIC and indeed it happens in the "Building
executables..." phase, as the shared lib gets built just fine before that (with
-fPIC obviously) and I see that einfo message some lines above the actual
error..
Herbie: thanks for posting your specs, I'll try to see if there are any
differences between our systems, although on a quick glance they seem to be
pretty similar..
Danny: can you please post your emerge info here as well? Its quite puzzling
indeed that it fails here on my system..
George
Here you are:
dvandyk@phi video $ emerge info
Portage 2.0.53_rc7 (default-linux/amd64/2005.1, gcc-3.4.3-20050110,
glibc-2.3.5-r2, 2.6.14-gentoo-r2 x86_64)
=================================================================
System uname: 2.6.14-gentoo-r2 x86_64 AMD Athlon(tm) 64 Processor 3200+
Gentoo Base System version 1.12.0_pre9
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
dev-lang/python: 2.2.3-r5, 2.3.5, 2.4.2
sys-apps/sandbox: 1.2.13
sys-devel/autoconf: 2.13, 2.59-r7
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils: 2.15.92.0.2-r10, 2.16-r1, 2.16.1, 2.16.91.0.4
sys-devel/libtool: 1.5.20
virtual/os-headers: 2.6.11-r2
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=k8 -O2 -pipe -ftracer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env
/usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3.4/env
/usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d"
CXXFLAGS="-march=k8 -O2 -pipe -ftracer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig buildsyspkg cvs distlocks sandbox sfperms sign strict"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/
ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/"
LINGUAS="de"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://sigma/gentoo-portage"
USE="amd64 X aalib acl adns alsa arts artswrappersuid audiofile avi
bash-completition berkdb bitmap-fonts blas bonobo bzip2 cdb cdr client-only
crypt cups curl directfb dvb dvd dvdr eds emboss encode esd exif expat f77 fam
ffmpeg flac foomaticdb fortran freetype gd gdbm geoip ggi gif glut gmp gphoto2
gpm gtk gtk2 gtkhtml guile idn imagemagick imlib insecure-drivers ipv6 jack jpeg
kde lcms ldap libcaca libwww lua lzw lzw-tiff mad mhash mikmod mjpeg mng
moznocompose moznoirc mp3 mpeg multislot nas ncurses nls nptl ogg openal opengl
pam pcre pdflib perl plotutils png ppds python qt qt-mt quicktime readline
recode ruby scanner sdl slang spell ssl svg tcltk tcpd tetex tiff truetype
truetype-fonts type1 type1-fonts udev usb userlocales vorbis wmf xine xml xml2
xmms xosd xpm xprint xv xvid zlib linguas_de userland_GNU kernel_linux elibc_glibc"
Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, MAKEOPTS
Oh, just thought of one thing:
I have been porting dev-lang/gnat (the Ada compiler) to amd64 and I have
problems with shared libs. For some reason I could not get that -fPIC passed
down to the build system, even with extensive seding of Makefile's. I wonder if
this might be related.. I guess should be some toolchain difference than.
In any case I would appreciate if somebody would test dev-lang/gnat-3.44. Right
now it is package-masked (this particular version) and has shared lib build
commented out. To test please unmask the package and, in the ebuild, uncomment
the
# MAKEOPTS=-j1 emake -C gcc gnatlib-shared LIBRARY_VERSION=3.4 || die
"gnatlib-shared failed"
line at the very end of src_compile..
This should tell me at least whether there is something weird with my setup
(although I never had any other trouble with -fPIC or no -fPIC..)
George
Ok, so I did a comparison and I thought I was onto something:
There were only two differences that could be at all relevant, these are
multilib-strict in Herbie's FEATURES (just what is that? I don't see it
documented anywhere inside /etc) and pic in my USE. Only the last one (pic in
USE) was a common difference and by description I thought it might be
involved..
The thing is, this flag is used only by a handfull of packages, not by pari
itself. However glibc uses it, so it might have been that just rebuilding glibc
without it would help, at least that was an idea (and sorry, it took some time,
but I had to proceed carefully, its a glibc after all..). Well, it did not,
still get this linking failure after changing glibc and rebooting..
Right now I am rebuilding system against new glibc (well, so much for that hope
to avoid rebuilding the rest of the stuff..), will see if it helps when it
finishes. However I suspect that chances are slim, as other stuff that used pic
was pretty much only gzip of what I have installed; gcc *does not* care about
this flag..
So, considering this outcome I would really appreciate if somebody for whom
pari compiles without that append_flag line would test dev-lang/gnat-3.44 ..
George
Just to add that sci-mathematics/pari-2.1.7 compiles fine here after commenting
out the fPIC lines in the ebuild. On the other hand dev-lang/gnat-3.44 fails
here with the following error,
make[4]: Leaving directory
`/mnt/gentoo/var/tmp/portage/portage/gnat-3.44/work/build/gcc/ada'
rm -f rts/libgnat.so rts/libgnarl.so
cd rts; ../../xgcc -B../../ -shared -fPIC \
-o libgnat-3.4.so \
a-caldel.o a-calend.o a-chahan.o a-charac.o a-chlat1.o a-chlat9.o
a-colien.o a-colire.o a-comlin.o a-cwila1.o a-cwila9.o a-decima.o a-diocst.o
a-direio.o a-einuoc.o a-elchha.o a-except.o a-exctra.o a-filico.o a-finali.o
a-flteio.o a-fwteio.o a-inteio.o a-ioexce.o a-iwteio.o a-lfteio.o a-lfwtio.o
a-liteio.o a-liwtio.o a-llftio.o a-llfwti.o a-llitio.o a-lliwti.o a-ncelfu.o
a-ngcefu.o a-ngcoty.o a-ngelfu.o a-nlcefu.o a-nlcoty.o a-nlelfu.o a-nllcef.o
a-nllcty.o a-nllefu.o a-nscefu.o a-nscoty.o a-nselfu.o a-nucoty.o a-nudira.o
a-nuelfu.o a-nuflra.o a-numaux.o a-numeri.o a-sequio.o a-sfteio.o a-sfwtio.o
a-siocst.o a-siteio.o a-siwtio.o a-ssicst.o a-ssitio.o a-ssiwti.o a-stmaco.o
a-storio.o a-strbou.o a-stream.o a-strfix.o a-string.o a-strmap.o a-strsea.o
a-strsup.o a-strunb.o a-ststio.o a-stunau.o a-stwibo.o a-stwifi.o a-stwima.o
a-stwise.o a-stwisu.o a-stwiun.o a-suteio.o a-swuwti.o a-swmwco.o a-tags.o
a-teioed.o a-textio.o a-ticoau.o a-ticoio.o a-tideau.o a-tideio.o a-tienau.o
a-tienio.o a-tifiio.o a-tiflau.o a-tiflio.o a-tigeau.o a-tiinau.o a-tiinio.o
a-timoau.o a-timoio.o a-tiocst.o a-titest.o a-unccon.o a-uncdea.o a-witeio.o
a-wtcoau.o a-wtcoio.o a-wtcstr.o a-wtdeau.o a-wtdeio.o a-wtedit.o a-wtenau.o
a-wtenio.o a-wtfiio.o a-wtflau.o a-wtflio.o a-wtgeau.o a-wtinau.o a-wtinio.o
a-wtmoau.o a-wtmoio.o a-wttest.o ada.o calendar.o g-arrspl.o g-awk.o g-bubsor.o
g-busora.o g-busorg.o g-calend.o g-casuti.o g-catiio.o g-cgi.o g-cgicoo.o
g-cgideb.o g-comlin.o g-comver.o g-crc32.o g-ctrl_c.o g-curexc.o g-debuti.o
g-debpoo.o g-diopit.o g-dirope.o g-dyntab.o g-except.o g-excact.o g-exctra.o
g-expect.o g-flocon.o g-heasor.o g-hesora.o g-hesorg.o g-htable.o g-io.o
g-io_aux.o g-locfil.o g-md5.o g-memdum.o g-moreex.o g-os_lib.o g-perhas.o
g-pehage.o g-regexp.o g-regpat.o g-sestin.o g-soccon.o g-socket.o g-socthi.o
g-soliop.o g-souinf.o g-speche.o g-spipat.o g-spitbo.o g-sptabo.o g-sptain.o
g-sptavs.o g-string.o g-strspl.o g-table.o g-tasloc.o g-traceb.o g-wistsp.o
gnat.o i-c.o i-cexten.o i-cobol.o i-cpoint.o i-cpp.o i-cstrea.o i-cstrin.o
i-fortra.o i-pacdec.o interfac.o ioexcept.o machcode.o s-addima.o s-arit64.o
s-assert.o s-atacco.o s-auxdec.o s-bitops.o s-boarop.o s-carsi8.o s-carun8.o
s-casi16.o s-casi32.o s-casi64.o s-casuti.o s-caun16.o s-caun32.o s-caun64.o
s-chepoo.o s-crtl.o s-crc32.o s-direio.o s-errrep.o s-except.o s-exctab.o
s-exnint.o s-exnllf.o s-exnlli.o s-expint.o s-explli.o s-expllu.o s-expmod.o
s-expuns.o s-fatflt.o s-fatgen.o s-fatlfl.o s-fatllf.o s-fatsfl.o s-ficobl.o
s-fileio.o s-finimp.o s-finroo.o s-fore.o s-geveop.o s-htable.o s-imgbiu.o
s-imgboo.o s-imgcha.o s-imgdec.o s-imgenu.o s-imgint.o s-imgllb.o s-imglld.o
s-imglli.o s-imgllu.o s-imgllw.o s-imgrea.o s-imguns.o s-imgwch.o s-imgwiu.o
s-io.o s-gloloc.o s-maccod.o s-mantis.o s-mastop.o s-osprim.o s-pack03.o
s-pack05.o s-pack06.o s-pack07.o s-pack09.o s-pack10.o s-pack11.o s-pack12.o
s-pack13.o s-pack14.o s-pack15.o s-pack17.o s-pack18.o s-pack19.o s-pack20.o
s-pack21.o s-pack22.o s-pack23.o s-pack24.o s-pack25.o s-pack26.o s-pack27.o
s-pack28.o s-pack29.o s-pack30.o s-pack31.o s-pack33.o s-pack34.o s-pack35.o
s-pack36.o s-pack37.o s-pack38.o s-pack39.o s-pack40.o s-pack41.o s-pack42.o
s-pack43.o s-pack44.o s-pack45.o s-pack46.o s-pack47.o s-pack48.o s-pack49.o
s-pack50.o s-pack51.o s-pack52.o s-pack53.o s-pack54.o s-pack55.o s-pack56.o
s-pack57.o s-pack58.o s-pack59.o s-pack60.o s-pack61.o s-pack62.o s-pack63.o
s-parame.o s-parint.o s-pooglo.o s-pooloc.o s-poosiz.o s-powtab.o s-purexc.o
s-rident.o s-rpc.o s-scaval.o s-secsta.o s-sequio.o s-shasto.o s-sopco3.o
s-sopco4.o s-sopco5.o s-stache.o s-stalib.o s-stoele.o s-stopoo.o s-stratt.o
s-strops.o s-soflin.o s-memory.o s-memcop.o s-traceb.o s-traces.o s-traent.o
s-unstyp.o s-vaflop.o s-valboo.o s-valcha.o s-valdec.o s-valenu.o s-valint.o
s-vallld.o s-vallli.o s-valllu.o s-valrea.o s-valuns.o s-valuti.o s-valwch.o
s-veboop.o s-vector.o s-vercon.o s-vmexta.o s-wchcnv.o s-wchcon.o s-wchjis.o
s-wchstw.o s-wchwts.o s-widboo.o s-widcha.o s-widenu.o s-widlli.o s-widllu.o
s-widwch.o s-wwdcha.o s-wwdenu.o s-wwdwch.o system.o text_io.o adaint.o argv.o
cio.o cstreams.o ctrl_c.o errno.o exit.o raise.o sysdep.o aux-io.o init.o cal.o
final.o tracebak.o expect.o mkdir.o socket.o \
-Wl,-soname,libgnat-3.4.so -lm
/usr/bin/ld: a-caldel.o: relocation R_X86_64_32 against `a local symbol' can
not be used when making a shared object; recompile with -fPIC
a-caldel.o: could not read symbols: Bad value
For your information here is my emerge info on that system,
cryos ~ # emerge info
Portage 2.0.53 (default-linux/amd64/2005.1, gcc-3.4.5, glibc-2.3.6-r1,
2.6.14-gentoo-r3 x86_64)
=================================================================
System uname: 2.6.14-gentoo-r3 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor
3800+
Gentoo Base System version 1.12.0_pre11
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632)
[disabled]
ccache version 2.3 [disabled]
dev-lang/python: 2.3.5-r2, 2.4.2
sys-apps/sandbox: 1.2.17
sys-devel/autoconf: 2.13, 2.59-r7
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils: 2.15.92.0.2-r2, 2.16.1-r1
sys-devel/libtool: 1.5.20-r1
virtual/os-headers: 2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=k8 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.2/share/config
/usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown
/usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown
/usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown
/usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/bind
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d"
CXXFLAGS="-march=k8 -O2 -pipe"
DISTDIR="/mnt/gentoo/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig buildpkg collision-protect cvs digest distlocks
multilib-strict sandbox sfperms sign"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LINGUAS="en_GB"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/mnt/gentoo/var/tmp/portage"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /usr/local/sci"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 X aac aalib aim alsa apache2 arts audiofile avi bash-completion
berkdb bitmap-fonts blas bluetooth bonobo bootsplash bzip2 bzlib cdparanoia cdr
crypt cscope cups curl dbus directfb divx4linux dlloader doc dvd dvdr dvdread
eds emboss encode esd ethereal evo exif expat fam fbcon ffmpeg fftw flac flash
foomaticdb fortran gb gd gdbm ggi gif gimpprint ginac glut gmp gnome gphoto2
gpm graphviz gstreamer gtk gtk2 gtkhtml guile hal icq idn imagemagick imap
imlib innodb ipv6 jabber java jikes joystick jpeg jpeg2k junit kde kerberos
lcms ldap libedit libg++ libwww lm_sensors lua lzw lzw-tiff mad mcal mhash ming
mng motif mp3 mpeg mpi msn mysql ncurses netcdf nls nptl nvidia octave odbc
offensive ogg oggvorbis openal openexr opengl oscar pam pcre pdflib perl
plotutils png postgres povray ppds python qt quicktime readline recode rtc ruby
samba sasl scanner sdl slp snmp spell sqlite ssl svg tcltk tcpd tetex theora
tiff truetype truetype-fonts type1-fonts udev unicode usb vhosts videos vorbis
wmf wxwindows xine xinerama xml xml2 xmms xpm xscreensaver xv xvid yahoo
zeroconf zlib linguas_en_GB userland_GNU kernel_linux elibc_glibc"
Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS
I found a reason for this build failure, and it is .. whoa!
Basically, this error triggers when PORTAGE_TMPDIR is mounted on tmpfs (i.e. in
memory), which is what I normally have setup, but not when it is on a
legitimate disk (which I sometimes do, when I need to work on a large package,
this time it was gnat-gcc-4.1). I am not sure whether this is due to filesystem
differences (reiserfs vs tmpfs) or something else, but in any case this just
should not happen..
So, this is not an issue with pari, but rather a bug in Linux filesystem code
or some such. Since the real issue looks way more important than what it was
originally, I am going to reassign it to our kernel folks, instead of closing
this bug. If some other action should be taken instead, please advise.
Greg: because you are the only one I know who is involved with this for sure I
am CC'ing you just in case. If you are on kernel alias please remove yourself..
My emerge info can be found in comment #7. Principal changes since then:
Portage 2.1_pre6-r5 (default-linux/amd64/2006.0, gcc-3.4.5, glibc-2.4-r0,
2.6.15-gentoo-r7 x86_64)
=================================================================
System uname: 2.6.15-gentoo-r7 x86_64 AMD Athlon(tm) 64 Processor 3200+
Gentoo Base System version 1.12.0_pre16
dev-lang/python: 2.4.2-r1
sys-apps/sandbox: 1.2.17
sys-devel/autoconf: 2.13, 2.59-r7
sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils: 2.16.1-r2
sys-devel/libtool: 1.5.22
virtual/os-headers: 2.6.11-r3
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O3 -pipe -ftracer -fomit-frame-pointer"
So this was universal for kernels from 2.6.13 to 2.6.15..
Just to add to this: this is the first time it happens so clearly. I
occasionally had -fPIC issues (albeit quite rarely, just few more times), but
they could all be resolved or attributed to known bugs before..
This only happens when I have tmpfs mounted on /var/tmp/portage (/dev/shm is
there and is a separate mount), but not when it is not mounted, i.e. it is just
a subdir of /var that is a normal partition, reiserfs; or not when I mount
--bind /some/real/dir /var/tmp/portage (over existing tmpfs mount). So it does
not look like it is mount related..
Marcus:
Thanks for testing gnat! This is indeed a completely different issue (as you
can see), which (for gnat) has been resolved long since.
George
Changed the subject to reflect new findings on the bug.
Also, in case anybody got two copies, sorry for double-posting - bugzilla went
unresponsive again when I was submitting :(.
George
Could someone summarize whats going on here? Is it that pari decides to use
different compiler flags when being compiled on tmpfs?
(In reply to comment #18)
> Could someone summarize whats going on here? Is it that pari decides to use
> different compiler flags when being compiled on tmpfs?
No, from what I can see in the build messages, the flags (passed to gcc) are
the same. The issue is that when built on tmpfs pari fails with the mentioned
complaint, while it builds successfully on the "real" filesystem..
Since the error itself was posted to other bugs (referenced in comment #2) but
not here I am posting the tail of failing build.
George
Principal parts of output:
...
x86_64-pc-linux-gnu-gcc -c -march=athlon64 -O3 -pipe -ftracer
-fomit-frame-pointer -DGCC_INLINE -fPIC -I. -I../src/headers -o sumiter.o
../src/language/sumiter.c
x86_64-pc-linux-gnu-gcc -c -march=athlon64 -O3 -pipe -ftracer
-fomit-frame-pointer -DGCC_INLINE -fPIC -I. -I../src/headers -o mpinl.o
../src/kernel/none/level1.c
GS> This is where it builds shared lib. Note the -fPIC flag
rm -f libpari.so.2.1.7
/usr/bin/ld -o libpari.so.2.1.7 -shared -soname libpari.so.1 -lc -lm kernel.o
mp.o alglin1.o alglin2.o arith1.o arith2.o base1.o base2.o base3.o base4.o
base5.o bibli1.o bibli2.o buch1.o buch2.o buch3.o buch4.o galconj.o gen1.o
gen2.o gen3.o ifactor1.o polarit1.o polarit2.o polarit3.o rootpol.o subgroup.o
trans1.o trans2.o trans3.o elliptic.o galois.o kummer.o mpqs.o nffactor.o
stark.o subfield.o thue.o anal.o compat.o errmsg.o es.o helpmsg.o init.o
sumiter.o mpinl.o
rm -f libpari.so.1
rm -f libpari.so
ln -s libpari.so.2.1.7 libpari.so.1
ln -s libpari.so.2.1.7 libpari.so
* Building executables...
x86_64-pc-linux-gnu-gcc -c -march=athlon64 -O3 -pipe -ftracer
-fomit-frame-pointer -DGCC_INLINE -I. -I../src/headers -I../src/language
-I/usr/include -o gp.o ../src/gp/gp.c
GS>many similar lines skipped. Note the absence of -fPIC. This is identical for
GS>build on both tmpfs and reiserfs.
x86_64-pc-linux-gnu-gcc -c -march=athlon64 -O3 -pipe -ftracer
-fomit-frame-pointer -DGCC_INLINE -I. -I../src/headers -o init.o
../src/language/init.c
rm -f libpari.so.2.1.7
/usr/bin/ld -o libpari.so.2.1.7 -shared -soname libpari.so.1 -lc -lm kernel.o
mp.o alglin1.o alglin2.o arith1.o arith2.o base1.o base2.o base3.o base4.o
base5.o bibli1.o bibli2.o buch1.o buch2.o buch3.o buch4.o galconj.o gen1.o
gen2.o gen3.o ifactor1.o polarit1.o polarit2.o polarit3.o rootpol.o subgroup.o
trans1.o trans2.o trans3.o elliptic.o galois.o kummer.o mpqs.o nffactor.o
stark.o subfield.o thue.o anal.o compat.o errmsg.o es.o helpmsg.o init.o
sumiter.o mpinl.o
/usr/bin/ld: kernel.o: relocation R_X86_64_32S against `a local symbol' can not
be used when making a shared object; recompile with -fPIC
kernel.o: could not read symbols: Bad value
make: *** [libpari.so.2.1.7] Error 1
GS> The build finishes successfully on reiserfs (real HDD partition).
Juts a suggestion:
could this be related to toolchain? After all in Gentoo gcc is called via
wrapper and not directly (see gcc-config or eselect-compiler sources).
George
Created an attachment (id=83090) [details]
The diff between objdump -x (all headers)
Additional info, as was requested on irc:
Comparison of kernel.o files generated on tmpfs (top line) and reiserfs (bottom
line):
aldar Olinux-x86_64 # ls -l kernel.o
-rw-r--r-- 1 portage portage 3992 2006-03-25 16:16 kernel.o
-rw-r--r-- 1 portage portage 3912 2006-03-25 16:52 kernel.o
Well, this is enough to show that they are different, but nonetheless
aldar Olinux-x86_64 # md5sum kernel.o
8124e0e044606d2f14b56f496156833e kernel.o
ed288821fb6003f6dcc8c3808dcc0377 kernel.o
The diff between objdump -x (all headers) is attached.
George
toolchain: any input here?
When pari is compiled on tmpfs, this error appears:
/usr/bin/ld: kernel.o: relocation R_X86_64_32S against `a local symbol' can not
be used when making a shared object; recompile with -fPIC
kernel.o: could not read symbols: Bad value
kernel.o is a pari object being linked, not related to the Linux kernel. -fPIC
was already used.
On a "real" filesystem, this problem does not happen. George has attached the
results of objdump tinkering.
no, that isnt it at all ... compare the log files
when on a tmpfs system, libpari.so.2.1.7 is compiled twice ... first by the
shared make, and then again during the executable make
the reason comes down to the .headers and pariinl.h files ... just do this,
once on tmpfs and once on non-tmpfs:
# cd $S/Olinux-*
# make clean ; rm -f .headers
# make lib-dyn --debug=b
# make lib-dyn --debug=b
notice how the second make doesnt do anything on non-tmpfs ... if you read the
start, you can see why:
Updating goal targets....
File `lib-dyn' does not exist.
Prerequisite `pariinl.h' is newer than target `.headers'.
Must remake target `.headers'.
Successfully remade target file `.headers'.
Prerequisite `.headers' is newer than target `kernel.o'.
Must remake target `kernel.o'.
you can figure out if this is a feature or a bug of tmpfs, i dont really care,
or you could simply fix the makefile to not suck
two ways this can be done ... use order-only style prerequisites for .headers,
or just stick the list of headers into a variable and have the targets depend
on that
i'll post patches for either and you guys can pick
Created an attachment (id=83657) [details]
pari-order-only.patch
this will work regardless of timestamps ... seems that the other solution i
proposed would fail in the same way because of time differences between the
headers and the sources
this patch however may require GNU make-3.80+ ...
more on the timestamp issue ... maybe someone could explain why pariinl.h gets
a newer timestamp even though it was clearly created first ...
# make lib-dyn --debug=b
Updating goal targets....
File `lib-dyn' does not exist.
File `libpari.so.2.1.7' does not exist.
File `kernel.o' does not exist.
File `pariinl.h' does not exist.
Must remake target `pariinl.h'.
cat ../src/kernel/none/level0.h ../src/kernel/none/level1.h > pariinl.h
Successfully remade target file `pariinl.h'.
Must remake target `kernel.o'.
x86_64-pc-linux-gnu-gcc -c -DGCC_INLINE -Wall -Wno-implicit -O2 -march=k8 -pipe
-fPIC -I. -I../src/headers -o kernel.o ../src/kernel/none/level0.c
Successfully remade target file `kernel.o'.
# stat pariinl.h kernel.o
File: `pariinl.h'
Size: 19505 Blocks: 40 IO Block: 4096 regular file
Device: 13h/19d Inode: 30736291 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2006-04-01 17:07:48.000000000 -0500
Modify: 2006-04-01 17:07:46.810435250 -0500
Change: 2006-04-01 17:07:46.810435250 -0500
File: `kernel.o'
Size: 3976 Blocks: 8 IO Block: 4096 regular file
Device: 13h/19d Inode: 30736294 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2006-04-01 17:07:48.000000000 -0500
Modify: 2006-04-01 17:07:46.000000000 -0500
Change: 2006-04-01 17:07:46.000000000 -0500
*** Bug 131038 has been marked as a duplicate of this bug. ***
*** Bug 123736 has been marked as a duplicate of this bug. ***
*** Bug 114241 has been marked as a duplicate of this bug. ***
*** Bug 134836 has been marked as a duplicate of this bug. ***
I'm looking at this now, because it's hitting me in a lot of packages (gcc and
pciutils to start).
I've noticed one very specific thing so far:
Makefiles that have combinations of high and low resolution timestamps in their
targets are affected, regardless of the filesystem.
GCC provides high resolution timestamps on it's output, but ccache does not
presently. io redirection on the shell does not provide high resolution
timestamps either.
To prove it's not only tmpfs, I set up a high speed reiserfs instance, that was
memory backed, and mounted it at /mnt/cdrom. Here's a route using ccache on top
of that to show the problem nicely (by no means the only route, but definetly
the most common route).
# FEATURES=ccache PORTAGE_TMPDIR=/mnt/cdrom ebuild pciutils-2.2.3.ebuild clean
compile (ignore the output the first time).
# FEATURES=ccache PORTAGE_TMPDIR=/mnt/cdrom ebuild pciutils-2.2.3.ebuild clean
compile
...
# ls -tl /mnt/cdrom/portage/pciutils-2.2.3/work/pciutils-2.2.3/lib/ --full-time
total 348
-rwxr-xr-x 1 portage portage 40839 2006-06-08 22:42:42.321253192 -0700
libpci.so
-rw-r--r-- 1 portage portage 33378 2006-06-08 22:42:42.262253723 -0700 libpci.a
-rw-r--r-- 1 portage portage 156 2006-06-08 22:42:42.127254938 -0700
config.mk
-rw-r--r-- 1 portage portage 366 2006-06-08 22:42:42.125254956 -0700 config.h
-rw-r--r-- 1 portage portage 6108 2006-06-08 22:42:42.000000000 -0700
access.lo
-rw-r--r-- 1 portage portage 6112 2006-06-08 22:42:42.000000000 -0700 access.o
-rw-r--r-- 1 portage portage 3672 2006-06-08 22:42:42.000000000 -0700 dump.lo
-rw-r--r-- 1 portage portage 3536 2006-06-08 22:42:42.000000000 -0700 dump.o
-rw-r--r-- 1 portage portage 2848 2006-06-08 22:42:42.000000000 -0700
filter.lo
-rw-r--r-- 1 portage portage 2916 2006-06-08 22:42:42.000000000 -0700 filter.o
-rw-r--r-- 1 portage portage 3276 2006-06-08 22:42:42.000000000 -0700
generic.lo
-rw-r--r-- 1 portage portage 3152 2006-06-08 22:42:42.000000000 -0700
generic.o
-rw-r--r-- 1 portage portage 7080 2006-06-08 22:42:42.000000000 -0700 names.lo
-rw-r--r-- 1 portage portage 7288 2006-06-08 22:42:42.000000000 -0700 names.o
-rw-r--r-- 1 portage portage 4208 2006-06-08 22:42:42.000000000 -0700 proc.lo
-rw-r--r-- 1 portage portage 4028 2006-06-08 22:42:42.000000000 -0700 proc.o
-rw-r--r-- 1 portage portage 5304 2006-06-08 22:42:42.000000000 -0700 sysfs.lo
-rw-r--r-- 1 portage portage 5176 2006-06-08 22:42:42.000000000 -0700 sysfs.o
-rw-r--r-- 1 portage portage 1486 2006-06-08 22:42:38.274289615 -0700 Makefile
-rw-r--r-- 1 portage portage 3001 2006-05-05 05:29:45.000000000 -0700
obsd-device.c
...
Look at the timestamps for config.h "22:42:42.125254956 -0700" and any of the
object files "22:42:42.000000000 -0700".
config.h is generated at the start of the build process, by a means that give
it a high resolution timestamp.
ccache then outputs the object files, and because of a bug I found in it, it
causes the times of those files to get rounded off. They actually occured at
22:42:42.2* (after the config.h), but the time gets rounded, and ends up BEFORE
the config.h file was generated.
On the next make pass, Make sees that config.h is newer than the object files,
and rebuilds everything.
I cooked up a testcase that tested for sub-second resolution on various
filesystems, and vapier was right about one thing, there IS additional
brokenness on tmpfs.
xfs/jfs here fully support sub-second resolution.
hfs won't ever. ext3 might. reiserfs has it available as a patch, but it was
rejected by linus (I was actually running it without realizing it earlier).
The testcase first does the creat system call, and follows that with a stat to
check the timestamps.
Then it does futimes with a hard-coded value followed by another stat,
and lastly plain utimes with a hard-coded value and another stat.
The hardcoded values all have 3 siginificent digits after the decimal, and you
can see those preserved in jfs/xfs, and not in ext3/hfs.
If the timestamps from the creat call have sub-second precision, then that
should be preserved throughout the testcase.
tmpfs however seems to break this. I'm busy tracing this down further, and then
I'll post here again.
Long term however, almost filesystem and a sufficently fast machine is still
vulnerable to this problem. xfs/jfs take a specific approach and forcably set
the nanoseconds field to the correct current value for the utime(...,NULL) and
utimes(...,NULL) calls. This workaround means a lot less stuff gets broken on
them. tmpfs should probably follow that lead.
jfs
---
creat: testfile.foo mtime=1149842017.182958606 ctime=1149842017.182958606
atime=1149842017.182958606
futimes: testfile.foo mtime=1100000000.456000 ctime=1149842017.182958606
atime=1000000000.123000
utimes: testfile.foo mtime=2100000000.456000 ctime=1149842017.182958606
atime=2000000000.123000
xfs
---
creat: testfile.foo mtime=1149842158.307688472 ctime=1149842158.307688472
atime=1149842158.307688472
futimes: testfile.foo mtime=1100000000.456000 ctime=1149842158.307688472
atime=1000000000.123000
utimes: testfile.foo mtime=2100000000.456000 ctime=1149842158.307688472
atime=2000000000.123000
hfs
---
creat: testfile.foo mtime=1149842296.0 ctime=1149842296.0 atime=1149842296.0
futimes: testfile.foo mtime=1100000000.0 ctime=1149842296.0 atime=1000000000.0
utimes: testfile.foo mtime=2100000000.0 ctime=1149842296.0 atime=2000000000.0
ext3
----
creat: testfile.foo mtime=1149842367.0 ctime=1149842367.0 atime=1149842367.0
futimes: testfile.foo mtime=1100000000.0 ctime=1149842367.0 atime=1000000000.0
utimes: testfile.foo mtime=2100000000.0 ctime=1149842367.0 atime=2000000000.0
tmpfs
-----
creat: testfile.foo mtime=1149842410.769416296 ctime=1149842410.769416296
atime=1149842410.769416296
futimes: testfile.foo mtime=1100000000.0 ctime=1149842410.0 atime=1000000000.0
utimes: testfile.foo mtime=2100000000.0 ctime=1149842410.0 atime=2000000000.0
*** Bug 134836 has been marked as a duplicate of this bug. ***
*** Bug 135450 has been marked as a duplicate of this bug. ***
Linus has accepted my patch for 2.6.17 (as of -rc6-git5). It's also queued for
2.6.16.21.
I'm closing this bug now since it's fixed, and will be available in the tree
once those kernels are released.
For those that need another workaround in the meantime, for building small
packages only, you can try ramfs instead. Or just add the single line to your
kernel source and recompile.
I'll do one better and include it in the upcoming gentoo-sources (coming very
soon)
By the way, thanks muchly for investigating and patching this, much appreciated
Fixed in gentoo-sources-2.6.16-r10