Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 415439 - Bootstrapping coreutils fails on Gentoo FreeBSD 9 (amd64-fbsd)
Summary: Bootstrapping coreutils fails on Gentoo FreeBSD 9 (amd64-fbsd)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All FreeBSD
: Normal normal (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard: Verify fix is in >=coreutils-8.18
Keywords: Bug, REGRESSION
: 412485 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-05-11 01:25 UTC by Richard Yao (RETIRED)
Modified: 2012-08-10 09:39 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Log of bootstrap failure (coreutils.log,54.11 KB, text/plain)
2012-05-11 01:25 UTC, Richard Yao (RETIRED)
Details
Patch from sys-apps/coreutils-8.17 patchset 1.1 (050_all_coreutils-libpthread-detection.patch,2.68 KB, patch)
2012-07-11 14:35 UTC, Richard Yao (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Yao (RETIRED) gentoo-dev 2012-05-11 01:25:29 UTC
Created attachment 311393 [details]
Log of bootstrap failure

I tried installing Gentoo Prefix on a virtual machine running Gentoo FreeBSD 9 (amd64-fbsd). Unfortunatley, that fails with the following:

sort.o: In function `sortlines':
/home/richard/gentoo/tmp/var/tmp/coreutils-8.16/coreutils-8.16/src/sort.c:3572: undefined reference to `pthread_create'

Autotools assumes that the presence of pthread_join() in the system library implies the presence of pthread_create(), so the ./configure script will check for pthread_join() in libc and assume that pthread_create() is also in libc. This is not the case in FreeBSD 9's libc, which results in a Makefile that does not pass -lpthread to the linker. This causes a linker failure in sortlines.
Comment 1 Richard Yao (RETIRED) gentoo-dev 2012-05-11 03:43:48 UTC
The FreeBSD bootstrap procedure still says to use coreutils8, but that was removed from bootstrap-prefix.sh. Reintroducing it will workaround this issue.
Comment 2 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2012-05-11 15:26:59 UTC
Is the solution REALLY to use a back-level version of coreutils or can something be fixed upstream? I have a hard time believing that coreutils stopped supporting freebsd or such.
Comment 3 Fabian Groffen gentoo-dev 2012-05-11 15:56:29 UTC
checking for pthread.h... (cached) yes
checking for pthread_kill in -lpthread... yes

feels more like they forgot PTHREAD_LIBS or whatever
Comment 4 Richard Yao (RETIRED) gentoo-dev 2012-05-11 18:46:01 UTC
(In reply to comment #3)
> checking for pthread.h... (cached) yes
> checking for pthread_kill in -lpthread... yes
> 
> feels more like they forgot PTHREAD_LIBS or whatever

Currently autotools sets LIB_PTHREAD=-lpthread iff pthread_join() is in libc. This causes a failure on FreeBSD where pthread_join() is in libc and pthread_create() is in libpthread.so.

Switching the pthread_join check to a pthread_create check would be the simplest way of addressing this on FreeBSD, but the big question is whether or not that will cause a similar regression on another platform, which would require that a more elaborate check be done.

(In reply to comment #2)
> Is the solution REALLY to use a back-level version of coreutils or can
> something be fixed upstream? I have a hard time believing that coreutils
> stopped supporting freebsd or such.

Which version of coreutils should be patched? bootstrap-prefix.sh currently installs 8.16, but 8.17 is out. It might be better to patch the new version.
Comment 5 Fabian Groffen gentoo-dev 2012-05-11 19:12:20 UTC
sed -i -e s/LIB_PTHREAD/LIBMULTITHREAD/ src/Makefile.{am,in}
Comment 6 Richard Yao (RETIRED) gentoo-dev 2012-05-11 19:30:34 UTC
(In reply to comment #5)
> sed -i -e s/LIB_PTHREAD/LIBMULTITHREAD/ src/Makefile.{am,in}

Is that appropriate to send upstream?
Comment 7 Fabian Groffen gentoo-dev 2012-05-11 20:13:22 UTC
Does it work in the first place?
Comment 8 Richard Yao (RETIRED) gentoo-dev 2012-05-11 21:01:52 UTC
(In reply to comment #7)
> Does it work in the first place?

I tried editing the bootstrap script to do this. It does not appear to work. My change was to add this to bootstrap_gnu below the existing coreutils check:

[[ ${A%-*} == "coreutils" ]] && \
sed -i -e s/LIB_PTHREAD/LIBMULTITHREAD/ "${S}/src/Makefile".{am,in}
Comment 9 Fabian Groffen gentoo-dev 2012-05-12 06:55:33 UTC
I would give --disable-threads a try first
Comment 10 Fabian Groffen gentoo-dev 2012-05-13 09:39:06 UTC
I cannot try this, because I'm still on 8.2 where this seems to work fine
Comment 11 Richard Yao (RETIRED) gentoo-dev 2012-05-13 16:33:29 UTC
(In reply to comment #10)
> I cannot try this, because I'm still on 8.2 where this seems to work fine

I had some things come up yesterday, which prevented me from testing this. Anyway. --disable-threads does not have any effect.

The sortlines function in ./src/sort.c does have any preprocessor directives to permit --disable-threads to disable its dependency on pthreads.
Comment 12 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2012-05-22 14:15:05 UTC
*** Bug 412485 has been marked as a duplicate of this bug. ***
Comment 13 Richard Yao (RETIRED) gentoo-dev 2012-07-05 01:24:34 UTC
Running `cd ${EPREFIX}/tmp/var/tmp/coreutils-8.16/coreutils-8.16 && sed -i -e 's/LIB_PTHREAD =/& -lpthread/' ./src/Makefile && make && make install` will workaround this problem.

Fixing it will require modifying ./m4/pthread.m4 and rebuilding the ./configure script.
Comment 14 Richard Yao (RETIRED) gentoo-dev 2012-07-05 01:36:04 UTC
./m4/pthread.m4 does not exist in the coreutils repository. Where does this file originate?
Comment 15 Richard Yao (RETIRED) gentoo-dev 2012-07-05 01:43:26 UTC
I have the answer to my own question. Someone put it in the tarball that is on the FSF's FTP server. This is not under version control. I am going to contact the upstream maintainers.
Comment 16 Richard Yao (RETIRED) gentoo-dev 2012-07-05 02:11:50 UTC
It seems that ./m4/pthread.m4 is from gnulib, which coreutils uses. This regression was caused by commit a6322904ea944ba119aaa62c0c3ee7cfade8b59c:

http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blobdiff;f=m4/pthread.m4;h=1ac8027dda9ee18ae38895295f2b18dc65ad9595;hp=b60dbc850633fecbffbf4d9c4d6a923316f413b7;hb=a6322904ea944ba119aaa62c0c3ee7cfade8b59c;hpb=6563cd7648f78ce137eb66d4a9a77f9f1666d17f

Coreutils 8.12 is the last version that predates that commit.
Comment 17 Richard Yao (RETIRED) gentoo-dev 2012-07-05 02:21:26 UTC
I have sent the gnulib developers an email describing the problem. Hopefully, they will produce a proper fix.
Comment 18 Richard Yao (RETIRED) gentoo-dev 2012-07-05 06:16:14 UTC
Upstream has written a patch:

http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00043.html

I have verified that it works.
Comment 19 Richard Yao (RETIRED) gentoo-dev 2012-07-05 06:48:43 UTC
(In reply to comment #18)
> Upstream has written a patch:
> 
> http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00043.html
> 
> I have verified that it works.

The patch has been committed:

http://lists.gnu.org/archive/html/bug-gnulib/2012-07/msg00045.html
Comment 20 Fabian Groffen gentoo-dev 2012-07-05 06:49:14 UTC
so, you need 8.12 until a coreutils gets released with the pushed fix to gnulib?

Guess I/we can live with 8.12...
Comment 21 Richard Yao (RETIRED) gentoo-dev 2012-07-05 12:11:50 UTC
(In reply to comment #20)
> so, you need 8.12 until a coreutils gets released with the pushed fix to
> gnulib?

There is the option of adopting a patched coreutils tarball like the python tarball. Otherwise, this is correct.

> Guess I/we can live with 8.12...

This is the easiest thing to do.
Comment 22 Richard Yao (RETIRED) gentoo-dev 2012-07-11 14:35:23 UTC
Created attachment 317928 [details, diff]
Patch from sys-apps/coreutils-8.17 patchset 1.1

We could modify bootstrap prefix to fetch and apply a patch to workaround this. I have attached the patch that went into the coreutils patchset to resolve this.
Comment 23 Richard Yao (RETIRED) gentoo-dev 2012-07-11 14:52:52 UTC
(In reply to comment #22)
> Created attachment 317928 [details, diff] [details, diff]
> Patch from sys-apps/coreutils-8.17 patchset 1.1
> 
> We could modify bootstrap prefix to fetch and apply a patch to workaround
> this. I have attached the patch that went into the coreutils patchset to
> resolve this.

The prefix-overlay coreutils ebuild is unaffected by this issue because of these lines:

# m4/pthread.m4 on FreeBSD thinks pthread_join doesn't need any libaries
# sort.c:(.text+0x4f25): undefined reference to `pthread_create'
[[ ${CHOST} == *-freebsd* ]] && export gl_cv_search_pthread_join=-pthread

Those lines could be deleted if patchset 1.1 is imported from the main tree. With that said, we could modify bootstrap-prefix.sh to do this, which would close this bug.
Comment 24 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2012-07-11 14:55:47 UTC
(In reply to comment #22)
> Created attachment 317928 [details, diff] [details, diff]
> Patch from sys-apps/coreutils-8.17 patchset 1.1
> 
> We could modify bootstrap prefix to fetch and apply a patch to workaround
> this. I have attached the patch that went into the coreutils patchset to
> resolve this.

That isn't advisable. The priority here is low so we can leave the bug open and wait for two things 1) a new release that has the fix, 2) other users to complain and thus the priority increases.

For now, I've just added back 8.12 to bootstrap-prefix.sh. Feel free to fix the documentation, please.

% svn di
Index: bootstrap-prefix.sh
===================================================================
--- bootstrap-prefix.sh (revision 61051)
+++ bootstrap-prefix.sh (working copy)
@@ -819,6 +819,11 @@
        bootstrap_gnu coreutils 8.16
 }
 
+# bug 415439
+bootstrap_coreutils812() {
+       bootstrap_gnu coreutils 8.12
+}
+
 bootstrap_tar() {
        bootstrap_gnu tar 1.26
 }
Comment 25 Richard Yao (RETIRED) gentoo-dev 2012-07-11 14:59:56 UTC
(In reply to comment #24)
> (In reply to comment #22)
> > Created attachment 317928 [details, diff] [details, diff] [details, diff]
> > Patch from sys-apps/coreutils-8.17 patchset 1.1
> > 
> > We could modify bootstrap prefix to fetch and apply a patch to workaround
> > this. I have attached the patch that went into the coreutils patchset to
> > resolve this.
> 
> That isn't advisable. The priority here is low so we can leave the bug open
> and wait for two things 1) a new release that has the fix, 2) other users to
> complain and thus the priority increases.

I agree. A simpler option would be to modify bootstrap-prefix.sh to do `[[ ${CHOST} == *-freebsd* ]] && export gl_cv_search_pthread_join=-pthread` when bootstrapping coreutils.

I spotted that in the prefix overlay's coreutils ebuild when trying to understand why it did not need my patch.
Comment 26 Fabian Groffen gentoo-dev 2012-08-10 09:39:03 UTC
bootstrap-prefix.sh now does a iterative loop over versions until one succeeds