| Summary: | >=sys-fs/cryptsetup-1.6.0[gcrypt,static]: libgcrypt with threads enabled is not compatible with static linking | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Marko D <markodevelop> |
| Component: | [OLD] Core system | Assignee: | Gentoo's Team for Core System packages <base-system> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | alonbl, amade, anton.kochkov, antu456, bitlord0xff, crypto+disabled, freddie, gentoo, gentoobugs, guillaume.ramelet, help, info, kripton, mark, pageexec, rei4dan, Storklerk, ViliusSutkus89 |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=520450 | ||
| Whiteboard: | Maintainer note: REQUIRED_USE="static? ( !gcrypt )" in ebuild to prevent this bug. | ||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | 521236 | ||
| Bug Blocks: | |||
| Attachments: |
emerge --info output
build.log |
||
|
Description
Marko D
2013-12-31 18:52:07 UTC
Created attachment 366656 [details]
emerge --info output
Please attach the entire build log to this bug report. Created attachment 366704 [details]
build.log
Here it is
relies heavily on the fact that libgcrypt doesn't provide a pkg-config file, so querying for its linkage info is hard (impossible?) maybe we just add REQUIRED_USE="static? ( !gcrypt )" (In reply to SpanKY from comment #4) > relies heavily on the fact that libgcrypt doesn't provide a pkg-config file, > so querying for its linkage info is hard (impossible?) there is libgcrypt-config, but I don't see libgcrypt.so linked against -lpthread on my system looks like a direct duplicate of bug 491806 instead, broken .pc file in lvm2 *** This bug has been marked as a duplicate of bug 491806 *** (In reply to Samuli Suominen from comment #5) look at the first error in the list: /usr/lib64/libgcrypt.a(libgcrypt_la-ath.o): In function `_gcry_ath_mutex_lock': (.text+0x284): undefined reference to `pthread_mutex_lock' that's the libgcrypt library using pthread symbols note that simply not being linked against libpthread doesn't mean a library can't support pthreads. the C library provides dummy symbols for a lot of pthread_* funcs so that a library can transparently support threads, but only if the application itself has linked against it. (In reply to SpanKY from comment #6) > (In reply to Samuli Suominen from comment #5) > > look at the first error in the list: > > /usr/lib64/libgcrypt.a(libgcrypt_la-ath.o): In function > `_gcry_ath_mutex_lock': > (.text+0x284): undefined reference to `pthread_mutex_lock' > > that's the libgcrypt library using pthread symbols > > note that simply not being linked against libpthread doesn't mean a library > can't support pthreads. the C library provides dummy symbols for a lot of > pthread_* funcs so that a library can transparently support threads, but > only if the application itself has linked against it. What if we tried -Wl,--whole-archive -pthread -Wl,--no-whole-archive? Pretty common on Linux boxes that anything involving pthreads + static linking has fits without that. I know that dev-util/pkgconfig respects the whole archive wrapping now (I believe it was fixed in 0.26 or 0.27 and we have 0.28 stable) but not sure about dev-util/pkgconf. (In reply to Doug Goldstein from comment #7) pretty sure we don't want to do that. just make sure -lpthread shows up after the libs that need it (gcrypt in this case). It compiles with "-Wl,--whole-archive -pthread -Wl,--no-whole-archive". Cryptsetup linked without pthreads library with other linking optimization flags, even if I pasted "-lpthread" in Makefiles. At libcrypt-1.6 side we have: --- /* Define if the POSIX multithreading library can be used. */ #define USE_POSIX_THREADS 1 /* Define if references to the POSIX multithreading library should be made weak. */ #define USE_POSIX_THREADS_WEAK 1 #if USE_POSIX_THREADS_WEAK # include <pthread.h> #endif /* On an ELF system it is easy to use pthreads using weak references. Take care not to test the address of a weak referenced function we actually use; some GCC versions have a bug were &foo != NULL is always evaluated to true in PIC mode. USING_PTHREAD_AS_DEFAULT is used by ath_install to detect the default usage of pthread. */ #if USE_POSIX_THREADS_WEAK # pragma weak pthread_cancel # pragma weak pthread_mutex_init # pragma weak pthread_mutex_lock # pragma weak pthread_mutex_unlock # pragma weak pthread_mutex_destroy #endif --- As much as I hate these magics, I do not think it is capable of working in static mode. Maybe add --disable-threads at libgcrypt to be able to be linked as static? *** Bug 506242 has been marked as a duplicate of this bug. *** My personal solution for static linking (yes, it fails horribly, causing segfaults, see #506242):
sed -i 's/#define USE_POSIX_THREADS_WEAK 1//' ${S}_build/config.h
sed -i 's/libs="-lgcrypt"/libs="-lgcrypt -lpthread"/' ${S}_build/src/libgcrypt-config
in the end of libgcrypt's src_configure section, after configure has run. This will turn the weak reference into a hard reference, and tell all programs that use libgcrypt-config to link against pthreads as well.
WorksForMe™
*** Bug 516058 has been marked as a duplicate of this bug. *** *** Bug 516224 has been marked as a duplicate of this bug. *** What about users who need cryptsetup statically linked binary for their initrd to open an encrypted root fs? (In reply to Felix Tiede from comment #15) > What about users who need cryptsetup statically linked binary for their > initrd to open an encrypted root fs? Users who use initrd specifically *don't* need static cryptsetup or lvm2, they should simply copy the dynamic libraries cryptsetup or lvm2 needs to the initrd, genkernel for example has copied the libraries it needs for quite a while now USE="static" is a specific hack for users who *don't* use initrd/initramfs and who use separate /usr, and by the way, it has been decided such an setup is not really viable, which is why you don't see people actively working on the bug, which is why USE="static" is now in package.use.stable.mask My workaround for both this bug and #516224:
darkhorse ~ # cat /etc/portage/env/pthread.conf
LDFLAGS="${LDFLAGS} -pthread"
darkhorse ~ # grep cryptsetup /etc/portage/package.env/local
sys-fs/cryptsetup pthread.conf
Hmm, my bad, this workaround makes cryptsetup compile, but it segfaults when trying to use luksOpen command:( (In reply to parafin from comment #18) > Hmm, my bad, this workaround makes cryptsetup compile, but it segfaults when > trying to use luksOpen command:( Yep, that's why I didn't commit the compile fixes for static + pthread because the end result was just crashing here as well That's why USE="static" is now in /usr/portage/profiles/base/package.use.stable.mask, and I'm considering making it a package.use.mask instead, so it'll be non-option also for ~arch users The current situation is like... - Running `sed` on configure of udev (systemd) to allow --enable-static flag to work again, despite udev upstream explicitely declaring static linking unsupported This all to get libudev.a for USE="static" of lvm2, cryptsetup - Carrying dozen patches to make the static build work, none of them have been accepted by upstream, and more patching is required So, I think it might be time to drop USE=static for cryptsetup and lvm2 It crashes only with gcrypt backend, tested with nettle USE flag - works fine (workaround is still needed because of bug #516224). I would prefer static USE flag to remain, even if it will block some others (gcrypt, udev) - it is much easier to build custom initrd using static binaries. The segfaults really seems specific to the gcrypt, because the openssl backend is working fine for me (See: https://bugs.gentoo.org/show_bug.cgi?id=516224#c10 ) So would some USEDEP like 'static? (!gcrypt)' acceptable instead of masking 'static'? I am using cryptsetup in a custom initrd and I am rather happy to just to put busybox, a static mdadm and a static cryptsetup into it. ldd shows 12 libraries for the nonstatic build and I would like to stay with one file instead of 13. (And I'm not sure what else would be needed for the dynamic linker. Do I need to create a /etc/ld.so.conf for the initrd?) Wrt to udev: http://code.google.com/p/cryptsetup/source/browse/lib/libdevmapper.c#67 That seems to be the only place in cryptsetup that has anything to do with the udev option from configure. http://code.google.com/p/cryptsetup/issues/detail?id=167#c3 seems to agree with that: Udev is not really needed for cryptsetup, only as an indirect dependency via libdevmapper / lvm2. As udev is an optional dependency of lvm2, why not add a USEDEP like static? (!udev). For the case of an simple initrd the missing udev support would not really matter. (My initrd does not contain anything from udev...) Maybe add a comment to that dependency pointing to the upstreams decision to drop static support from libudev and their bugzilla, so anyone who hits it can complain there. (In reply to Torsten Kaiser from comment #21) > The segfaults really seems specific to the gcrypt, because the openssl > backend is working fine for me (See: > https://bugs.gentoo.org/show_bug.cgi?id=516224#c10 ) > > So would some USEDEP like 'static? (!gcrypt)' acceptable instead of masking > 'static'? and the link to problems with openssl backend is here, bug 516224 and even if both gcrypt and openssl are excluded, it's not enough, because there is also bug 520450 and once all these 3 are solved, i'm sure more will crop up as well (In reply to Samuli Suominen from comment #22) > [3 current issues with USE=static] 1. Issue: This bug wrt gcrypt backend. I dont know the cause of it or how to fix it, thats why I suggested disabling this backend instead of completly killig USE=static 2. Issue: bug 516224 wrt openssl backend: I filed that bug and as written believe that is easy to fix. It looks to me that what is causing this bug is not a general problem with static cryptsetup, but that the known and working fix for lvm2 not declaring its dependencys correctly (missing "Libs.private: -L${libdir} @PTHREAD_LIBS@" line in its pkconfig file) was accidently dropped when regeneration the lvm2 patch that fixed this. Why was this hunk dropped? The CVS commit says "refresh 2 patches so that they apply again (no real changes in them)" which very strongly suggest that the removal was *not* intended. 3. Issue: bug 520450: lvm2-2.02.109[static] fails. I never hit that bug. Probably because only [static-libs] are needed to build the static cryptsetup that I needed / requested. Yes, it might make sense to drop USE=static from lvm2, if #520450 can't be solved. It might even be nessesary to drop it, if a USEDEP of static?(!udev) gets intoduced, if too much of lvm2 really need udev. But I'm not seeing this blocking cryptsetup[static]... > i'm sure more will crop up as well ... after rebuilding lvm2-2.02.109 with the fixed patch, cryptsetup[static] works fine for me, Not only mounting /home during the normal boot phase, but also in my custom initrd for mounting /. ... on my stable amd64 system the upgrade to lvm2-2.02.109 also worked fine, when it was marked stable. Its only that there cryptsetup is now a dynamic executable, because its 'static' has been masked. For cryptsetup, REQUIRED_USE="static? ( !gcrypt )" is in place now, and bug 521236 has been opened to get libgcrypt[-threads] for static linking For lvm2, 2.02.109-r1 to ~arch was revision bumped with REQUIRED_USE="static? ( !udev )" because of >=util-linux-2.25 -- no idea yet what to do with this one, this might be the end of USE="static" for lvm2 :-/ Bottom line is, USE="static" masks are removed now, *for now* "cryptsetup static gcrypt" builds and works fine with sys-fs/lvm2-2.02.103, which is the only stable package for it now and why I simply masked newer versions. So forcing "static? ( !gcrypt )" (even for the stable versions of cryptsetup) really looks inappropiate to me. So, no one feels the need to comment this? I now added cryptsetup to my overlay with changed ebuilds. Still compiling and working fine with +gcrypt +static in combination with sys-fs/lvm2-2.02.103, which is a stable release inside the official tree. (In reply to Alexander Hof from comment #26) > So, no one feels the need to comment this? I now added cryptsetup to my > overlay with changed ebuilds. Still compiling and working fine with +gcrypt > +static in combination with sys-fs/lvm2-2.02.103, which is a stable release > inside the official tree. I also have copy of ebuild in local overlay, and using 'static gcrypt', works fine for me on two systems. sys-fs/cryptsetup-1.6.5 (gcrypt nls python_single_target_python2_7 python_targets_python2_7 python_targets_python3_3 static udev) sys-fs/lvm2-2.02.109 (readline static static-libs thin) I did have some issues with 'libgcrypt' so I masked it and using (runtime issues I think related to bug #497654 ) dev-libs/libgcrypt-1.5.4-r1 (abi_x86_64 static-libs) I am running cryptsetup-2.3.2 with USE="gcrypt static" with the following modification in a local overlay:
--- /var/db/repos/gentoo/sys-fs/cryptsetup/cryptsetup-2.3.2.ebuild 2020-05-20 16:09:05.000000000 -0000
+++ cryptsetup-2.3.2.ebuild 2020-07-19 16:32:28.180027126 -0000
@@ -18,8 +18,7 @@
# and it's missing ripemd160 support so it can't provide full backward compatibility
IUSE="${CRYPTO_BACKENDS} +argon2 libressl luks1_default nls pwquality reencrypt static static-libs +udev urandom"
REQUIRED_USE="^^ ( ${CRYPTO_BACKENDS//+/} )
- libressl? ( openssl )
- static? ( !gcrypt )" #496612
+ libressl? ( openssl )"
LIB_DEPEND="
dev-libs/json-c:=[static-libs(+)]
Can we remove this restriction from upstream?
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eb2306ff56867a518eea3b5369eb3cac7f4a35ec commit eb2306ff56867a518eea3b5369eb3cac7f4a35ec Author: Luca Barbato <lu_zero@gentoo.org> AuthorDate: 2024-07-24 17:01:26 +0000 Commit: Luca Barbato <lu_zero@gentoo.org> CommitDate: 2024-07-24 17:04:41 +0000 sys-fs/cryptsetup: Remove the restriction on gcrypt gcrypt now builds correctly as +static-libs and cryptsetup +static builds and works correctly. Closes: https://bugs.gentoo.org/496612 Signed-off-by: Luca Barbato <lu_zero@gentoo.org> sys-fs/cryptsetup/cryptsetup-2.7.3.ebuild | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) |
