Created attachment 645744 [details] build.log All failed builds in this log have the same error: USE='-acl -examples -iconv -ipv6 lz4 ssl -static -stunnel -system-zlib -xattr -xxhash -zstd' succeeded for =net-misc/rsync-3.2.0-r1 USE='-acl examples -iconv -ipv6 lz4 -ssl static -stunnel -system-zlib -xattr -xxhash zstd' failed for =net-misc/rsync-3.2.0-r1 log has been saved as /root/tatt/logs/net-misc_rsync-3.2.0-r1_use_vZmMq USE='-acl -examples iconv ipv6 lz4 -ssl static -stunnel system-zlib -xattr -xxhash zstd' failed for =net-misc/rsync-3.2.0-r1 log has been saved as /root/tatt/logs/net-misc_rsync-3.2.0-r1_use_ARxyk USE='-acl examples -iconv ipv6 -lz4 ssl -static stunnel system-zlib -xattr -xxhash zstd' succeeded for =net-misc/rsync-3.2.0-r1 USE='-acl examples iconv -ipv6 -lz4 -ssl static -stunnel -system-zlib xattr -xxhash zstd' failed for =net-misc/rsync-3.2.0-r1 log has been saved as /root/tatt/logs/net-misc_rsync-3.2.0-r1_use_ptEwX USE='acl examples -iconv -ipv6 -lz4 ssl -static -stunnel system-zlib xattr -xxhash zstd' succeeded for =net-misc/rsync-3.2.0-r1 USE='acl -examples iconv -ipv6 -lz4 ssl -static -stunnel system-zlib xattr -xxhash zstd' succeeded for =net-misc/rsync-3.2.0-r1 USE='-acl examples -iconv ipv6 lz4 ssl -static stunnel -system-zlib -xattr xxhash zstd' succeeded for =net-misc/rsync-3.2.0-r1 USE='acl examples -iconv -ipv6 lz4 -ssl -static -stunnel -system-zlib xattr xxhash zstd' succeeded for =net-misc/rsync-3.2.0-r1 USE='acl -examples -iconv -ipv6 -lz4 ssl -static stunnel system-zlib xattr xxhash zstd' succeeded for =net-misc/rsync-3.2.0-r1
Created attachment 645746 [details] build.log There is a distinct, but probably related build error here: token.c: In function ‘send_zstd_token’: token.c:685:2: error: unknown type name ‘ZSTD_EndDirective’ 685 | ZSTD_EndDirective flush = ZSTD_e_continue; | ^~~~~~~~~~~~~~~~~
Does this really happen with USE="-zstd" as well?
No, USE=-zstd builds were fine.
So to summarize this, we have two maybe unrelated issues with USE="zstd": * On hppa, configure already fails with: checking whether to enable zstd compression... yes checking for library containing ZSTD_minCLevel... no configure.sh: error: Failed to find ZSTD_minCLevel function in zstd lib. Use --disable-zstd to continue without zstd compression. * On sparc, configure succeeds but build later fails with: token.c: In function ‘send_zstd_token’: token.c:685:2: error: unknown type name ‘ZSTD_EndDirective’ and some more error messages (too many to post them here all. See second build.log attachment for more). We should now figure out if these issues come from something being wrong in the app-arch/zstd package or in the net-misc/rsync package.
Correct. Here is the state on sparc: * Found these USE flags for app-arch/zstd-1.4.4-r4: U I - - lz4 : Enable support for lz4 compression (as implemented in app-arch/lz4) - - static-libs : Build static versions of dynamic libraries as well + + threads : Add threads support for various packages. Usually pthreads And this is hppa: * Found these USE flags for app-arch/zstd-1.4.4-r4: U I - + lz4 : Enable support for lz4 compression (as implemented in app-arch/lz4) - + static-libs : Build static versions of dynamic libraries as well + + threads : Add threads support for various packages. Usually pthreads Then I went and used USE="lz4 static-libs" on sparc also, and: checking for library containing ZSTD_minCLevel... no configure.sh: error: Failed to find ZSTD_minCLevel function in zstd lib. Use --disable-zstd to continue without zstd compression. And config.log shows what's going wrong: configure.sh:6322: sparc-unknown-linux-gnu-gcc -o conftest -O2 -mcpu=niagara2 -pipe -Wcast-align -DHAVE_CONFIG_H -Wall -W -Wl,-O1 -Wl,--as-needed -static conftest.c -lzstd >&5 /usr/lib/gcc/sparc-unknown-linux-gnu/9.3.0/../../../../sparc-unknown-linux-gnu/bin/ld: /usr/lib/gcc/sparc-unknown-linux-gnu/9.3.0/../../../../lib/libzstd.a(zstdmt_compress.o): in function `ZSTDMT_freeJobsTable': (.text+0x12c): undefined reference to `pthread_mutex_destroy'
I think our current stable zstd[threads] is completely broken due to bug #713940 (shared library is underlinked against -lpthread).
Rolf, can you try with latest unstable app-arch/zstd and report back your results?
The underlinking is still the same with 1.4.5.
(In reply to Rolf Eike Beer from comment #5) > Correct. Here is the state on sparc: > > * Found these USE flags for app-arch/zstd-1.4.4-r4: > U I > - - lz4 : Enable support for lz4 compression (as implemented in > app-arch/lz4) > - - static-libs : Build static versions of dynamic libraries as well > + + threads : Add threads support for various packages. Usually pthreads > > And this is hppa: > > * Found these USE flags for app-arch/zstd-1.4.4-r4: > U I > - + lz4 : Enable support for lz4 compression (as implemented in > app-arch/lz4) > - + static-libs : Build static versions of dynamic libraries as well > + + threads : Add threads support for various packages. Usually pthreads > > Then I went and used USE="lz4 static-libs" on sparc also, and: > > checking for library containing ZSTD_minCLevel... no > configure.sh: error: Failed to find ZSTD_minCLevel function in zstd lib. > Use --disable-zstd to continue without zstd compression. > > And config.log shows what's going wrong: > > configure.sh:6322: sparc-unknown-linux-gnu-gcc -o conftest -O2 > -mcpu=niagara2 -pipe -Wcast-align -DHAVE_CONFIG_H -Wall -W -Wl,-O1 > -Wl,--as-needed -static conftest.c -lzstd >&5 > /usr/lib/gcc/sparc-unknown-linux-gnu/9.3.0/../../../../sparc-unknown-linux- > gnu/bin/ld: > /usr/lib/gcc/sparc-unknown-linux-gnu/9.3.0/../../../../lib/libzstd. > a(zstdmt_compress.o): in function `ZSTDMT_freeJobsTable': > (.text+0x12c): undefined reference to `pthread_mutex_destroy' That is with USE=static for rsync, right? I noticed /usr/lib64/pkgconfig/libzstd.pc also does not refer -pthread in any form in private libs (zstd bug). If rsync uses pkg-config that will explain link failures as well.
just a note that this is present in app-arch/zstd-1.4.5 on amd64 if these *FLAGS are used: CFLAGS="-Os -pipe -march=ivybridge -mtune=skylake -frecord-gcc-switches -fdiagnostics-color=never -ffunction-sections -fdata-sections -flto" CXXFLAGS="-Os -pipe -march=ivybridge -mtune=skylake -frecord-gcc-switches -fdiagnostics-color=never -ffunction-sections -fdata-sections -flto" FCFLAGS="-Os -pipe -march=ivybridge -mtune=skylake" FFLAGS="-Os -pipe -march=ivybridge -mtune=skylake" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--gc-sections -flto"
It fails with rsync 3.2.3 and zstd 1.3.xx You see the doc INSTALL.md, rsync 3.2.x need at least version 1.4 ----------------------- ## zstd The [zstd library][2] compression algorithm that uses less CPU than the default zlib algorithm at the same compression level. Note that you need at least version 1.4, so you might need to skip the zstd compression if you can only install a 1.3 release. Installing this development library adds zstd compression as the default compression algorithm. [2]: http://facebook.github.io/zstd/ ----------------------- path to solving the problem ----------------------- --- rsync-3.2.3.ebuild.old 2020-09-07 11:09:38.000000000 +0200 +++ rsync-3.2.3.ebuild 2020-09-17 09:06:40.493533175 +0200 @@ -37,7 +37,7 @@ system-zlib? ( sys-libs/zlib[static-libs(+)] ) xattr? ( kernel_linux? ( sys-apps/attr[static-libs(+)] ) ) xxhash? ( dev-libs/xxhash[static-libs(+)] ) - zstd? ( app-arch/zstd[static-libs(+)] ) + zstd? ( >=app-arch/zstd-1.4[static-libs(+)] ) >=dev-libs/popt-1.5[static-libs(+)]" RDEPEND="!static? ( ${LIB_DEPEND//\[static-libs(+)]} ) iconv? ( virtual/libiconv )" -----------------------
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6e754603cf2eb976b3d1c0975db1d79904df43b8 commit 6e754603cf2eb976b3d1c0975db1d79904df43b8 Author: Lars Wendler <polynomial-c@gentoo.org> AuthorDate: 2020-09-18 10:39:23 +0000 Commit: Lars Wendler <polynomial-c@gentoo.org> CommitDate: 2020-09-18 10:39:36 +0000 net-misc/rsync: Adjusted dependency on app-arch/zstd Bug: https://bugs.gentoo.org/729186 Package-Manager: Portage-3.0.7, Repoman-3.0.1 Signed-off-by: Lars Wendler <polynomial-c@gentoo.org> net-misc/rsync/rsync-3.2.3.ebuild | 2 +- net-misc/rsync/rsync-9999.ebuild | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
