Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 820284 - net-dns/bind uses wrong openssl/opensslconf.h when cross compiling
Summary: net-dns/bind uses wrong openssl/opensslconf.h when cross compiling
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: ARM64 Linux
: Normal normal (vote)
Assignee: Mikle Kolyada (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-26 04:23 UTC by Janpieter Sollie
Modified: 2021-11-10 05:40 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build log of aarch64 bind compile (build.log,832.04 KB, application/octet-stream)
2021-10-26 04:58 UTC, Janpieter Sollie
Details
build log of aarch64 bind compile (build.log,832.04 KB, application/octet-stream)
2021-10-26 04:58 UTC, Janpieter Sollie
Details
build.log of aarch64 openssl compile (build.log,748.31 KB, application/octet-stream)
2021-10-26 05:34 UTC, Janpieter Sollie
Details
cross-emerge --target aarch4-unknown-linux-gnueabi --info (file_820284.txt,9.69 KB, text/plain)
2021-10-26 05:36 UTC, Janpieter Sollie
Details
net-dns/bind-9.16.21: Use ESYSROOT. Specify dlz-bdb path. (bind-9.16.21.ebuild.patch,751 bytes, patch)
2021-10-26 13:00 UTC, Tee KOBAYASHI
Details | Diff
(revised) net-dns/bind-9.16.21: Use ESYSROOT. Specify dlz-bdb path. (bind-9.16.21.ebuild-r1.patch,569 bytes, patch)
2021-10-26 13:14 UTC, Tee KOBAYASHI
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Janpieter Sollie 2021-10-26 04:23:17 UTC
when cross compiling (cross-emerge) from x86_64 to aarch64, any packages (eg bind) which use openssl do not compile as the aarch64 /arm64 / arm component in /usr/include/openssl/opensslconf.h is not declared, while other arches (mips, powerpc) are

Reproducible: Always

Steps to Reproduce:
1.create cross-compile toolchain for aarch64-unknown-linux-gnueabi or gnu
2.emerge openssl
3.cross-emerge bind + openssl
Actual Results:  
compile fails: ABI is not supported

Expected Results:  
bind compiles correctly

Portage 3.0.28 (python 3.9.7-final-0, default/linux/amd64/17.1, gcc-10.3.0, glibc-2.33-r7, 5.14.12release+ x86_64)                                             =================================================================                                                                                              System uname: Linux-5.14.12release+-x86_64-AMD_Ryzen_Threadripper_2950X_16-Core_Processor-with-glibc2.33                                                       KiB Mem:    32785512 total,   1483364 free                                                                                                                     KiB Swap:   97022380 total,  97020076 free                                                                                                                     Timestamp of repository gentoo: Mon, 25 Oct 2021 08:15:01 +0000                                                                                                Head commit of repository gentoo: 9f30818120d8d4b59cae1403d12a1affcf7a4b1a                                                                                     sh bash 5.1_p8                                                                                                                                                 ld GNU ld (Gentoo 2.37_p1 p0) 2.37                                                                                                                             distcc 3.4 x86_64-pc-linux-gnu [disabled]                                                                                                                      app-shells/bash:          5.1_p8::gentoo                                                                                                                       dev-java/java-config:     2.3.1::gentoo                                                                                                                        dev-lang/perl:            5.34.0-r3::gentoo                                                                                                                    dev-lang/python:          2.7.18_p13::gentoo, 3.8.12_p1::gentoo, 3.9.7_p1::gentoo                                                                              dev-lang/rust:            1.53.0::gentoo                                                                                                                       dev-util/cmake:           3.21.3::gentoo                                                                                                                       sys-apps/baselayout:      2.7-r3::gentoo                                                                                                                       sys-apps/openrc:          0.43.3::gentoo                                                                                                                       sys-apps/sandbox:         2.27::gentoo                                                                                                                         sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo, 2.71-r1::gentoo                                                                                    sys-devel/automake:       1.13.4-r2::gentoo, 1.16.5::gentoo                                                                                                    sys-devel/binutils:       2.36.1-r2::gentoo, 2.37_p1::gentoo                                                                                                   sys-devel/gcc:            9.3.0-r2::gentoo, 10.3.0::gentoo, 11.2.0::gentoo                                                                                     sys-devel/gcc-config:     2.4::gentoo                                                                                                                          sys-devel/libtool:        2.4.6-r6::gentoo                                                                                                                     sys-devel/make:           4.3::gentoo                                                                                                                          sys-kernel/linux-headers: 5.14::gentoo (virtual/os-headers)                                                                                                    sys-libs/glibc:           2.33-r7::gentoo                                                                                                                      Repositories:                                                                                                                                                                                                                                                                                                                 gentoo                                                                                                                                                             location: /var/db/repos/gentoo                                                                                                                                 sync-type: rsync                                                                                                                                               sync-uri: rsync://rsync.de.gentoo.org/gentoo-portage/                                                                                                          priority: -1000                                                                                                                                                sync-rsync-verify-metamanifest: yes                                                                                                                            sync-rsync-verify-jobs: 1                                                                                                                                      sync-rsync-extra-opts:                                                                                                                                         sync-rsync-verify-max-age: 24                                                                                                                                                                                                                                                                                             gamarouns                                                                                                                                                          location: /var/lib/layman/gamarouns                                                                                                                            masters: gentoo                                                                                                                                                priority: 50                                                                                                                                                                                                                                                                                                              guru                                                                                                                                                               location: /var/lib/layman/guru                                                                                                                                 sync-type: laymansync                                                                                                                                          sync-uri: https://anongit.gentoo.org/git/repo/proj/guru.git                                                                                                    masters: gentoo                                                                                                                                                priority: 50                                                                                                                                                                                                                                                                                                              vmacs                                                                                                                                                              location: /var/lib/layman/vmacs                                                                                                                                masters: gentoo                                                                                                                                                priority: 50                                                                                                                                                                                                                                                                                                              ACCEPT_KEYWORDS="amd64 ~amd64"                                                                                                                                 ACCEPT_LICENSE="*"                                                                                                                                             CBUILD="x86_64-pc-linux-gnu"                                                                                                                                   CFLAGS="-march=znver1 -O3 -pipe"                                                                                                                               CHOST="x86_64-pc-linux-gnu"                                                                                                                                    CONFIG_PROTECT="/etc /etc/stunnel/stunnel.conf /usr/share/gnupg/qualified.txt /var/bind"                                                                       CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.0/ext-active/ /etc/php/cgi-php8.0/ext-active/ /etc/php/cli-php8.0/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"                                               CXXFLAGS="-march=znver1 -O3 -pipe"                                                                                                                             DISTDIR="/data/distfiles"                                                                                                                                      ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"                                                                                                                   FCFLAGS="-march=znver1 -O3 -pipe"                                                                                                                              FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned qa-unresolved-soname-deps sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersync xattr"                                                                                                                                                 FFLAGS="-march=znver1 -O3 -pipe"                                                                                                                               GENTOO_MIRRORS="http://ftp.snt.utwente.nl/pub/os/linux/gentoo"                                                                                                 LANG="en_US.utf8"                                                                                                                                              LDFLAGS="-Wl,-O1 -Wl,--as-needed"                                                                                                                              MAKEOPTS="-j40"                                                                                                                                                PKGDIR="/var/cache/binpkgs"                                                                                                                                    PORTAGE_CONFIGROOT="/"                                                                                                                                         PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"                                                                              PORTAGE_TMPDIR="/var/tmp"                                                                                                                                      USE="acl amd64 bind bzip2 caps cli crypt dbus dri fortran gdbm gnutls iconv ipv6 libglvnd libtirpc lzma mesa multilib ncurses nfs nftables nptl opencl openmp openssl pam pcre python readline samba seccomp sha3 split-usr ssl sslv2 sslv3 tcpd udev unicode urandom usb xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2019" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes sha sse sse2 sse3 avx avx2 bmi1 bmi2 sse4_1 sse4_2 ssse3 pni mmx mmxext fma3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AMDGPU X86" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_8 python3_9" RUBY_TARGETS="ruby27 ruby30" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"                                                                                                                                          Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-10-26 04:28:53 UTC
COuld you please share the log of e.g. bind failing to build, and ideally the build log from openssl too? (ebuild ... clean install will help you do this).
Comment 2 Janpieter Sollie 2021-10-26 04:53:50 UTC
(In reply to Sam James from comment #1)
> COuld you please share the log of e.g. bind failing to build, and ideally
> the build log from openssl too? (ebuild ... clean install will help you do
> this).

do you want the openssl native build or cross compile build log?

anyway, I found a workaround for this issue:
1. add the following code to /usr/include/openssl/opensslconf.h: 
> #elif defined(__aarch64__) 
> # include <aarch64-unknown-linux-gnueabi/openssl/opensslconf.h> 
> 2. create a symbolic link to aarch64 includes: 
> ln -s /usr/aarch64-unknown-linux-gnueabi/include /usr/include/aarch64-unknown-linux-gnueabi
but bind still doesn't compile correctly, dlz loading causes issues, but that's probably bind/dlz and not gentoo related
I'll post the bind emerge logs
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-10-26 04:58:34 UTC
(In reply to Janpieter Sollie from comment #2)
> (In reply to Sam James from comment #1)
> > COuld you please share the log of e.g. bind failing to build, and ideally
> > the build log from openssl too? (ebuild ... clean install will help you do
> > this).
> 
> do you want the openssl native build or cross compile build log?

cross please if you can?

> 
> anyway, I found a workaround for this issue:
> 1. add the following code to /usr/include/openssl/opensslconf.h: 
> > #elif defined(__aarch64__) 
> > # include <aarch64-unknown-linux-gnueabi/openssl/opensslconf.h> 
> > 2. create a symbolic link to aarch64 includes: 
> > ln -s /usr/aarch64-unknown-linux-gnueabi/include /usr/include/aarch64-unknown-linux-gnueabi
> but bind still doesn't compile correctly, dlz loading causes issues, but
> that's probably bind/dlz and not gentoo related
> I'll post the bind emerge logs

Thanks for the help so far.
Comment 4 Janpieter Sollie 2021-10-26 04:58:58 UTC
Created attachment 746730 [details]
build log of aarch64 bind compile
Comment 5 Janpieter Sollie 2021-10-26 04:58:59 UTC
Created attachment 746733 [details]
build log of aarch64 bind compile
Comment 6 Tee KOBAYASHI 2021-10-26 05:04:29 UTC
Seems like multilib is somehow enabled, which should not be by default for crossdev.

The output of the command

aarch64-unknown-linux-gnueabi-emerge --info

might help us also.
Comment 7 Janpieter Sollie 2021-10-26 05:34:02 UTC
Created attachment 746736 [details]
build.log of aarch64 openssl compile

build.log of aarch64 openssl
Comment 8 Janpieter Sollie 2021-10-26 05:36:04 UTC
Created attachment 746739 [details]
cross-emerge --target aarch4-unknown-linux-gnueabi --info
Comment 9 Janpieter Sollie 2021-10-26 05:54:21 UTC
(In reply to Tee KOBAYASHI from comment #6)
> Seems like multilib is somehow enabled, which should not be by default for
> crossdev.

Could it be related to using "embedded" profile instead of default/arm64 in portage?
Comment 10 Tee KOBAYASHI 2021-10-26 06:37:23 UTC
Ah, I was wrong. The multilib in crossdev is not enabled in the first place. Never mind.

The problem seems to be that when cross-emerging net-dns/bind etc. /usr/include/openssl/opensslconf.h is read where /usr/aarch64-unknown-linux-gnueabi/usr/include/openssl/opensslconf.h should be.
Comment 11 Tee KOBAYASHI 2021-10-26 06:42:00 UTC
Looks similar to what media-gfx/povray suffers from for dependency on dev-libs/boost: https://bugs.gentoo.org/810227
Comment 12 Janpieter Sollie 2021-10-26 07:18:56 UTC
seems like for DLZ with berkdb, the same issue occurs in /usr/include/db.h (with the same workaround described above), but not sure whether that's another bug or the same root cause
Comment 13 James Le Cuirot gentoo-dev 2021-10-26 12:02:18 UTC
Correct, Bind's contrib/dlz/config.dlz.in is hardcoded to look in /usr/include, among other places, which is very bad. It should use pkg-config instead, or at least try that first. I don't maintain or even use Bind though, so I'll leave this to someone else.
Comment 14 Tee KOBAYASHI 2021-10-26 12:34:20 UTC
As for net-dns/bind[berkdb], we could use the configure argument --with-dlz-bdb=PATH. For prefix systems --with-dlz-bdb="${EPREFIX}"/usr would be sufficient.

The problem is that this does not work correctly for crossdev environments, because "${EPREFIX}" is expanded to an empty string. Instead we need to pass as PATH something like "${EPREFIX}"/usr/${CTARGET}/usr (e.g. /usr/aarch64-unknown-linux-gnueabi/usr).
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-10-26 12:37:29 UTC
(In reply to Tee KOBAYASHI from comment #14)
> As for net-dns/bind[berkdb], we could use the configure argument
> --with-dlz-bdb=PATH. For prefix systems --with-dlz-bdb="${EPREFIX}"/usr
> would be sufficient.
> 
> The problem is that this does not work correctly for crossdev environments,
> because "${EPREFIX}" is expanded to an empty string. Instead we need to pass
> as PATH something like "${EPREFIX}"/usr/${CTARGET}/usr (e.g.
> /usr/aarch64-unknown-linux-gnueabi/usr).

You probably want ESYSROOT for cross.
Comment 16 Tee KOBAYASHI 2021-10-26 13:00:56 UTC
Created attachment 746796 [details, diff]
net-dns/bind-9.16.21: Use ESYSROOT. Specify dlz-bdb path.

So this patch for ebuild of net-dns/bind might work.
Comment 17 James Le Cuirot gentoo-dev 2021-10-26 13:09:33 UTC
Don't change the --prefix path, that's used for installation.
Comment 18 Tee KOBAYASHI 2021-10-26 13:14:20 UTC
Created attachment 746799 [details, diff]
(revised) net-dns/bind-9.16.21: Use ESYSROOT. Specify dlz-bdb path.

Is this OK?
Comment 19 James Le Cuirot gentoo-dev 2021-10-26 13:27:28 UTC
Yep. I'm not sure about the effect on --with-openssl, but it's probably correct. I'd like the reporter to test this before we merge it.
Comment 20 Tee KOBAYASHI 2021-10-26 13:55:03 UTC
FWIW, the following packages (found by grepping with 'ssl.*EPREFIX') may suffer from the same issue:

app-admin/apache-tools
app-benchmarks/siege
app-crypt/heimdal
dev-db/freetds
dev-java/tomcat-native
dev-lang/erlang
dev-lang/php
dev-perl/DBD-mysql
dev-tcltk/tls
mail-client/alpine
net-dns/bind-tools
net-dns/bind
net-dns/unbound
net-ftp/lftp
net-im/pidgin
net-irc/unrealircd
net-libs/ldns
net-libs/neon
net-mail/fetchmail
net-misc/curl
net-misc/stunnel
www-client/elinks
www-client/lynx
www-client/w3mmee
Comment 21 Mike Gilbert gentoo-dev 2021-10-26 15:43:42 UTC
(In reply to Tee KOBAYASHI from comment #20)

We will probably need separate bugs for any affected packages. Or at least 1 bug per maintainer.

Alternatively, if someone could put a pull request together to fix the issue in all affected packages, we could probably get maintainers to sign off on them via Github.
Comment 22 Tee KOBAYASHI 2021-10-26 16:23:52 UTC
Yeah. And we will need a clever way to do that. Simply substituting ${ESYSROOT} for ${EPREFIX} does not work, as shown above.
Comment 23 Larry the Git Cow gentoo-dev 2021-11-10 05:40:33 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=209d2266879aa701b67812f45aa35ed067e0ff21

commit 209d2266879aa701b67812f45aa35ed067e0ff21
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2021-11-10 05:37:18 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2021-11-10 05:37:18 +0000

    net-dns/bind: fix cross-compilation for openssl and berkdb
    
    Closes: https://bugs.gentoo.org/820284
    Thanks-to: Tee KOBAYASHI <xtkoba@gmail.com>
    Signed-off-by: Sam James <sam@gentoo.org>

 net-dns/bind/bind-9.16.22.ebuild | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Additionally, it has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4d8bb09a33ef4eb06eaa82d73db423fd76654c83

commit 4d8bb09a33ef4eb06eaa82d73db423fd76654c83
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2021-11-10 05:39:34 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2021-11-10 05:40:06 +0000

    net-dns/bind-tools: fix cross compilation for libidn
    
    Bug: https://bugs.gentoo.org/820284
    Closes: https://bugs.gentoo.org/417131
    Thanks-to: Bertrand Jacquin <bertrand@jacquin.bzh>
    Signed-off-by: Sam James <sam@gentoo.org>

 net-dns/bind-tools/bind-tools-9.16.22.ebuild | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)