Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 680220

Summary: app-arch/bzip2 cannot be cross-emerged with mingw32
Product: Gentoo Linux Reporter: Michał Dec <moog621>
Component: Current packagesAssignee: Gentoo's Team for Core System packages <base-system>
Status: UNCONFIRMED ---    
Severity: normal CC: crossdev, h.mth, jj, prefix
Priority: Normal Keywords: PATCH
Version: unspecified   
Hardware: x86   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: partial log of emerge-wrapper --target i686-w64-mingw32 -1v dev-libs/boost
Patch bzip2-1.0.8-r1.ebuild
Patch bzip2-1.0.8-r1.ebuild

Description Michał Dec 2019-03-13 11:20:24 UTC
The package actually compiles. The failure occurs in installation phase. Since mingw toolchains produce .dll and .exe files, dobin gets confused when looking for bzip2recover. There is only bzip2recover.exe. The same applies to both i686-w64-mingw32 and x86_64-w64-mingw32 toolchains. I have configured both for the cygwin profile in Portage.
Comment 1 Michał Dec 2019-03-13 11:21:26 UTC
Created attachment 568974 [details]
partial log of emerge-wrapper --target i686-w64-mingw32 -1v dev-libs/boost
Comment 2 Michał Dec 2019-03-13 11:31:55 UTC
Proposed changes:
1. In function multilib_src_install:
Declare local variables bzip2recover_binary="bzip2recover" and bzip2_binary="bzip2". Next, if ${KERNEL} equals "Winnt", add ".exe" to these 2 variables. Next, call dobin on variable names instead of assigned variable names.

2. The rest of the names must also be changed. Notice the merging step:
>>> Merging app-arch/bzip2-1.0.6-r10 to /usr/i686-w64-mingw32/
--- /usr/i686-w64-mingw32/usr/
--- /usr/i686-w64-mingw32/usr/lib/
>>> /usr/i686-w64-mingw32/usr/lib/libbz2.so.1.0.6
>>> /usr/i686-w64-mingw32/usr/lib/libbz2.so -> libbz2.so.1.0.6
--- /usr/i686-w64-mingw32/usr/include/
>>> /usr/i686-w64-mingw32/usr/include/bzlib.h
--- /usr/i686-w64-mingw32/usr/bin/
>>> /usr/i686-w64-mingw32/usr/bin/bzdiff
>>> /usr/i686-w64-mingw32/usr/bin/bzmore
>>> /usr/i686-w64-mingw32/usr/bin/bzcmp -> bzdiff
>>> /usr/i686-w64-mingw32/usr/bin/bzip2recover.exe
>>> /usr/i686-w64-mingw32/usr/bin/bzgrep
>>> /usr/i686-w64-mingw32/usr/bin/bzfgrep -> bzgrep
>>> /usr/i686-w64-mingw32/bin/
>>> /usr/i686-w64-mingw32/bin/bzip2.exe
>>> /usr/i686-w64-mingw32/usr/bin/bzegrep -> bzgrep
>>> /usr/i686-w64-mingw32/usr/bin/bzless -> bzmore
>>> /usr/i686-w64-mingw32/usr/lib/libbz2.so.1 -> libbz2.so.1.0.6
>>> /usr/i686-w64-mingw32/usr/lib/libbz2.so.1.0 -> libbz2.so.1.0.6
 * QA Notice: Symbolic link /bin/bzcat points to /bin/bzip2 which does not exist.
>>> /usr/i686-w64-mingw32/bin/bzcat -> bzip2
 * QA Notice: Symbolic link /bin/bunzip2 points to /bin/bzip2 which does not exist.
>>> /usr/i686-w64-mingw32/bin/bunzip2 -> bzip2
>>> app-arch/bzip2-1.0.6-r10 merged.

Files that are symlinks need not be installed at all on Windows, and dynamic libraries need to be called libbz2.dll instead of libbz2.so.${gradual_version_numbers_and_then_eventually_full_version_number}
Comment 3 Michał Dec 2019-03-13 12:45:30 UTC
/usr/i686-w64-mingw32/bin does not exist. Files that go there (bzip2.exe, bzcat and bunzip2) should go to /usr/i686-w64-mingw32/usr/bin perhaps?
Comment 4 Michael Haubenwallner (RETIRED) gentoo-dev 2019-03-13 12:48:12 UTC
(In reply to moog621 from comment #0)
> I have configured both for the cygwin profile in Portage.

What exactly do you mean by that?

(In reply to moog621 from comment #1)
> partial log of emerge-wrapper --target i686-w64-mingw32 -1v dev-libs/boost

What is that 'emerge-wrapper' about?


Maybe this bug is not related to prefix but crossdev?
Comment 5 Michał Dec 2019-03-13 13:00:25 UTC
>What exactly do you mean by that?
/usr/i686-w64-mingw32/etc/portage/make.profile is a symlink to /usr/portage/profiles/prefix/windows/cygwin/x86
Likewise for the amd64 toolchain, the last directory pointed by the symlink is x64 instead of x86.

>What is that 'emerge-wrapper' about?
I was taught that cross-compiling packages requires the use of emerge-wrapper command. For each session:

emerge-wrapper --target your-crossdev-toolchain-here --init
emerge-wrapper --target your-crossdev-toolchain-here ${use_like_normal_portage}
# e.g. emerge-wrapper --target i686-w64-mingw32 -1v app-arch/bzip2

>Maybe this bug is not related to prefix but crossdev?
Not sure. I just checked sys-libs/zlib, which always installs fine under crossdev toolchains. Here are a couple of interesting pieces:

src_prepare() {
        epatch "${FILESDIR}"/${PN}-1.2.11-fix-deflateParams-usage.patch
        epatch "${FILESDIR}"/${PN}-1.2.11-minizip-drop-crypt-header.patch #658536

        if use minizip ; then
                cd contrib/minizip || die
                eautoreconf
        fi

        case ${CHOST} in
        *-mingw*|mingw*)
                # uses preconfigured Makefile rather than configure script
                multilib_copy_sources
                ;;
        esac
}

And next,

multilib_src_configure() {
        case ${CHOST} in
        *-mingw*|mingw*)
                ;;
        *)      # not an autoconf script, so can't use econf
                local uname=$("${EPREFIX}"/usr/share/gnuconfig/config.sub "${CHOST}" | cut -d- -f3) #347167
                echoit "${S}"/configure \
                        --shared \
                        --prefix="${EPREFIX}/usr" \
                        --libdir="${EPREFIX}/usr/$(get_libdir)" \
                        ${uname:+--uname=${uname}} \
                        || die
                ;;
        esac

The rest of the function has nothing to do with mingw. And next,

multilib_src_compile() {
        case ${CHOST} in
        *-mingw*|mingw*)
                emake -f win32/Makefile.gcc STRIP=true PREFIX=${CHOST}-
                sed \
                        -e 's|@prefix@|/usr|g' \
                        -e 's|@exec_prefix@|${prefix}|g' \
                        -e 's|@libdir@|${exec_prefix}/'$(get_libdir)'|g' \
                        -e 's|@sharedlibdir@|${exec_prefix}/'$(get_libdir)'|g' \
                        -e 's|@includedir@|${prefix}/include|g' \
                        -e 's|@VERSION@|'${PV}'|g' \
                        zlib.pc.in > zlib.pc || die
                ;;
        *)
                emake
                ;;
        esac

As per last time. Next,

multilib_src_install() {
        case ${CHOST} in
        *-mingw*|mingw*)
                emake -f win32/Makefile.gcc install \
                        BINARY_PATH="${ED}/usr/bin" \
                        LIBRARY_PATH="${ED}/usr/$(get_libdir)" \
                        INCLUDE_PATH="${ED}/usr/include" \
                        SHARED_MODE=1
                # overwrites zlib.pc created from win32/Makefile.gcc #620136
                insinto /usr/$(get_libdir)/pkgconfig
                doins zlib.pc
                ;;

        *)
                emake install DESTDIR="${D}" LDCONFIG=:
                gen_usr_ldscript -a z
                ;;
        esac

As per last time. This was the last mention of mingw-specific code in the build of sys-libs/zlib-1.2.11-r2. It doesn't contain any mention of Winnt or cygwin, so I guess the correct way is referring to mingw. From what I see, app-arch/bzip2-1.0.6-r10 contains only 1 reference to mingw and it's one of the patches, bzip2-1.0.6-mingw.patch that refers to Gentoo bug #393573. The patch is irrelevant for this bug because it only fixes compilation issues with the source code. This is a problem in the install and merge steps.
Comment 6 Michael Haubenwallner (RETIRED) gentoo-dev 2019-03-13 13:33:17 UTC
(In reply to moog621 from comment #5)
> >What exactly do you mean by that?
> /usr/i686-w64-mingw32/etc/portage/make.profile is a symlink to
> /usr/portage/profiles/prefix/windows/cygwin/x86

The prefix/* profiles are intended for *native* Prefix compilation (like prefix/windows/cygwin for *native Cygwin*), not sure about how useful they are with cross compiling.

> >What is that 'emerge-wrapper' about?
> I was taught that cross-compiling packages requires the use of
> emerge-wrapper command. For each session:
> 
> emerge-wrapper --target your-crossdev-toolchain-here --init

Does this command create above make.profile symlink?
To me that would feel strange at least.
Comment 7 Michał Dec 2019-03-13 14:26:51 UTC
>not sure about how useful they are with cross compiling
They do their job just fine for my use cases. These are compiling latest versions of ioquake3 and OpenJK from git for Windows x86 and amd64. I wanted to expand the scope of my CI setup by also building amd64 builds of VCMI. This requires dev-libs/boost which in turn requires app-arch/bzip2 and here I am. The only caveat is that after I'm done compiling I need to copy the .dll files to the distribution package. ioquake3 for example requires we copy OpenAL32.dll (x86) but for amd64 this library needs to be renamed to OpenAL64.dll in the destination directory. OpenAL emerges fine with emerge-wrapper for mingw :)

>Does this command create above make.profile symlink?
No. Not at all. The user has to adjust their toolchain's make.profile according to the toolchain's purpose. emerge-wrapper takes care of errors where your-toolchain-here-emerge would fail miserably, i.e. compiling something that depends on another library - emerge-wrapper will instruct all the tools that libraries for this toolchain live in /usr/${toolchain}/usr/lib and not /usr/lib. Not adhering to this will definitely break most packages as they, for example, try to link an arm object against amd64 libm.so.
Comment 8 Michael Haubenwallner (RETIRED) gentoo-dev 2019-03-14 06:44:49 UTC
Assigning to bzip2 maintainers then.
Comment 9 Hanno Zysik (geki) 2020-11-28 16:52:19 UTC
Created attachment 675607 [details, diff]
Patch bzip2-1.0.8-r1.ebuild

This is how it should be done accordingly to https://wiki.gentoo.org/wiki/Mingw. There are still some short-commings, like the shell scripts, though, the bzip2 exe and library are built and ebuild merges successfully.
Comment 10 Hanno Zysik (geki) 2020-11-28 18:05:21 UTC
Created attachment 675622 [details, diff]
Patch bzip2-1.0.8-r1.ebuild

Version2: Adjust Makefile-libbz2_so for right library extesion.
Comment 11 Mike Gilbert gentoo-dev 2023-02-09 20:08:36 UTC
Comment on attachment 675622 [details, diff]
Patch bzip2-1.0.8-r1.ebuild

Your patch incorrectly drops "inherit toolchain-funcs".