Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 588330

Summary: Cross compiling ebuilds inheriting multilib fail for mingw32
Product: Gentoo Linux Reporter: Gerhard Bräunlich <g.braeunlich>
Component: EclassesAssignee: Multilib team <multilib+disabled>
Status: RESOLVED FIXED    
Severity: normal CC: toolchain
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=432620
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: Fixes the issue
New patch
New get_exeext function for multilib.eclass
Extension check for multilib executables
List of executables

Description Gerhard Bräunlich 2016-07-08 13:04:50 UTC
When cross compiling an ebuild, emerge (multilib-build.eclass) will try to move each file specified in MULTILIB_CHOST_TOOLS. When using mingw32, executables get the extension ".exe" and the mv command will fail (searching for the file without the extension).

There should be a new profile dependent variable for the executable extension, which should be appended to the files in MULTILIB_CHOST_TOOLS.
Comment 1 Gerhard Bräunlich 2016-07-09 19:19:37 UTC
Created attachment 440218 [details, diff]
Fixes the issue
Comment 2 Gerhard Bräunlich 2016-07-14 08:00:34 UTC
Created attachment 440656 [details, diff]
New patch

Some ebuilds produce executable scripts which do not get an extra exe extension.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-07-14 08:24:44 UTC
I'm afraid this is a non-trivial problem. Grepping config.log can't really work since the eclass supports different build systems. I suspect we could use hardcoded suffixes per-CHOST alike get_libname from multilib.eclass, with the additional check for suffix-less files as you propose.

@toolchain, any advice?
Comment 4 Gerhard Bräunlich 2016-07-15 15:04:01 UTC
Created attachment 440768 [details, diff]
New get_exeext function for multilib.eclass

OK. So here is my next proposal.
Comment 5 Gerhard Bräunlich 2016-07-15 15:04:52 UTC
Created attachment 440772 [details, diff]
Extension check for multilib executables
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-07-20 20:49:31 UTC
Gerhard, thanks for your patch. After some thinking, I'd like to gather some more data. Could you possibly answer the following questions:

1. What is the case of .exe vs no-.exe? Are there any native program executables that do not get .exe, or are those only scripts? Does mingw support running Windows executables with no suffix?

2. Is there any case when scripts get different suffix on mingw? For example, is there at least a theoretical case for having 'foo' installed as 'foo.py' or 'foo.sh' on mingw?

3. Is there a possible case where both variants would be installed, i.e. both 'foo' and 'foo.exe'? Or some special fs magic that would cause 'foo' to match 'foo.exe' implicitly? (I guess cygwin is capable of doing random tricks like that)


What I'm thinking is... I think that we can reliably predict which of MULTILIB_CHOST_TOOLS would actually need the potential suffix. If that is true, I'd rather not introduce any guessing logic in the eclass just asking to be proven wrong, and instead add $(get_EXEEXT) to MULTILIB_CHOST_TOOLS whenever appropriate (note that this requires moving it from global scope to e.g. src_install()). In other words, something like this:

  src_install() {
    local MULTILIB_CHOST_TOOLS=( /usr/bin/foo$(get_EXEEXT) )
    multilib-minimal_src_install
  }

Do you think it would work fine? I know it's more work but I think that'd be much more correct.
Comment 7 Gerhard Bräunlich 2016-07-21 09:19:41 UTC
I created a list of all my cross compiled executables (in my case this is gtk+ with all its dependencies).

According to my list:

1. There is one case of a native program executable with no ".exe" extension:
dev-libs/expat-2.1.1-r2: /usr/bin/xmlwf
I also discovered, that due to my patch, executables loose their extension when they are moved by multilib-build.eclass (fn is set before f gets the extension).

2. In my list, there is no such case.

3. That did not happen in my case.

Does your suggestion mean, that we would have to change all ebuilds setting the variable MULTILIB_CHOST_TOOLS? Or why are you specifying the concrete executable "/usr/bin/foo"?
Comment 8 Gerhard Bräunlich 2016-07-21 09:20:36 UTC
Created attachment 441294 [details]
List of executables

This is my list of cross compiled executables.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-07-21 10:14:10 UTC
Yes, exactly. Change the ebuilds where .exe suffix is actually used.
Comment 10 Gerhard Bräunlich 2016-07-21 12:56:03 UTC
OK. That is what I did in the first place in my overlay. This will definitely work and I think it would be the cleaner solution.
But what I am afraid of is that the ebuild maintainers won't notice / forget / do not care about the get_EXEEXT feature and won't implement it and it would get very nasty to remember them over and over.
Or who will be responsible for the change in the ebuilds?
What do you think?
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-07-21 13:41:48 UTC
Remember that it's not just about MULTILIB_CHOST_TOOLS. Unless I'm mistaken, any call to ebuild helpers or pretty much any system tool will require specifying the suffix. It's only consistent to require it everywhere rather than having a subset of tools that are implicitly aware of the problem.

Well, cross-compilation is a similar deal as Prefix. People are not expected to understand it or care for it -- after all, covering the whole range of systems is non-trivial and often requires many fixes.

So, pretty much people who want cross-compilation working need to provide necessary fixes.

I'll send get_EXEEXT for review and try to get it for you today. Then we can decide how to proceed next. Probably a pull request would be fastest, and if it doesn't break anything, we should be able to push it through easily.
Comment 12 SpanKY gentoo-dev 2016-07-21 15:58:32 UTC
the context of bug 432620 (i.e. PMS) should be considered first before we start expanding eclasses
Comment 13 Gerhard Bräunlich 2016-07-21 18:20:15 UTC
OK. So you are suggesting, that something like a get_EXEEXT function should go to ebuild.sh instead of some eclass?
Comment 14 Gerhard Bräunlich 2016-08-22 11:38:05 UTC
Any news here?
If I can help somehow, please let me know!
Comment 15 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2016-09-03 11:07:24 UTC
commit 9f670e5d23d09556e3b52916c896e2ea32508981
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: Thu Jul 21 17:17:14 2016
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: Sat Sep 3 13:05:53 2016

    multilib.eclass: get_exeext for mingw/cygwin exe suffix, #588330
    
    Add a get_exeext function that can be used to obtain executable program
    suffix specific to the platform, in line with get_libname and
    get_modname. It is necessary to correctly reference built programs on
    non-standard platforms where executables use some suffix (like .exe).
    
    Original author: Gerhard Bräunlich
Comment 16 Patrice Clement gentoo-dev 2016-10-15 12:50:59 UTC
commit 9388d6df383bb9c81d685e70af45378925e1dd3a (HEAD -> master, origin/master, origin/HEAD)
Author:     Gerhard Bräunlich <wippbox@gmx.net>
AuthorDate: Fri Oct 14 17:49:26 2016 +0200
Commit:     Patrice Clement <monsieurp@gentoo.org>
CommitDate: Sat Oct 15 14:50:23 2016 +0200

dev-libs/glib: Adding $(get_exeext) to MULTILIB_CHOST_TOOLS.

Gentoo-Bug: https://bugs.gentoo.org/588330

Package-Manager: portage-2.3.0
Closes: https://github.com/gentoo/gentoo/pull/2557

Signed-off-by: Patrice Clement <monsieurp@gentoo.org>

dev-libs/glib/glib-2.48.2.ebuild | 2 +-
dev-libs/glib/glib-2.50.0.ebuild | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

Merged, thanks!
Comment 17 Michael Huss 2019-02-21 03:02:13 UTC
app-arch/bzip2 is affected. I had to add $(get_exeext) to each of the dobin lines in multilib_src_install (but NOT in multilib_src_install_all) to get it to merge.