This is a feature request to consider adding cross-compiling support for Emacs versions 27 and later. Cross-compiling Emacs versions prior to 27 led to a bad experience because it couldn't execute the binary on the build system to call unexec to generate the final program. Since version 27 implemented the portable dumping feature which supports reading/writing a separate file, it is possible to cross-compile a fully functional Emacs that allows the user to generate the dump file at runtime. Reproducible: Always I linked a URL to the cross-compiling process I proposed upstream a while ago, but I have not been able to invest the time in implementing things in the preferred manner enumerated in the response. I'm opening this bug thinking it might be feasible for the Gentoo maintainers to support my workaroundish process until an upstreamable solution is implemented. I'd propose the following changes to the ebuild: - Add IUSE="+dump" that replaces "--with-dumping=pdumper" with "--with-dumping=none --with-pdumper" when unset. (This builds the portable dumping functionality without trying to execute it at build time.) - When $CHOST is not $CBUILD, configure the source to natively build the uninstalled helper programs and Emacs to byte-compile lisp files. (This works around the build system not differentiating native tools from cross tools.) - When $CHOST is not $CBUILD, cross-compile the uninstalled programs, then overwrite them with the native versions, then proceed with the build as normal. (This keeps Make timestamps in order, so the helper programs are executable during the cross-build.) Here is a prototype ebuild patch I've been applying for a few months. I use it to cross-compile to six architectures daily with no issues. My pdump files are generated from either a QEMU-user call on the build system or a boot-time service on the target system, and everything works. --- app-editors/emacs/emacs-27.0.90.ebuild +++ app-editors/emacs/emacs-27.0.90.ebuild @@ -250,6 +250,11 @@ src_configure() { myconf+=" --without-x --without-ns" fi + CFLAGS= CPPFLAGS= LDFLAGS= ./configure --without-all --without-x-toolkit + make -j$(nproc) lisp {C,CPP,LD}FLAGS= + make -C lib-src -j$(nproc) blessmail {C,CPP,LD}FLAGS= + mv lib-src/make-docfile{,.save} ; mv lib-src/make-fingerprint{,.save} + make clean econf \ --program-suffix="-${EMACS_SUFFIX}" \ --includedir="${EPREFIX}"/usr/include/${EMACS_SUFFIX} \ @@ -259,7 +264,7 @@ src_configure() { --without-compress-install \ --without-hesiod \ --without-pop \ - --with-dumping=pdumper \ + --with-dumping=none --with-pdumper \ --with-file-notification=$(usev inotify || usev gfile || echo no) \ $(use_enable acl) \ $(use_with dbus) \ @@ -279,6 +284,9 @@ src_configure() { $(use_with wide-int) \ $(use_with zlib) \ ${myconf} + emake -C lib all + mv lib-src/make-docfile{.save,} ; mv lib-src/make-fingerprint{.save,} + touch lib-src/make-{docfile,fingerprint} } #src_compile() {
Nice work! This seems like the right approach. I use Emacs myself so I'll give this a go soon.
I guess that it won't be possible to move these emake calls from src_configure to src_compile?
The lines after econf could be moved to src_compile, but that function isn't defined in the ebuild, so I wrote it like that to keep a smaller diff. I don't know if anything can be done about the lines before econf, though, since it needs to make the lisp target with a native configuration. (The two native binary programs it needs to save could theoretically be built in a separately configured directory.) I don't know if this procedure can be expressed in a maintainable Gentoo-approved way.
Created attachment 674554 [details, diff] Support cross-compilation I made an attempt at ordering things in a way that might be acceptable for Gentoo. This patch skips dumping the executable when cross-compiling rather than adding another USE flag. (I'm guessing EXTRA_ECONF is good enough for toggling that.) Maybe it could warn/info about manually running dump-emacs-portable at runtime in pkg_postinst.
(In reply to David Michael from comment #4) > Created attachment 674554 [details, diff] [details, diff] > Support cross-compilation > > I made an attempt at ordering things in a way that might be acceptable for > Gentoo. This patch skips dumping the executable when cross-compiling rather > than adding another USE flag. (I'm guessing EXTRA_ECONF is good enough for > toggling that.) Maybe it could warn/info about manually running > dump-emacs-portable at runtime in pkg_postinst. Two small comments: - Please avoid "cd -" but use pushd/popd (as in the example in the econf_build documentation). - Adding --with-pdumper to the global econf options (just before --with-dumping=pdumper) might be clearer.
Created attachment 674557 [details, diff] Support cross-compilation Okay, done. (I put --with-pdumper at the end since it looks like it's sorted alphabetically.)
I don't fully understand the changes but it certainly works! (it didn't before) > This is GNU Emacs, one component of the GNU/Linux operating system. > > GNU Emacs 27.1 (build 1, m68k-unknown-linux-gnu) > of 2020-11-23 > Copyright (C) 2020 Free Software Foundation, Inc.
The changes do two things: - Disable dumping the binary at build time, because it requires executing the final emacs binary which is probably incompatible with the build system architecture. The dump file can be generated at runtime with `emacs --batch --eval='(dump-emacs-portable "/usr/libexec/emacs/27.1/$CHOST/emacs.pdmp")'`. (I just include a service to run that on boot if it doesn't exist.) - Natively compile three tools that need to run at build time (make-docfile, make-fingerprint, and emacs as the lisp compiler), then compile lisp files. This stuff is built with a weird ordering so that the cross-compiled dependencies are older than the natively built tools so that Make can execute them without triggering rebuilds of anything. Unfortunately the Emacs build system doesn't understand different CBUILD/CHOST compilers, so it has to be juggled this way.
I was mainly meaning why this bit is now able to go away? > # Disable sandbox when dumping. For the unbelievers, see bug #131505 > # emake RUN_TEMACS="SANDBOX_ON=0 LD_PRELOAD= env ./temacs"
It's a comment that was not being run, and according to bug #700182 it's not relevant anymore, so I just dropped it. If there is any value in keeping the comment, it can be restored.
Oh, haha, didn't even notice it was commented.
One more thing (sorry but I notice it just now): "cp -t" is a GNUism and should be avoided; instead, simply put the target directory last. Plus, that line is missing a "|| die". I was also wondering what is special about blessmail that it needs to be listed as an explicit make target? We don't even install that file. (In reply to David Michael from comment #4) > [...] Maybe it could warn/info about manually running dump-emacs-portable > at runtime in pkg_postinst. Could you compose an appropriate message? It should be added to DOC_CONTENTS in src_install. Last, we need a copyright signoff for your changes. Either git format-patch it and make sure that the commit contains a "Signed-off-by: Name <e-mail>" line, or just post that line as a comment to this bug. See https://www.gentoo.org/glep/glep-0076.html#certificate-of-origin for details.
(In reply to Ulrich Müller from comment #12) > I was also wondering what is special about blessmail that it needs to be > listed as an explicit make target? We don't even install that file. In fact, that's a bug which has been there for many years. movemail must belong to the mail group and its setgid bit must be set (which can be done by running blessmail). I guess it wasn't reported because nobody uses movemail (or a local mail spool) any more. I've fixed this now, and I've also added an empty BLESSMAIL_TARGET to emake install. Could you test if things still work when you remove blessmail from emake in src_complie?
Created attachment 675208 [details, diff] Support cross-compilation Yes, it works without when BLESSMAIL_TARGET is empty.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=42fbfc55f945efef8291022f02051373df7a4cc5 commit 42fbfc55f945efef8291022f02051373df7a4cc5 Author: David Michael <fedora.dm0@gmail.com> AuthorDate: 2020-11-26 22:25:57 +0000 Commit: Ulrich Müller <ulm@gentoo.org> CommitDate: 2020-11-28 12:19:22 +0000 app-editors/emacs: Support cross-compiling since Emacs 27 Closes: https://bugs.gentoo.org/718760 Package-Manager: Portage-3.0.9, Repoman-3.0.2 Signed-off-by: David Michael <fedora.dm0@gmail.com> [Revision bump, minor formatting tweaks] Signed-off-by: Ulrich Müller <ulm@gentoo.org> app-editors/emacs/emacs-27.1-r3.ebuild | 459 ++++++++++++++++++++++++++++ app-editors/emacs/emacs-27.1.9999-r1.ebuild | 37 ++- app-editors/emacs/emacs-28.0.9999.ebuild | 37 ++- 3 files changed, 521 insertions(+), 12 deletions(-)
Thank you very much for the patch. It is a clean and unobtrusive solution.
(In reply to David Michael from comment #14) > Created attachment 675208 [details, diff] [details, diff] > Support cross-compilation + tc-is-cross-compiler && DOC_CONTENTS+="\\n\\nEmacs did not write a portable + dump file due to being cross-compiled. To create this file at run + time, execute the following command:\\nemacs + --batch --eval='(dump-emacs-portable + \"/usr/libexec/emacs/${FULL_VERSION}/${CHOST}/emacs.pdmp\")'" I stumbled upon this command line today. Maybe we should add -Q to the options?
Sounds okay to me. I tried it, and it works fine.
