Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 689598 - dev-lang/lua-5.1.5-r5 changes
Summary: dev-lang/lua-5.1.5-r5 changes
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: William Hubbs
URL:
Whiteboard:
Keywords: EBUILD, PATCH
Depends on:
Blocks:
 
Reported: 2019-07-10 15:49 UTC by Vince C.
Modified: 2019-07-14 10:32 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
New ebuild for lua as dev-lang/lua-5.1.5-r5 (lua-5.1.5-r5.ebuild,3.03 KB, text/plain)
2019-07-10 15:52 UTC, Vince C.
Details
Gentoo build patch for makefiles (lua-5.1.5-gentoo-build.patch,2.69 KB, patch)
2019-07-10 15:53 UTC, Vince C.
Details | Diff
Reworked/consolidated version of the deprecated symbols removal patches (lua-5.1.5-disable-deprecated.patch,1.78 KB, patch)
2019-07-10 15:55 UTC, Vince C.
Details | Diff
Reworked version for disabling readline (lua-5.1.5-disable-readline.patch,638 bytes, patch)
2019-07-10 15:58 UTC, Vince C.
Details | Diff
Reworked version for varargs calls fix (lua-5.1.5-fix_vararg_calls.patch,450 bytes, patch)
2019-07-10 16:01 UTC, Vince C.
Details | Diff
Updated ebuild for dev-lang/lua-5.1.5-r5 with 32/64 ABI (lua-5.1.5-r5.ebuild,3.06 KB, text/plain)
2019-07-12 10:22 UTC, Vince C.
Details
Gentoo build patch for makefiles, updated for 32/64 ABI (lua-5.1.5-gentoo-build.patch,3.02 KB, patch)
2019-07-12 10:27 UTC, Vince C.
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vince C. 2019-07-10 15:49:39 UTC
Massive refactoring of lua 5.1 build and installation. Since cross-compiling fails, this ebuild is a revision bump (r4 -> r5) with fixes that allow at least cross compiling for the ARM platform. This ebuild comes with a complete set of patches plus EAPI version 6 compliance.

Reproducible: Always




This ebuild completely removes libtool dependency as the latter reportedly causes cross compilation to fail. The build process follows upstream as much as possible while mimicking libtool build phases as for building the shared library.

Changelog:

 * EAPI=6
 * Removal of libtool dependency
 * USE=static removed (static build by default)
 * interpreter and compiler now statically linked to the library modules
 * liblua.so symbolic link removed (not required as per -soname)
 * Documentation conditional with USE=doc
 * flag-o-matic eclass added as a dependency
 * new patch set with previous patches merged/reworked:
   - lua-5.1.5-disable-deprecated.patch
   - lua-5.1.5-disable-readline.patch
   - lua-5.1.5-gentoo-build.patch
   - lua-5.1.5-fix_vararg_calls.patch (heading reworked)

Building this package/revision has been successful on the following architectures:

 * x86_64-pc-linux-gnu, native
 * armv7a-hardfloat-linux-gnueabi, cross-compiled (Raspberry Pi)

No change have been made to the architecture keywords so further testing is required.
Comment 1 Vince C. 2019-07-10 15:52:11 UTC
Created attachment 582368 [details]
New ebuild for lua as dev-lang/lua-5.1.5-r5
Comment 2 Vince C. 2019-07-10 15:53:53 UTC
Created attachment 582374 [details, diff]
Gentoo build patch for makefiles

This patch removes libtool dependency and adds the required details for successful cross-compilation.
Comment 3 Vince C. 2019-07-10 15:55:52 UTC
Created attachment 582376 [details, diff]
Reworked/consolidated version of the deprecated symbols removal patches
Comment 4 Vince C. 2019-07-10 15:58:41 UTC
Created attachment 582378 [details, diff]
Reworked version for disabling readline
Comment 5 Vince C. 2019-07-10 16:01:28 UTC
Created attachment 582380 [details, diff]
Reworked version for varargs calls fix

This patch is a replacement candidate for the one coming with revision 4
Comment 6 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2019-07-11 18:38:56 UTC
1. It fails to build here. amd64 ABI_X86="32 (64)":
x86_64-pc-linux-gnu-gcc -m32-ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o	# DLL needs all object files
x86_64-pc-linux-gnu-gcc: error: rcu: No such file or directory
x86_64-pc-linux-gnu-gcc: error: liblua.a: No such file or directory
x86_64-pc-linux-gnu-gcc: error: unrecognized command line option ‘-m32-ar’

The 64-bit variant DOES build fine; just amd64+32-bit fails.

2. The slotted-lua work needs to be integrated to this update.

3. Please restore the optional USE=static; it still saves 160KiB for uses that care about raw space usage.
Comment 7 Vince C. 2019-07-11 23:46:37 UTC
(In reply to Robin Johnson from comment #6)
> 1. It fails to build here. amd64 ABI_X86="32 (64)":
> x86_64-pc-linux-gnu-gcc -m32-ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o
> ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o
> lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o
> ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o
> # DLL needs all object files
> x86_64-pc-linux-gnu-gcc: error: rcu: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: liblua.a: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: unrecognized command line option ‘-m32-ar’
> 
> The 64-bit variant DOES build fine; just amd64+32-bit fails.

I think I see why it fails to compile. Is the compiler variable, $CC, passed by the ebuild to the make process as "x86_64-pc-linux-gnu-gcc -m32" ? If yes, then I'm somehow screwed for I relied upon the compiler executable name to be passed *only* as the binary executable name with no argument (e.g. x86_64-pc-linux-gnu-gcc) to compute the other executable names, such as $AR or $RANLIB (instead of hard-coding them in the makefile patch). And if yes, shouldn't the "-m32" passed through the $CFLAGS variable, which would be quite logical? (I'm trying to [make sure I] understand, it's no criticism on my part.)


> 2. The slotted-lua work needs to be integrated to this update.

What do you mean "slotted"? Is it about lua versions 5.2 and 5.3? Or the 32/64 ABI as stated above?


> 3. Please restore the optional USE=static; it still saves 160KiB for uses
> that care about raw space usage.

Well, I removed it first off because it was *immensely* simpler for me (I spent about one whole week on that ebuild to come up with that) — I still don't know so far how to link an executable against a dynamic library. Then also note that on Debian both the interpreter and compiler are statically linked to LUA library modules, performance-wise, as per upstream specification, I suppose.

So I'm back to the drawing board then... I'll keep you posted.
Comment 8 Vince C. 2019-07-12 10:22:08 UTC
Created attachment 582602 [details]
Updated ebuild for dev-lang/lua-5.1.5-r5 with 32/64 ABI

This updated ebuild now compiles and installs with respect to 32/64 ABI and keeps ARM cross-compiling successful. Successfully tested on an x86_64 arch (both ABI and ARM cross-compiled). It needs the updated gentoo-build patch.
Comment 9 Vince C. 2019-07-12 10:27:42 UTC
Created attachment 582604 [details, diff]
Gentoo build patch for makefiles, updated for 32/64 ABI

Makefiles shall no longer "compute" the tool names as they're exported from the ebuild with "tc-export", which this patch reflects. Note that makefiles were tweaked so as to be as little intrusive as possible, i.e. to be compatible with the initial makefiles when they are run outside of portage.
Comment 10 Vince C. 2019-07-12 10:36:51 UTC
(In reply to Robin Johnson from comment #6)
> 1. It fails to build here. amd64 ABI_X86="32 (64)":
> x86_64-pc-linux-gnu-gcc -m32-ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o
> ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o
> lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o
> ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o
> # DLL needs all object files
> x86_64-pc-linux-gnu-gcc: error: rcu: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: liblua.a: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: unrecognized command line option ‘-m32-ar’
> 
> The 64-bit variant DOES build fine; just amd64+32-bit fails.

Here's a version that compiles and is fully functional. At least now there should be no more compile/install error.

> 2. The slotted-lua work needs to be integrated to this update.

I still need to be sure what you mean with slotted so, please, could you clarify/confirm ?

> 3. Please restore the optional USE=static; it still saves 160KiB for uses
> that care about raw space usage.

That is still work-in-progress...
Comment 11 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2019-07-12 18:53:17 UTC
(In reply to Vince C. from comment #10)
> (In reply to Robin Johnson from comment #6)
> > 1. It fails to build here. amd64 ABI_X86="32 (64)":
...
> > The 64-bit variant DOES build fine; just amd64+32-bit fails.
> Here's a version that compiles and is fully functional. At least now there
> should be no more compile/install error.
Thanks, I confirm both 32+64 do build now, but there's other issues, see below.

> > 2. The slotted-lua work needs to be integrated to this update.
> I still need to be sure what you mean with slotted so, please, could you
> clarify/confirm ?
See lua-5.1.5-r10x series. The library generated is liblua5.1.so, with matching binaries (/usr/bin/lua5.1) and pkgconfig. The 5.2/5.3 stuff also generates similar numbered non-conflicting libraries & binaries.

> > 3. Please restore the optional USE=static; it still saves 160KiB for uses
> > that care about raw space usage.
> That is still work-in-progress...
Ok.

Two breakages I see from the old series vs the one ebuild:
4. * QA Notice: Files built without respecting LDFLAGS have been detected
 *  Please include the following list of files in your report:
 * /usr/bin/lua
 * /usr/bin/luac
 * /usr/lib32/liblua.so.5.1.5
 * /usr/lib64/liblua.so.5.1.5

5. "emake -j1" => there was a minimal fix to the upstream Makefiles to support parallel build properly. It should be preserved.
Comment 12 Vince C. 2019-07-14 09:14:30 UTC
(In reply to Robin Johnson from comment #11)

> 5. "emake -j1" => there was a minimal fix to the upstream Makefiles to
> support parallel build properly. It should be preserved.

Oops! The "-j1" was an oversight, I forgot to remove it before submitting. It merely was to serialize the build phase otherwise I'd get confused. I'll remove it.


> > > 2. The slotted-lua work needs to be integrated to this update.
> > I still need to be sure what you mean with slotted so, please, could you
> > clarify/confirm ?
> See lua-5.1.5-r10x series. The library generated is liblua5.1.so, with
> matching binaries (/usr/bin/lua5.1) and pkgconfig. The 5.2/5.3 stuff also
> generates similar numbered non-conflicting libraries & binaries.

Ok, I'll see what I can do. It doesn't sound too much difficult with my level of understanding.


> Two breakages I see from the old series vs the one ebuild:
> 4. * QA Notice: Files built without respecting LDFLAGS have been detected
>  *  Please include the following list of files in your report:
>  * /usr/bin/lua
>  * /usr/bin/luac
>  * /usr/lib32/liblua.so.5.1.5
>  * /usr/lib64/liblua.so.5.1.5

This is interesting as I don't see that message on my system when emerging r5. I have absolutely no clue. What does it mean?

Anyway thanks for your guidance in this process. I really appreciate.
Comment 13 Vince C. 2019-07-14 10:32:18 UTC
(In reply to Robin Johnson from comment #11)
> Two breakages I see from the old series vs the one ebuild:
> 4. * QA Notice: Files built without respecting LDFLAGS have been detected
>  *  Please include the following list of files in your report:
>  * /usr/bin/lua
>  * /usr/bin/luac
>  * /usr/lib32/liblua.so.5.1.5
>  * /usr/lib64/liblua.so.5.1.5

Maybe I have a clue after all: in the ebuild I added that line:

    append-ldflags "-Wl,-E"

and it turns out it's nowhere to be seen in the build log, which means it's not used. And as a matter of fact, no target in the Makefiles use the default rules, in which case $(LDFLAGS) would have been pulled. I'll just remove that line from the ebuild as it only serves building the shared object and the previous released didn't include that linker flag. I'll transfer it to the SOFLAGS in the Makefile.

Did I get it right?

Now as for the slotted version, I do realize there's a bit of work ahead for what I see of eselect-lua, the installed include directories also include the version. I have questions about slotted packages:

 * does eselect-<package> alone manage all those symbolic links?
 * or can the ebuild install symbolic links to the risk of creating collisions?
 * or must the ebuild call eselect?