Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 689598

Summary: dev-lang/lua-5.1.5-r5 changes
Product: Gentoo Linux Reporter: Vince C. <vincent.cadet>
Component: Current packagesAssignee: William Hubbs <williamh>
Status: RESOLVED FIXED    
Severity: normal CC: robbat2
Priority: Normal Keywords: EBUILD, PATCH
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=627330
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: New ebuild for lua as dev-lang/lua-5.1.5-r5
Gentoo build patch for makefiles
Reworked/consolidated version of the deprecated symbols removal patches
Reworked version for disabling readline
Reworked version for varargs calls fix
Updated ebuild for dev-lang/lua-5.1.5-r5 with 32/64 ABI
Gentoo build patch for makefiles, updated for 32/64 ABI

Description Vince C. 2019-07-10 15:49:39 UTC
Massive refactoring of lua 5.1 build and installation. Since cross-compiling fails, this ebuild is a revision bump (r4 -> r5) with fixes that allow at least cross compiling for the ARM platform. This ebuild comes with a complete set of patches plus EAPI version 6 compliance.

Reproducible: Always




This ebuild completely removes libtool dependency as the latter reportedly causes cross compilation to fail. The build process follows upstream as much as possible while mimicking libtool build phases as for building the shared library.

Changelog:

 * EAPI=6
 * Removal of libtool dependency
 * USE=static removed (static build by default)
 * interpreter and compiler now statically linked to the library modules
 * liblua.so symbolic link removed (not required as per -soname)
 * Documentation conditional with USE=doc
 * flag-o-matic eclass added as a dependency
 * new patch set with previous patches merged/reworked:
   - lua-5.1.5-disable-deprecated.patch
   - lua-5.1.5-disable-readline.patch
   - lua-5.1.5-gentoo-build.patch
   - lua-5.1.5-fix_vararg_calls.patch (heading reworked)

Building this package/revision has been successful on the following architectures:

 * x86_64-pc-linux-gnu, native
 * armv7a-hardfloat-linux-gnueabi, cross-compiled (Raspberry Pi)

No change have been made to the architecture keywords so further testing is required.
Comment 1 Vince C. 2019-07-10 15:52:11 UTC
Created attachment 582368 [details]
New ebuild for lua as dev-lang/lua-5.1.5-r5
Comment 2 Vince C. 2019-07-10 15:53:53 UTC
Created attachment 582374 [details, diff]
Gentoo build patch for makefiles

This patch removes libtool dependency and adds the required details for successful cross-compilation.
Comment 3 Vince C. 2019-07-10 15:55:52 UTC
Created attachment 582376 [details, diff]
Reworked/consolidated version of the deprecated symbols removal patches
Comment 4 Vince C. 2019-07-10 15:58:41 UTC
Created attachment 582378 [details, diff]
Reworked version for disabling readline
Comment 5 Vince C. 2019-07-10 16:01:28 UTC
Created attachment 582380 [details, diff]
Reworked version for varargs calls fix

This patch is a replacement candidate for the one coming with revision 4
Comment 6 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2019-07-11 18:38:56 UTC
1. It fails to build here. amd64 ABI_X86="32 (64)":
x86_64-pc-linux-gnu-gcc -m32-ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o	# DLL needs all object files
x86_64-pc-linux-gnu-gcc: error: rcu: No such file or directory
x86_64-pc-linux-gnu-gcc: error: liblua.a: No such file or directory
x86_64-pc-linux-gnu-gcc: error: unrecognized command line option ‘-m32-ar’

The 64-bit variant DOES build fine; just amd64+32-bit fails.

2. The slotted-lua work needs to be integrated to this update.

3. Please restore the optional USE=static; it still saves 160KiB for uses that care about raw space usage.
Comment 7 Vince C. 2019-07-11 23:46:37 UTC
(In reply to Robin Johnson from comment #6)
> 1. It fails to build here. amd64 ABI_X86="32 (64)":
> x86_64-pc-linux-gnu-gcc -m32-ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o
> ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o
> lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o
> ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o
> # DLL needs all object files
> x86_64-pc-linux-gnu-gcc: error: rcu: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: liblua.a: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: unrecognized command line option ‘-m32-ar’
> 
> The 64-bit variant DOES build fine; just amd64+32-bit fails.

I think I see why it fails to compile. Is the compiler variable, $CC, passed by the ebuild to the make process as "x86_64-pc-linux-gnu-gcc -m32" ? If yes, then I'm somehow screwed for I relied upon the compiler executable name to be passed *only* as the binary executable name with no argument (e.g. x86_64-pc-linux-gnu-gcc) to compute the other executable names, such as $AR or $RANLIB (instead of hard-coding them in the makefile patch). And if yes, shouldn't the "-m32" passed through the $CFLAGS variable, which would be quite logical? (I'm trying to [make sure I] understand, it's no criticism on my part.)


> 2. The slotted-lua work needs to be integrated to this update.

What do you mean "slotted"? Is it about lua versions 5.2 and 5.3? Or the 32/64 ABI as stated above?


> 3. Please restore the optional USE=static; it still saves 160KiB for uses
> that care about raw space usage.

Well, I removed it first off because it was *immensely* simpler for me (I spent about one whole week on that ebuild to come up with that) — I still don't know so far how to link an executable against a dynamic library. Then also note that on Debian both the interpreter and compiler are statically linked to LUA library modules, performance-wise, as per upstream specification, I suppose.

So I'm back to the drawing board then... I'll keep you posted.
Comment 8 Vince C. 2019-07-12 10:22:08 UTC
Created attachment 582602 [details]
Updated ebuild for dev-lang/lua-5.1.5-r5 with 32/64 ABI

This updated ebuild now compiles and installs with respect to 32/64 ABI and keeps ARM cross-compiling successful. Successfully tested on an x86_64 arch (both ABI and ARM cross-compiled). It needs the updated gentoo-build patch.
Comment 9 Vince C. 2019-07-12 10:27:42 UTC
Created attachment 582604 [details, diff]
Gentoo build patch for makefiles, updated for 32/64 ABI

Makefiles shall no longer "compute" the tool names as they're exported from the ebuild with "tc-export", which this patch reflects. Note that makefiles were tweaked so as to be as little intrusive as possible, i.e. to be compatible with the initial makefiles when they are run outside of portage.
Comment 10 Vince C. 2019-07-12 10:36:51 UTC
(In reply to Robin Johnson from comment #6)
> 1. It fails to build here. amd64 ABI_X86="32 (64)":
> x86_64-pc-linux-gnu-gcc -m32-ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o
> ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o
> lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o
> ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o
> # DLL needs all object files
> x86_64-pc-linux-gnu-gcc: error: rcu: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: liblua.a: No such file or directory
> x86_64-pc-linux-gnu-gcc: error: unrecognized command line option ‘-m32-ar’
> 
> The 64-bit variant DOES build fine; just amd64+32-bit fails.

Here's a version that compiles and is fully functional. At least now there should be no more compile/install error.

> 2. The slotted-lua work needs to be integrated to this update.

I still need to be sure what you mean with slotted so, please, could you clarify/confirm ?

> 3. Please restore the optional USE=static; it still saves 160KiB for uses
> that care about raw space usage.

That is still work-in-progress...
Comment 11 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2019-07-12 18:53:17 UTC
(In reply to Vince C. from comment #10)
> (In reply to Robin Johnson from comment #6)
> > 1. It fails to build here. amd64 ABI_X86="32 (64)":
...
> > The 64-bit variant DOES build fine; just amd64+32-bit fails.
> Here's a version that compiles and is fully functional. At least now there
> should be no more compile/install error.
Thanks, I confirm both 32+64 do build now, but there's other issues, see below.

> > 2. The slotted-lua work needs to be integrated to this update.
> I still need to be sure what you mean with slotted so, please, could you
> clarify/confirm ?
See lua-5.1.5-r10x series. The library generated is liblua5.1.so, with matching binaries (/usr/bin/lua5.1) and pkgconfig. The 5.2/5.3 stuff also generates similar numbered non-conflicting libraries & binaries.

> > 3. Please restore the optional USE=static; it still saves 160KiB for uses
> > that care about raw space usage.
> That is still work-in-progress...
Ok.

Two breakages I see from the old series vs the one ebuild:
4. * QA Notice: Files built without respecting LDFLAGS have been detected
 *  Please include the following list of files in your report:
 * /usr/bin/lua
 * /usr/bin/luac
 * /usr/lib32/liblua.so.5.1.5
 * /usr/lib64/liblua.so.5.1.5

5. "emake -j1" => there was a minimal fix to the upstream Makefiles to support parallel build properly. It should be preserved.
Comment 12 Vince C. 2019-07-14 09:14:30 UTC
(In reply to Robin Johnson from comment #11)

> 5. "emake -j1" => there was a minimal fix to the upstream Makefiles to
> support parallel build properly. It should be preserved.

Oops! The "-j1" was an oversight, I forgot to remove it before submitting. It merely was to serialize the build phase otherwise I'd get confused. I'll remove it.


> > > 2. The slotted-lua work needs to be integrated to this update.
> > I still need to be sure what you mean with slotted so, please, could you
> > clarify/confirm ?
> See lua-5.1.5-r10x series. The library generated is liblua5.1.so, with
> matching binaries (/usr/bin/lua5.1) and pkgconfig. The 5.2/5.3 stuff also
> generates similar numbered non-conflicting libraries & binaries.

Ok, I'll see what I can do. It doesn't sound too much difficult with my level of understanding.


> Two breakages I see from the old series vs the one ebuild:
> 4. * QA Notice: Files built without respecting LDFLAGS have been detected
>  *  Please include the following list of files in your report:
>  * /usr/bin/lua
>  * /usr/bin/luac
>  * /usr/lib32/liblua.so.5.1.5
>  * /usr/lib64/liblua.so.5.1.5

This is interesting as I don't see that message on my system when emerging r5. I have absolutely no clue. What does it mean?

Anyway thanks for your guidance in this process. I really appreciate.
Comment 13 Vince C. 2019-07-14 10:32:18 UTC
(In reply to Robin Johnson from comment #11)
> Two breakages I see from the old series vs the one ebuild:
> 4. * QA Notice: Files built without respecting LDFLAGS have been detected
>  *  Please include the following list of files in your report:
>  * /usr/bin/lua
>  * /usr/bin/luac
>  * /usr/lib32/liblua.so.5.1.5
>  * /usr/lib64/liblua.so.5.1.5

Maybe I have a clue after all: in the ebuild I added that line:

    append-ldflags "-Wl,-E"

and it turns out it's nowhere to be seen in the build log, which means it's not used. And as a matter of fact, no target in the Makefiles use the default rules, in which case $(LDFLAGS) would have been pulled. I'll just remove that line from the ebuild as it only serves building the shared object and the previous released didn't include that linker flag. I'll transfer it to the SOFLAGS in the Makefile.

Did I get it right?

Now as for the slotted version, I do realize there's a bit of work ahead for what I see of eselect-lua, the installed include directories also include the version. I have questions about slotted packages:

 * does eselect-<package> alone manage all those symbolic links?
 * or can the ebuild install symbolic links to the risk of creating collisions?
 * or must the ebuild call eselect?
Comment 14 William Hubbs gentoo-dev 2020-12-30 20:04:56 UTC
I am looking at integrating this into the slotted lua ebuilds.

Removing the static use flag is fine; QA advised me that we are not
allowed to install the static libraries. Also, I agree with linking the
lua and luac binaries the way upstream does, so I will be doing that.
Comment 15 Larry the Git Cow gentoo-dev 2022-09-05 08:20:07 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=678e40a0f28403a4e6ef76695e33429fe6cefe00

commit 678e40a0f28403a4e6ef76695e33429fe6cefe00
Author:     David Seifert <soap@gentoo.org>
AuthorDate: 2022-09-05 08:20:00 +0000
Commit:     David Seifert <soap@gentoo.org>
CommitDate: 2022-09-05 08:20:00 +0000

    dev-lang/lua: update EAPI 7 -> 8
    
    Closes: https://github.com/gentoo/gentoo/pull/27061
    Bug: https://bugs.gentoo.org/472230
    Bug: https://bugs.gentoo.org/627330
    Bug: https://bugs.gentoo.org/689598
    Bug: https://bugs.gentoo.org/706378
    Bug: https://bugs.gentoo.org/765712
    Bug: https://bugs.gentoo.org/791451
    Bug: https://bugs.gentoo.org/834911
    Signed-off-by: David Seifert <soap@gentoo.org>

 dev-lang/lua/Manifest              |  1 +
 dev-lang/lua/lua-5.4.4-r100.ebuild | 39 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=818a17909877d016c6baa9ab04ea875e98fd80bb

commit 818a17909877d016c6baa9ab04ea875e98fd80bb
Author:     David Seifert <soap@gentoo.org>
AuthorDate: 2022-09-05 08:19:59 +0000
Commit:     David Seifert <soap@gentoo.org>
CommitDate: 2022-09-05 08:19:59 +0000

    dev-lang/lua: update EAPI 7 -> 8
    
    Bug: https://bugs.gentoo.org/472230
    Bug: https://bugs.gentoo.org/627330
    Bug: https://bugs.gentoo.org/689598
    Bug: https://bugs.gentoo.org/706378
    Bug: https://bugs.gentoo.org/765712
    Bug: https://bugs.gentoo.org/791451
    Bug: https://bugs.gentoo.org/834911
    Signed-off-by: David Seifert <soap@gentoo.org>

 dev-lang/lua/Manifest              |  1 +
 dev-lang/lua/lua-5.3.6-r100.ebuild | 39 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fab25cc8ac44fa73bb14821560ac1ba1829447df

commit fab25cc8ac44fa73bb14821560ac1ba1829447df
Author:     David Seifert <soap@gentoo.org>
AuthorDate: 2022-09-05 08:19:58 +0000
Commit:     David Seifert <soap@gentoo.org>
CommitDate: 2022-09-05 08:19:58 +0000

    dev-lang/lua: add 5.1.6
    
    * add two patches from Debian
    
    Bug: https://bugs.gentoo.org/472230
    Bug: https://bugs.gentoo.org/627330
    Bug: https://bugs.gentoo.org/689598
    Bug: https://bugs.gentoo.org/706378
    Bug: https://bugs.gentoo.org/765712
    Bug: https://bugs.gentoo.org/791451
    Bug: https://bugs.gentoo.org/834911
    Signed-off-by: David Seifert <soap@gentoo.org>

 dev-lang/lua/Manifest                              |  1 +
 dev-lang/lua/files/5.1/0001-extern_C.patch         | 20 +++++++++
 ...02-Fix-stack-overflow-in-vararg-functions.patch | 17 ++++++++
 dev-lang/lua/lua-5.1.6.ebuild                      | 49 ++++++++++++++++++++++
 4 files changed, 87 insertions(+)
Comment 16 Larry the Git Cow gentoo-dev 2022-10-13 13:19:56 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1bc1f784b5c91f2e0be1aa06b155cff958ba22a0

commit 1bc1f784b5c91f2e0be1aa06b155cff958ba22a0
Author:     David Seifert <soap@gentoo.org>
AuthorDate: 2022-10-13 13:19:37 +0000
Commit:     David Seifert <soap@gentoo.org>
CommitDate: 2022-10-13 13:19:37 +0000

    dev-lang/lua: drop 5.1.5-r109, 5.3.6-r5, 5.4.4-r2
    
    Bug: https://bugs.gentoo.org/520480
    Bug: https://bugs.gentoo.org/717780
    Closes: https://bugs.gentoo.org/460114
    Closes: https://bugs.gentoo.org/462064
    Closes: https://bugs.gentoo.org/539826
    Closes: https://bugs.gentoo.org/627330
    Closes: https://bugs.gentoo.org/689598
    Closes: https://bugs.gentoo.org/706378
    Closes: https://bugs.gentoo.org/791772
    Closes: https://bugs.gentoo.org/834153
    Closes: https://bugs.gentoo.org/834911
    Closes: https://bugs.gentoo.org/843320
    Signed-off-by: David Seifert <soap@gentoo.org>

 dev-lang/lua/Manifest                         |   5 -
 dev-lang/lua/files/configure.in               |   5 -
 dev-lang/lua/files/lua-5.1-module_paths.patch |  30 -----
 dev-lang/lua/files/lua-5.1-readline.patch     |  10 --
 dev-lang/lua/files/lua-5.1.4-deprecated.patch |  46 -------
 dev-lang/lua/files/lua-5.1.5-make.patch       |  97 -------------
 dev-lang/lua/files/lua-5.3.6-make.patch       |  91 -------------
 dev-lang/lua/files/lua-5.4.2-r2-make.patch    |  99 --------------
 dev-lang/lua/files/lua.pc                     |  31 -----
 dev-lang/lua/lua-5.1.5-r109.ebuild            | 145 --------------------
 dev-lang/lua/lua-5.3.6-r5.ebuild              | 187 --------------------------
 dev-lang/lua/lua-5.4.4-r2.ebuild              | 184 -------------------------
 dev-lang/lua/metadata.xml                     |  23 ++--
 13 files changed, 11 insertions(+), 942 deletions(-)