Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 124719 - dev-lang/lua-5.1 (version bump)
Summary: dev-lang/lua-5.1 (version bump)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Matti Bickel (RETIRED)
URL: http://lua.org
Whiteboard:
Keywords:
Depends on: 155518
Blocks: 128217 136077 147440 148996
  Show dependency tree
 
Reported: 2006-03-02 16:53 UTC by thedcm
Modified: 2007-03-10 14:58 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
bzip2'ed patch to autotoolize Lua 5.1 tree. (lua-5.1-autotoolize-r1.patch.bz2,209.98 KB, application/octet-stream)
2006-05-06 02:06 UTC, Petri Lehtinen
Details
Quick and dirty lua-5.1.1.ebuild (lua-5.1.1.ebuild,979 bytes, text/plain)
2006-06-28 14:51 UTC, Cyril Romain
Details
lua-5.1.1.ebuild - quick but not so dirty ;) (lua-5.1.1.ebuild,661 bytes, text/plain)
2006-07-31 01:17 UTC, Natanael Copa
Details
Updated ebuild (lua-5.1.1.ebuild,760 bytes, text/plain)
2006-07-31 02:04 UTC, Natanael Copa
Details
lua-5.1.1-luaconf-paths.patch (lua-5.1.1-luaconf-paths.patch,959 bytes, patch)
2006-09-06 02:36 UTC, Natanael Copa
Details | Diff
files/lua-5.1.1-no-readline.patch (lua-5.1.1-no-readline.patch,745 bytes, patch)
2006-09-30 17:10 UTC, Natanael Copa
Details | Diff
lua-5.1.1.ebuild (update) (lua-5.1.1.ebuild,876 bytes, text/plain)
2006-09-30 17:12 UTC, Natanael Copa
Details
lua-5.1.1 ebuild w/ pkg-config (lua-5.1.1.ebuild,1.30 KB, text/plain)
2006-11-01 10:30 UTC, Andre Bogus
Details
files/lua-5.1.1-make_dynamic.patch (lua-5.1.1-make_dynamic.patch,552 bytes, patch)
2006-11-20 23:36 UTC, Natanael Copa
Details | Diff
lua-5.1.1 ebuild w/ pkg-config; libdir fixed (lua-5.1.1.ebuild,1.40 KB, text/plain)
2007-01-03 02:25 UTC, Dmitry S. Kulyabov
Details
unpacking fails with USE=static (lua-5.1.1-make_static.patch-9348.out,2.60 KB, text/plain)
2007-01-23 09:30 UTC, Vadim Zborovskii
Details
modified make_static patch that works with new lua-5.1.1 ebuild (lua-5.1.1-make_static.patch,523 bytes, patch)
2007-01-23 11:17 UTC, Vadim Zborovskii
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description thedcm 2006-03-02 16:53:54 UTC
http://www.lua.org/ftp/lua-5.1.tar.gz
Comment 1 Tom Payne (RETIRED) gentoo-dev 2006-03-03 01:04:07 UTC
Hi thedcm,

Thanks for the bug report. From looking at the package it seems that the default lua-5.1 build system only builds static libraries. Many existing applications in portage expect Lua to havee shared libraries (although you could work around this by running revdep-rebuild after emergeing lua-5.1).

Lua 5.1's configuration system is also a bit simplistic: what they consider a 'linux' platform does not correspond very well to Gentoo (Gentoo includes a lot more) and with paths hardcoded in C header files (src/luaconf.h) it's not particularly easy to use.

Overall, the existing 5.1 build system doesn't play well with Gentoo. So, I can forsee lua-5.1 going into portage packaged.mask'd so that it's available if people really want it, but otherwise it won't go in to unstable until the build system is upgraded. I'm not sure if/when I'll have time to do this.

Regards,

Tom
Comment 2 Petri Lehtinen 2006-05-05 00:07:50 UTC
I created an autotoolized tree of Lua 5.1. It can be obtained from http://www.digip.org/~akheron/lua-at/lua-at-5.1.tar.gz.

I hope this helps getting Lua 5.1 to Portage. I'll be glad to accept any suggestions about the package.

By the way, Lua 5.1 is not backwards compatible with Lua 5.0.x. I think this means it should go to a slot or something?
Comment 3 Petri Lehtinen 2006-05-06 02:06:57 UTC
Created attachment 86242 [details]
bzip2'ed patch to autotoolize Lua 5.1 tree.

Better still, here's a patch. It also has some fixes over the tarball linked in the last post.
Comment 4 Brad Allen 2006-06-22 14:11:15 UTC
Looks like I *might* look at that autotooled version.  ion-3ds-20060519 from http://modeemi.fi/~tuomov/ion/logs/ion-3-relnotes.html.gz requires Lua 5.1, and according to http://www.lua.org/versions.html#5.1 Lua 5.1.1 is now released (so version bump to Lua 5.1.1 if possible).

I'm not sure which version of Ion I may test.  Starting with 1, then will try 2 or 3 after that.  Currently fairly happy with ratpoison, so time may intercede and stop me from trying anything.  :(
Comment 5 Xavier Maillard 2006-06-27 16:51:06 UTC
Any update for this version bump integration ?
Comment 6 Xavier Maillard 2006-06-27 16:51:53 UTC
Any update for this version bump integration ?
Comment 7 Brad Allen 2006-06-28 11:36:06 UTC
(In reply to comment #5)
> Any update for this version bump integration ?
> 

Well, I patched Lua 5.1.1 with the above 5.1 patch with a standard patch -p1 command, ignored the rejects which just removed files, then built and installed it into local, unmerged the Gentoo lua so it wouldn't be used, and then built and installed the latest development ion into local, and uninstalled the Gentoo ion as well.  In an Xvnc, it ran OK for the few seconds I tested it.  I have no idea how to stress test it, since I was investigating ion to replace ratpoison, but didn't like it yet since the default keys are difficult and I simply haven't had the time to fully test it in the main screen and tailor it.

Now, since I hate autoconf like the plague (it is horrendously slow and impossible to program with), I naturally used the default "make" commands to build, never touching ./configure at all.  Is that the point of this discussion, per chance?  Maybe I missed out completely ... duh.
Comment 8 Brad Allen 2006-06-28 11:42:52 UTC
(In reply to comment #7)

> Now, since I hate autoconf like the plague (it is horrendously slow and
> impossible to program with), I naturally used the default "make" commands to
> build, never touching ./configure at all.  Is that the point of this
> discussion, per chance?  Maybe I missed out completely ... duh.

Ok, so now I'm trying both with ./configure.  I removed the 4 files that were patch rejects of removing files, and then ran "autoconf", then "./configure --prefix=/usr/local".  I got a VERY typical autoconf error:

then mv -f ".deps/loadlib.Tpo" ".deps/loadlib.Plo"; else rm -f ".deps/loadlib.Tpo"; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I.. -Wall -g -O2 -MT loadlib.lo -MD -MP -MF .deps/loadlib.Tpo -c loadlib.c  -fPIC -DPIC -o .libs/loadlib.o
loadlib.c: In function 'luaopen_package':
loadlib.c:646: error: 'LUA_PATH' undeclared (first use in this function)
loadlib.c:646: error: (Each undeclared identifier is reported only once
loadlib.c:646: error: for each function it appears in.)
loadlib.c:647: error: 'LUA_CPATH' undeclared (first use in this function)
make[3]: *** [loadlib.lo] Error 1
make[3]: Leaving directory `/tmp/gentoo/lua-5.1.1/src'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/tmp/gentoo/lua-5.1.1/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/tmp/gentoo/lua-5.1.1'
make: *** [all] Error 2


That's why I hate autoconf.  It makes porting IMPOSSIBLE and compilation the most abysmally painful thing in the world.  And there's no way to fix it.  If it's in the source code, that can be fixed, but there's no way to fix autoconf without reading 1,000,000,000,000 lines of idiotic disorganized shell script.
Comment 9 Brad Allen 2006-06-28 11:49:19 UTC
(In reply to comment #8)
> I removed the 4 files that were
> patch rejects of removing files,

That's not what I meant.  I meant I removed the 4 files the rejected patch files were trying to remove.
Comment 10 Cyril Romain 2006-06-28 14:51:41 UTC
Created attachment 90379 [details]
Quick and dirty lua-5.1.1.ebuild

I'm an ebuild newbie so sorry if I didn't respect the proper way to do it. 
This ebuild
 - does not use 'make install' but 'insinto' + 'doins' to avoid ACCESS DENIED on mkdir and cp command.
 - use the 'linux' target plateform (don't know if a variable like $PLATEFORM exist to write a proper switch)

But it worked here on x86. For those who urgely need >=lua5.1 ...
Comment 11 Pablo Hess 2006-06-29 13:37:22 UTC
(In reply to comment #10)

It worked perfectly here on x86 too. All my scripts are working fine.
Plus, now the operation 4 % 3 works and gives me a 1, instead of an error. :)

Thanks for the ebuild.
Comment 12 Dennis Schridde 2006-07-09 09:50:46 UTC
Any progress in getting this into portage?
Comment 13 Natanael Copa 2006-07-31 01:17:27 UTC
Created attachment 93110 [details]
lua-5.1.1.ebuild - quick but not so dirty ;)

Here is a modified ebuild. It replaces INSTALL_TOP=.* with INSTALL_TOP=$(DESTDIR)/usr so einstall works. Diff below:

--- lua-5.1.1.ebuild.orig       2006-07-31 10:07:38.047264000 +0200
+++ lua-5.1.1.ebuild    2006-07-31 10:12:00.268264000 +0200
@@ -20,7 +20,7 @@

        cd ${S}

-       sed -i Makefile -e 's:^\(INSTALL_TOP= \)/usr/local:\1/usr:'
+       sed -i Makefile -e 's:^INSTALL_TOP=.*:INSTALL_TOP= $(DESTDIR)/usr:'
        sed -i doc/readme.html -e 's:\(/README\)\("\):\1.gz\2:g'
 }

@@ -29,17 +29,5 @@
 }

 src_install() {
-       insinto /usr/bin
-       doins src/lua src/luac
-       fperms 0755 /usr/bin/luac /usr/bin/lua
-       insinto /usr/include
-       doins src/lua.h src/luaconf.h src/lualib.h src/lauxlib.h etc/lua.hpp
-       insinto /usr/lib
-       doins src/liblua.a
-
-       dohtml doc/*.html doc/*.gif
-       for i in `find . -name README -printf "%h\n"`; do
-               docinto ${i#.}
-               dodoc ${i}/README
-       done
+       einstall
 }
Comment 14 Natanael Copa 2006-07-31 02:04:25 UTC
Created attachment 93113 [details]
Updated ebuild

Sorry, when I tested it I didn't that I emerge 5.0.2 and not my modified 5.1.1 ebuild. While fixing it I also removed the sed line changing the INSTALL_TOP in the Makefile and use a override instead. I also included the doc from the original 5.1.1 ebuild.

Tested and works on embedded (uclibc) and amd64. However, as mentioned as before, it should probably be slotted.

--- lua-5.1.1.ebuild.orig       2006-07-31 10:07:38.047264000 +0200
+++ lua-5.1.1.ebuild    2006-07-31 10:55:37.693540000 +0200
@@ -19,24 +19,15 @@
        unpack ${A}

        cd ${S}
-
-       sed -i Makefile -e 's:^\(INSTALL_TOP= \)/usr/local:\1/usr:'
        sed -i doc/readme.html -e 's:\(/README\)\("\):\1.gz\2:g'
 }

 src_compile() {
-       emake linux
+       emake linux INSTALL_TOP=${D}/usr
 }

 src_install() {
-       insinto /usr/bin
-       doins src/lua src/luac
-       fperms 0755 /usr/bin/luac /usr/bin/lua
-       insinto /usr/include
-       doins src/lua.h src/luaconf.h src/lualib.h src/lauxlib.h etc/lua.hpp
-       insinto /usr/lib
-       doins src/liblua.a
-
+       make install INSTALL_TOP=${D}/usr
        dohtml doc/*.html doc/*.gif
        for i in `find . -name README -printf "%h\n"`; do
                docinto ${i#.}
Comment 15 Petri Lehtinen 2006-08-08 01:12:07 UTC
(In reply to comment #14)
> Created an attachment (id=93113) [edit]
> Updated ebuild

The sed'ing in this ebuild is not enough. LUA_ROOT (and possibly LUA_LDIR and LUA_CDIR) in luaconf.h must also be modified. They define the default search paths used by lua module system to find libraries. The default search paths are under /usr/local/.

I'd also like to see an USE flag for readline support. make linux includes readline support by default. But after all, this is only a matter of some sed'ing, too.

Shared libraries are a trickier problem. The default build system has no option to build shared libraries. My opinion is that shared libraries need not be built at all, because:
1) The default build system doesn't do it so programs using Lua can't depend on it
2) The API seems to have backwards incompatible changes between minor versions (e.g. from 5.0 to 5.1). This means that different versions of Lua must go to slots anyway.

I'm not going to create another patch to autotoolize Lua 5.1.1 because it changes the build system dramatically and I think it's not necessary anymore because the main reason for autotooling was the shared libraries.

I don't have a Gentoo system around right now, so I can't test those ebuilds or try to modify them to reflect my suggestions, but I would appreciate that if some of you can.

Petri
Comment 16 Natanael Copa 2006-09-06 02:36:28 UTC
Created attachment 96154 [details, diff]
lua-5.1.1-luaconf-paths.patch

The patch is sets LUA_ROOT, LUA_LDIR and LUA_CDIR in luaconf.h. I had a look at the debian patches and this the way they do it.

I will also follow up and try to provide patches for dynamic linking using libtool. If Debian can do it then can Gentoo do it too. ;)

I'll try making it slot'able too.
Comment 17 Marcel Klein 2006-09-30 00:35:27 UTC
(In reply to comment #16)

> I will also follow up and try to provide patches for dynamic linking using
> libtool. If Debian can do it then can Gentoo do it too. ;)

Are there any updates on your patches, so we can see lua-5.1.1 in portage in the near future? :)
Comment 18 Maarten Aertsen 2006-09-30 10:48:43 UTC
(In reply to comment #17)
> Are there any updates on your patches, so we can see lua-5.1.1 in portage in
> the near future? :)

It would be great to have a lua ebuild in portage, especially with the new build of ion3 coming near! I am available to test on amd64 (with both --march=nocona and --march=athlon64), if someone has enough spare time to create a new updated build.

Thanks for the work on lua so far!

Comment 19 Natanael Copa 2006-09-30 15:42:36 UTC
(In reply to comment #17)
> (In reply to comment #16)
> 
> > I will also follow up and try to provide patches for dynamic linking using
> > libtool. If Debian can do it then can Gentoo do it too. ;)
> 
> Are there any updates on your patches, so we can see lua-5.1.1 in portage in
> the near future? :)

I'm sorry i haven't uploaded anythig yet. Since my local overlay orked just fine with the static lua, I havent really prioriticed it.

I'm not very experienced with either libtool or slots in gentoo, so I'll tell you what I'm thiking and are open for input.

I'm thinking of manually merging the patch for debian (see below) to the make file. It will create the .so files and it will append 5.1 to the binary (/usr/bin/lua5.1) and to the library (/usr/lib/lua5.1.so) That will let us have several versions of lua installed in parallell i slots. A lua-config script or something like that could make links  the desired verision. (/usr/bin/lua -> /usr/bin/lua5.1)

Thoughts?

--- trunk~/src/Makefile 2006-01-11 18:49:05.000000000 -0500
+++ trunk/src/Makefile  2006-01-22 22:11:48.000000000 -0500
@@ -165,3 +165,29 @@
   ltm.h lzio.h lmem.h lopcodes.h lundump.h

 # (end of Makefile)
+
+# The following rules use libtool for compiling and linking in order to
+# provide shared library support.  While we are at it, our desired version
+# suffixes are added to the targets, preventing conflicts with rules in
+# the upstream makefile.
+
+LIB_NAME = liblua$(V).la
+LIB_OBJS = $(CORE_O:.o=.lo) $(LIB_O:.o=.lo)
+
+%.lo %.o: %.c
+       $(LIBTOOL) --mode=compile $(CC) -c $(CPPFLAGS) $(CFLAGS) $<
+
+$(LIB_NAME) $(LIB_NAME:.la=.a): $(LIB_OBJS)
+       $(LIBTOOL) --mode=link $(CC) -version-info $(LIB_VERSION) \
+            -rpath $(RPATH) -o $(LIB_NAME) $(LIB_OBJS) $(LIB_LIBS)
+
+lua$(V): $(LUA_O) $(LIB_NAME)
+       $(LIBTOOL) --mode=link $(CC) -static -Wl,-E -o $@ $(LUA_O) $(LIB_NAME) $(LUA_LIBS)
+
+luac$(V): $(LUAC_O) $(LIB_NAME)
+       $(LIBTOOL) --mode=link $(CC) -static -o $@ $(LUAC_O) $(LIB_NAME)
+
+debian_clean:
+       $(LIBTOOL) --mode=clean $(RM) $(ALL_O:.o=.lo) $(LIB_NAME) lua$(V) luac$(V)
+
+debian_all: $(LIB_NAME) lua$(V) luac$(V)

Comment 20 Dennis Schridde 2006-09-30 15:54:47 UTC
Why is this needed? Can't we just use the standard, static library?
Slot or mask it so it doesn't clash with those apps requiring a dynamic library... (I'd be fine with masking.)
I guess that would be easier and bring me lua 5.1 much sooner...
Comment 21 Natanael Copa 2006-09-30 17:10:35 UTC
Created attachment 98486 [details, diff]
files/lua-5.1.1-no-readline.patch

patch to remove readline support
Comment 22 Natanael Copa 2006-09-30 17:12:57 UTC
Created attachment 98487 [details]
lua-5.1.1.ebuild (update)

Updated ebuild with readline useflag.

I think this ebuild could be commited to the portage tree. If shared library is really needed, a new bug could be opened. (or it could be proposed upstream)
Comment 23 Marcel Klein 2006-09-30 17:17:44 UTC
(In reply to comment #22)

> I think this ebuild could be commited to the portage tree.

Yep, works fine here. :)
Comment 24 Natanael Copa 2006-10-18 01:48:39 UTC
Just wondering what we are waiting for here. Are we waiting for me to provide a shared lib or are we waiting for someone with commit power to do a commit?
Comment 25 Marcel Klein 2006-10-18 06:48:27 UTC
I guess we are waiting for somebody with the power to commit the ebuild.
It works fine...
Comment 26 Jari-Matti Mäkelä 2006-10-21 06:37:43 UTC
(In reply to comment #25)
> I guess we are waiting for somebody with the power to commit the ebuild.
> It works fine...

It does not work fine. I guess it's missing some pkgconfig stuff. I was unable to compile lighttpd 1.4.13 against lua-5.1.1 using the three last files from this bug report. A quick diff against the newest lua from portage tree reveals that this ebuild does not have at least the following

>       cat >etc/lua.pc <<EOF
> prefix=/usr
> exec_prefix=\${prefix}
> includedir=\${prefix}/include
> libdir=\${exec_prefix}/$(get_libdir)
> interpreter=\${exec_prefix}/bin/lua
> compiler=\${exec_prefix}/bin/luac
>
> Name: Lua
> Description: An extension programming language
> Version: ${PV}
> Cflags: -I\${includedir}
> Libs: -L\${libdir} -llua -llualib -lm $(dlopen_lib)
> EOF
...
>       insinto /usr/$(get_libdir)/pkgconfig
>       doins etc/lua.pc

Maybe fixing those might help.
Comment 27 Johan Bergström 2006-10-21 14:40:27 UTC
Since im not the maintainer here i cant give you any straight answers to why the patch isn't accepted - but the thing is that this ebuild does not build DSO's (which for instance lighttpd >1.4.12 wants).. and there are probably a heap of ebuilds which expect DSO's from lua <5.1.
By dissecting the debian patch further (the other stuff in these ebuilds are practically cut 'n' pastes from that patch) we should have a dso built with not too much effort.

Just my 2 cents though :)
Comment 28 Christoph Kappel 2006-10-24 02:30:13 UTC
So, if I am getting all right there won't be an official ebuild for Lua that builds DSO's? I spent a lot of time to figure out _why_ the devs just retain DSO's for Linux. After reading some posts in the mailinglist I found two reasons:
1) creating DSO's with Libtool isn't portable
2) DSO's need an extra register which will slow down the Lua VM

Does it matter? I actually don't care about this register, the VM is fast enough for me and this is only Linux. I have Libtool and I dislike the idea of static linking every binary/lib that has Lua support.
Comment 29 Natanael Copa 2006-10-24 03:01:14 UTC
(In reply to comment #28)
> So, if I am getting all right there won't be an official ebuild for Lua that
> builds DSO's? I spent a lot of time to figure out _why_ the devs just retain
> DSO's for Linux. After reading some posts in the mailinglist I found two
> reasons:

This is the lua dev's right?

> 1) creating DSO's with Libtool isn't portable
> 2) DSO's need an extra register which will slow down the Lua VM
> 
> Does it matter? I actually don't care about this register, the VM is fast
> enough for me and this is only Linux. I have Libtool and I dislike the idea of
> static linking every binary/lib that has Lua support.

Thanks for the research. The only reason I havent made an DSO's of it yet is that it's not supported upstream. I have been thinking that if we want/need dynamic built lua, it should be fixed upstream.

I definitively agree with you that dynamic linking is worth the cost of speed.
We could even have support for the static useflag for those who need the speed.

So what do you say? Should we try maintaining our own libtoolified version of lua (like debian) or should we just support what upstream provides? I would like to hear what the gentoo devs thinks before I work more on it.

Comment 30 Christoph Kappel 2006-10-24 08:10:58 UTC
> This is the lua dev's right?
Of course it is, but I can't agree to their reasons. Their build system creates DSO's for OS X and Windows, what the heck is the problem to use Libtool on demand if someone wants it? Actually they replaces the autotool-scripts with a really frugal makefile.

> I definitively agree with you that dynamic linking is worth the cost of speed.
> We could even have support for the static useflag for those who need the speed.
I doubt that most of the users care about a special useflag or even about dynamic of static linking. In my opinion it would be a waste of disk space and really hard to maintain, because you can't be certain which version was built into the binary/library. And what happens on updates which rely on features of newer versions of Lua? Automagically revdep?

> So what do you say? Should we try maintaining our own libtoolified version of
> lua (like debian) or should we just support what upstream provides? I would
> like to hear what the gentoo devs thinks before I work more on it.
I would prefer a libtoolized version with DSOs, if it won't be part of upstream I will keep my overlayed version.

Actually there's another problem with the lack of DSO's: My current project uses Lua too, that's how I got into this. During tests on different dists I recognized the great ABI change bewteen 5.0 and 5.1*. It took me some time to figure out WHY static linking ended in undefined references to all calls of Lua functions. Later, after some researches again, I read about the required order of the linker options. I don't know if it was just mistaken by me, but are further checks of the linker options kind of mandatory for the projects with Lua support?

Comment 31 Natanael Copa 2006-10-24 08:47:04 UTC
(In reply to comment #30)
> > This is the lua dev's right?
> Of course it is, but I can't agree to their reasons. Their build system creates
> DSO's for OS X and Windows, what the heck is the problem to use Libtool on
> demand if someone wants it? Actually they replaces the autotool-scripts with a
> really frugal makefile.

I'll look at it (not today though). If more people ask about it they might understand.

Another option (for upstream) would be to have current "make linux", just as it is and just add "make libtool". It wouldnt break compatibility with current, and it would give the people who needs DSO's that possibility.

> > I definitively agree with you that dynamic linking is worth the cost of speed.
> > We could even have support for the static useflag for those who need the speed.
> I doubt that most of the users care about a special useflag or even about
> dynamic of static linking. In my opinion it would be a waste of disk space and
> really hard to maintain, because you can't be certain which version was built
> into the binary/library. And what happens on updates which rely on features of
> newer versions of Lua? Automagically revdep?
> 
> > So what do you say? Should we try maintaining our own libtoolified version of
> > lua (like debian) or should we just support what upstream provides? I would
> > like to hear what the gentoo devs thinks before I work more on it.
> I would prefer a libtoolized version with DSOs, if it won't be part of upstream
> I will keep my overlayed version.

I think gentoo could maintain it. After all, debian does (so we could copy the deiban patches). We should still poke the lua dev's again.

> Actually there's another problem with the lack of DSO's: My current project
> uses Lua too, that's how I got into this. During tests on different dists I
> recognized the great ABI change bewteen 5.0 and 5.1*. It took me some time to
> figure out WHY static linking ended in undefined references to all calls of Lua
> functions. Later, after some researches again, I read about the required order
> of the linker options. I don't know if it was just mistaken by me, but are
> further checks of the linker options kind of mandatory for the projects with
> Lua support?

require orer of linker opts? sounds wierd. Since you already have done some research on it, would you care to send me some links? I'd like to read the history about why lua build system refuses to support DSO's and try to give it another try upstream.
Comment 32 Christoph Kappel 2006-10-24 09:37:32 UTC
(In reply to comment #31)
> Another option (for upstream) would be to have current "make linux", just as it
> is and just add "make libtool". It wouldnt break compatibility with current,
> and it would give the people who needs DSO's that possibility.
> 
There are still discussion on their mailinglists, some suggestions are a target for DSO's as you said in your prior post. And of course about the different version numbers and how to name (SONAME) it.

> I think gentoo could maintain it. After all, debian does (so we could copy the
> deiban patches). We should still poke the lua dev's again.
Indeed, most of the dists do it anyway. 

> require orer of linker opts? sounds wierd. Since you already have done some
> research on it, would you care to send me some links? I'd like to read the
> history about why lua build system refuses to support DSO's and try to give it
> another try upstream.
Yes, the object files must be before the link option. Within the autotools I had to change LDFLAGS to LDADD to get this working.

Well, I don't know if I find this links again..

http://lua-users.org/lists/lua-l/2006-01/msg00663.html
http://lua-users.org/lists/lua-l/2006-01/msg00645.html
http://lua-users.org/lists/lua-l/2006-03/msg00119.html

Hm, I can't find the other post. Maybe later, I will add it here.
Comment 33 Cheba 2006-10-24 10:49:58 UTC
This ebuild does not create *.pc file.

-=>> pkg-config --libs --cflags 'lua'
Package lua was not found in the pkg-config search path.
Perhaps you should add the directory containing `lua.pc'
to the PKG_CONFIG_PATH environment variable
No package 'lua' found

-=>> pkg-config --libs --cflags 'lua5.1'
Package lua5.1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `lua5.1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'lua5.1' found

While 5.0.3 is OK.

-=>> pkg-config --libs --cflags 'lua'
 -llua -llualib -lm -ldl
Comment 34 Christoph Kappel 2006-10-24 15:12:53 UTC
(In reply to comment #33)
> This ebuild does not create *.pc file.
The ebuild has no .pc, but the tarballs contains one. Just copy the values either from the .pc or the ebuild of 5.0.*.

> -=>> pkg-config --libs --cflags 'lua'
>  -llua -llualib -lm -ldl
-lualib is deprecated, Lua 5.1.* is only one lib. Just: -llua -ln -ldl

Comment 35 Andre Bogus 2006-11-01 10:30:33 UTC
Created attachment 100980 [details]
lua-5.1.1 ebuild w/ pkg-config

This is the given ebuild plus a few lines in install() that manage the pkg-config and xpm icon stuff. (taken from 5.0.3 ebuild)
Comment 36 Maks Verver 2006-11-01 13:12:02 UTC
Shouldn't lua be compiled with -fpic on AMD64 so it can be linked to dynamic libraries?
Comment 37 Christoph Kappel 2006-11-01 16:02:25 UTC
According to the mailinglist the devs dissuade from using pic. The DSO problem is still the same as mentioned before.
Comment 38 Andre Bogus 2006-11-03 02:55:55 UTC
Can anyone explain what the DSO problem is about?
Comment 39 Christian Axelsson 2006-11-05 08:11:56 UTC
I'm getting a small (non-critical) error when installing the pkg-config version (I haven't tested the other ones):

install: cannot stat `etc/lua.xpm': No such file or directory

We might want to fix it before this ebuild goes into portage.
Comment 40 Christoph Kappel 2006-11-06 04:29:07 UTC
Andre Bogus(In reply to comment #38)
> Can anyone explain what the DSO problem is about?
The devs of Lua complain about the non-portable way of creating shared libs within makefiles. Within Linux/Unix environments they are mostly created with libtool which isn't portable at all. Additionately they also bother about the number of register on ia32. DSOs need one register for reference, so the Lua VM will be slower.
Actually both aren't obvious reasons for me to get rid of the targets for Linux/Unix DSOs. Most of the dists build them anyway and the other OS' still get their DSOs.

Comment 41 Clemens Fruhwirth 2006-11-07 01:36:55 UTC
* use libtool. Gentoo is a Unix distro, we don't care what upstream cares for. If you are afraid of addditional work, switch to Debian as upstream source. Btw. -fPICing is demanded by Gentoo policy.
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=3&chap=1

* lua 5.1.1 seems source incompatible with 5.0.2. ion-20060326 works with 5.0.2 but fails compiling with 5.1.1. You might have to slot lua.
Comment 42 Christoph Kappel 2006-11-07 02:41:12 UTC
I am not afraid, writing a makefile isn't much work. I just don't get the point of the developers. PIC is another topic of discussion in the mailinglist.

There was a major ABI change between 5.0.3 and 5.1. I rely on Lua too and had to set Lua 5.1 as required. Actually I am not sure how the slot mechanism works with the linker. I know it will prefer shared over static libs. So even when the newer version doesn't have a DSO?
Comment 43 Clemens Fruhwirth 2006-11-07 03:00:14 UTC
PIC is demanded by the Gentoo policy, so the library can be shared in memory among processes. This lowers the memory footprint. If you need the few extra percent performance, then go and modify your lua-using program to link against the static version (which will always be non-PIC). But for 98% of the users, lua is not interesting at all, and a -fPIC shared library is just fine. I guess that's the rational for the Gentoo policy.

> There was a major ABI change between 5.0.3 and 5.1. I rely on Lua too and had
> to set Lua 5.1 as required. Actually I am not sure how the slot mechanism works
> with the linker. I know it will prefer shared over static libs. So even when
> the newer version doesn't have a DSO?

Linker and the slot mechanism is totally unrelated. Usually slotted packages use pkg-config (lua does). For instance, glib is slotted, and provides two package config file, one for "glib" (the old one) and "glib-2.0" the new one, that when asked for --libs returns the proper linker flags to use glib-2.0.

So for lua, I think there exists a lua-5.1 pc file already. You just have to make sure that it returns the correct stuff when "pkg-config --libs lua-5.1". 
Comment 44 Christoph Kappel 2006-11-07 06:36:52 UTC
(In reply to comment #43)
> PIC is demanded by the Gentoo policy, so the library can be shared in memory
> among processes. This lowers the memory footprint. If you need the few extra
> percent performance, then go and modify your lua-using program to link against
> the static version (which will always be non-PIC). But for 98% of the users,
> lua is not interesting at all, and a -fPIC shared library is just fine. I guess
> that's the rational for the Gentoo policy.

That is one of the reasons why there is no ebuild upstream and of course the incompatibilities of the different versions. Actually I did exact what you suggested - but I took me a while to figure out why I got many undefined references. (LDADD vs LDDFLAGS)

> Linker and the slot mechanism is totally unrelated. Usually slotted packages
> use pkg-config (lua does). For instance, glib is slotted, and provides two
> package config file, one for "glib" (the old one) and "glib-2.0" the new one,
> that when asked for --libs returns the proper linker flags to use glib-2.0.
> 
> So for lua, I think there exists a lua-5.1 pc file already. You just have to
> make sure that it returns the correct stuff when "pkg-config --libs lua-5.1". 

Ah for sure. I always use pkg-config, but there is a problem with the different names. I read some discussions in the Debian mailinglist about this topic. Prior it was just lua, later lua50, than lua5.1 and finally lua5.1.1. I think the best workaround is to check the version of the lib with the version reply of the interpreter and test the common names like the makefile of the latest ion release does.

Comment 45 Matti Bickel (RETIRED) gentoo-dev 2006-11-17 13:30:35 UTC
I've just committed an lua-5.1.1 ebuild to the tree. I dropped keywords b/c of the API change. Additionally, due to the quite heavy patching and the open slotting issue, it's listed in package.mask.
I'd appreciate testing and reports what i broke :)

I'll leave that bug open until it's time for stablization in ~30 days.
Comment 46 Natanael Copa 2006-11-18 02:06:38 UTC
(In reply to comment #45)
> I've just committed an lua-5.1.1 ebuild to the tree. I dropped keywords b/c of
> the API change. Additionally, due to the quite heavy patching and the open
> slotting issue, it's listed in package.mask.
> I'd appreciate testing and reports what i broke :)
> 
> I'll leave that bug open until it's time for stablization in ~30 days.
> 

wow! thanks! :D

Could you please add keywords for amd64 and x86?
Comment 47 Matti Bickel (RETIRED) gentoo-dev 2006-11-18 02:37:05 UTC
That's up to the arches (and the reason behind this depending on bug 155518), so you can all watch progress there.
Comment 48 Natanael Copa 2006-11-20 00:11:12 UTC
I just noticed that libtoolified usr/bin/lua-5.1.1 is not linked against liblua which I kind of expected:

readelf -d /usr/bin/lua-5.1.1 | grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libm.so.0]
 0x00000001 (NEEDED)                     Shared library: [libdl.so.0]
 0x00000001 (NEEDED)                     Shared library: [libreadline.so.5]
 0x00000001 (NEEDED)                     Shared library: [libc.so.0]

I noticed because since the overall size increased to the double when I actually expected it to shrink. (I use lua for size reasons)
Comment 49 Natanael Copa 2006-11-20 01:28:17 UTC
Is it correct that LIB_VERSION = 6:1:1 and not LIB_VERSION = 5:1:1 in the makefile patch?
Comment 50 Matti Bickel (RETIRED) gentoo-dev 2006-11-20 14:54:53 UTC
For the first one: i *knew* i wanted to rip something off debians patches *gna*, but here's some of the discussion:
http://article.gmane.org/gmane.comp.lang.lua.general/18519
Seems like we won't have luac linked against liblua, as i'm not prepared to rip all the __attribute__((visibility="hidden")) out of upstreams code.

For the second: yes, that's kind of a hack, since i knew no other way to produce that .so.5.1.1 :)
Yes it's dumb, and if you know better, please tell me.

I'll add a wrapper to symlink lua[c]-$PV with lua[c] tomorrow, to make this package usable again. Before i do that however - is a slotted lua a thing worth having?
Comment 51 Dennis Schridde 2006-11-20 15:51:47 UTC
Doesn't libtool have this flag (instead of -version-info) to do this kind of hack internally? (I don't remember the name, though.)
Comment 52 Natanael Copa 2006-11-20 23:30:36 UTC
(In reply to comment #50)
> For the first one: i *knew* i wanted to rip something off debians patches
> *gna*, but here's some of the discussion:
> http://article.gmane.org/gmane.comp.lang.lua.general/18519

They say that 150K (for every app linked against lua) is better than losing 15-20% speed. This is generally true, but in embedded this makes difference. I would like to have a "minimal" use flag that will make it small and sacrifice the speed for it.

Comment 53 Natanael Copa 2006-11-20 23:36:53 UTC
Created attachment 102444 [details, diff]
files/lua-5.1.1-make_dynamic.patch

Patch to link lua dynamically (but not luac). Could be used together with USE=minimal as showed below.

--- lua-5.1.1.ebuild.orig       2006-11-21 07:22:42 +0000
+++ lua-5.1.1.ebuild    2006-11-21 07:27:57 +0000
@@ -11,7 +11,7 @@
 LICENSE="MIT"
 SLOT="0"
 KEYWORDS="~ppc ~x86"
-IUSE="readline"
+IUSE="minimal readline"
 
 RDEPEND="readline? ( sys-libs/readline )"
 DEPEND="${RDEPEND}"
@@ -28,6 +28,13 @@
        if ! use readline ; then
                epatch "${FILESDIR}"/${P}-readline.patch
        fi
+
+       # Using dynamic linked lua is not recommended upstream for performance
+       # reasons. http://article.gmane.org/gmane.comp.lang.lua.general/18519
+       # We still allow it for people who care more about size.
+       if use minimal ; then
+               epatch "${FILESDIR}"/${P}-make_dynamic.patch
+       fi
 }
 
 src_compile() {
Comment 54 Matti Bickel (RETIRED) gentoo-dev 2006-11-21 06:49:39 UTC
Well, it's not just embedded. The speed argument plain doesn't hit arches w/ plenty of GPRs.
So i'll go the other route, add a static useflag for lua and make lua build the interpreter statically if a user so desires.

Wrapper coming up later today.
Comment 55 Dennis Schridde 2006-11-21 08:34:44 UTC
Did I get this right? I can now choose whether to compile Lua as a static lib or not, depending on a USEflag? That's nice, thanks!
Comment 56 Matti Bickel (RETIRED) gentoo-dev 2006-11-21 12:26:48 UTC
Uh well, not yet ;)
I'll think about it. Currently only the lua interpreter is statically linked to the dynamic library liblua.
Since upstream encourages static librarys too, it would indeed make sense to build statically here, too.
Comment 57 Natanael Copa 2006-11-22 00:28:47 UTC
(In reply to comment #56)
> Uh well, not yet ;)
> I'll think about it. Currently only the lua interpreter is statically linked to
> the dynamic library liblua.
> Since upstream encourages static librarys too, it would indeed make sense to
> build statically here, too.
> 

Thats what I thought too. So by default it would build statically. However, in some situations you might want to build the interpreter dynamically anyway and ignore what they recommend upstream. So I was thinking USE=minimal or USE=dynamic.

The lua compiler (luac) could be built statically anyway to avoid removing the "__attribute__((visibility("hidden")))" as mentionen in mailing list. It would be a middle way.
Comment 58 Matti Bickel (RETIRED) gentoo-dev 2006-11-22 11:30:30 UTC
To make my plans more clear:
currently we have a USE="static". This controls whether you like to build lua (the interpreter) statically (eg linking to liblua.so with -static enabled). The default now is to build the interpreter dynamically to be more consistent with Gentoos overall preference of shared objects.
What i'm now thinking about is letting USE="static" control building the library itself as static. This is an interesting thing, but i've to speak with twp and the others devs using lua in their ebuilds first, as nobody would like ion3 failing to build b/c it expects a shared liblua...
Comment 59 Dmitry S. Kulyabov 2007-01-03 02:25:49 UTC
Created attachment 105276 [details]
lua-5.1.1 ebuild w/ pkg-config; libdir fixed
Comment 60 Matti Bickel (RETIRED) gentoo-dev 2007-01-03 15:24:23 UTC
I definitly encourage further submissions of ebuild code. But what's the point of this new ebuild? libdirs already gets set correctly with RPATH="/usr/$(get_libdir)/" and pkg-config is already installed from upstream (i'll look into that prefix thing though)

A more interesting point keeps coming up: with all the failures of apps linking against lua and expecting liblualib to be there (it has vanished upstream, and there's nothing i can/will do about it), i consider slotting lua so apps would fall back on lua-5.0* if needed. I'll try to work out details tomorrow.
Comment 61 Matti Bickel (RETIRED) gentoo-dev 2007-01-19 15:37:50 UTC
After bringing lua-5.1.1-r1 (the slotted glory) to the tree, i'm wondering what will actually benefit from it. I'm currently wading through the list of luas reverse deps and will remove the slotted lua if nothing really *needs* the old libs. The hassle of teaching every dep to play nice with the slotted versions are just not worth it. If i happen to stumble over a genuine dep will not be solved by upgrading/revbumping, i'll stay on with the slotted builds and put more work in to bring them to a usable state.
Comment 62 Vadim Zborovskii 2007-01-23 08:28:06 UTC
What about amd64? The keyword is still missing. I can test it if you want to.
Comment 63 Vadim Zborovskii 2007-01-23 09:30:21 UTC
Created attachment 107879 [details]
unpacking fails with USE=static
Comment 64 Vadim Zborovskii 2007-01-23 09:32:05 UTC
Without "static" useflag, compiles, tests, installs & merges successfully on amd64. I suggest adding ~amd64 keyword.
Comment 65 Vadim Zborovskii 2007-01-23 11:17:45 UTC
Created attachment 107888 [details, diff]
modified make_static patch that works with new lua-5.1.1 ebuild

I suggest modified version of make_static patch. Mainly line numbers and patch context have changed.
Comment 66 Matti Bickel (RETIRED) gentoo-dev 2007-01-23 17:38:57 UTC
Mh, i was sure that i posted to this bug o_O
The issue is 'fallout' from the slotted release, where it uses $(LUA_T) and $(LUAC_T), which are in fact defined in the original Makefile. I adapted this style to make it look more like upstreams :)

In any way: should be fixed now.
Comment 67 Vadim Zborovskii 2007-01-28 09:49:42 UTC
Ok, waiting for fixed ebuild :)
Comment 68 Johan Bergström 2007-03-10 10:23:23 UTC
dev-lang/lua-5.1.1 has been in the tree for a while now, any reason to keep this bug open?
Comment 69 Matti Bickel (RETIRED) gentoo-dev 2007-03-10 14:58:31 UTC
Other than displaying a lack of progress on lua issues? No ;)