Update: lua 5.3.0

[Schamschula (Marius Schamschula)]

lua has been updated to version 5.3.0: Its main new features are support for integers, bitwise operators, and a basic utf-8 library.

[rmerpes]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

And how backward-compatible is lua 5.3 with lua 5.2? Because that's been a problem with major lua releases in the past, to the extent that I've wanted there to be separate ports for each major lua version. We already have ports lua, lua50 and lua51; I propose we also have lua52 and lua53, make the lua port replaced_by lua53, and make sure all these ports install to similar but nonconflicting locations.

[Schamschula (Marius Schamschula)]

Replying to ryandesign@…:

And how backward-compatible is lua 5.3 with lua 5.2? Because that's been a problem with major lua releases in the past, to the extent that I've wanted there to be separate ports for each major lua version. We already have ports lua, lua50 and lua51; I propose we also have lua52 and lua53, make the lua port replaced_by lua53, and make sure all these ports install to similar but nonconflicting locations.

Agreed. I've updated all lua port files regarding conflicts.

[ryandesign (Ryan Carsten Schmidt)]

See also #46880.

[ryandesign (Ryan Carsten Schmidt)]

Replying to mschamschula@…:

Agreed. I've updated all lua port files regarding conflicts.

What we would want to do is make the ports not conflict.

[Schamschula (Marius Schamschula)]

Updated lua52 and lua53 Portfiles to use the correct distname.

[Schamschula (Marius Schamschula)]

Updated patch file for lua52

[hadrielk@…]

Replying to ryandesign@…:

Replying to mschamschula@…:

Agreed. I've updated all lua port files regarding conflicts.

What we would want to do is make the ports not conflict.

I think if port "lua" becomes Lua 5.3, you'll have to verify all other dependent ports still work in deployment, or change them to use "lua52". For example, wireshark-devel will not compile with Lua 5.3. Lua 5.3 is not API/ABI compatible with Lua 5.2. (nor was 5.2 with 5.1, but the differences are even bigger with 5.2->5.3)

Really the differences between Lua X.Y versions are like Python 2 vs. Python 3.

[mojca (Mojca Miklavec)]

Cc Me!

[mojca (Mojca Miklavec)]

An unexpected and untested upgrade has been done in r140191 with a few modifications later added by r140234. We should fix the situation as soon as possible.

See also:

• #48826 for gnuplot
• #48835 for wireshark

We need to fix the problem ASAP.

[m7.thon@…]

The update to lua 5.3 also breaks the following ports:

• #48873 for ipe
• #48874 for mpv

[m7.thon@…]

Cc Me!

[mojca (Mojca Miklavec)]

I committed r140262 with a new subport lua52.

Now one important remark: while the port works as-is and does not conflict with lua per se, any port that uses

#include <lua.h>

and tries to use -L/opt/local/include -L/opt/local/include/lua-5.2 flags will pick up lua.h from lua, not from lua52. I'm thinking of trying to temporarily deactivate lua before compiling the port and then activating lua again after compilation is done, but I need to remember how to do that.

If anyone feels like improving the port, feel free, but I had to have something to start with due to a flood of bug reports. The revision increase for lua was because I added some files to the documentation folder that were initially missing (some png icon and css styles).

[mojca (Mojca Miklavec)]

Just an additional note, here's how I did a quick fix for gnuplot:

@@ -251,7 +251,7 @@ LUA_GP_int_error(lua_State *L) {
       msg = luaL_checkstring(L, 1);
       break;
     case 2:
-      t_num = luaL_checkint(L, 1);
+      t_num = luaL_checkinteger(L, 1);
       msg  = luaL_checkstring(L, 2);
       break;
     default:
@@ -285,7 +285,7 @@ LUA_GP_int_warn(lua_State *L) {
       msg = luaL_checkstring(L, 1);
       break;
     case 2:
-      t_num = luaL_checkint(L, 1);
+      t_num = luaL_checkinteger(L, 1);
       msg = luaL_checkstring(L, 2);
       break;
     default:

[m7.thon@…]

Replying to mojca@…:

... any port that uses

#include <lua.h>

and tries to use -L/opt/local/include -L/opt/local/include/lua-5.2 flags will pick up lua.h from lua, not from lua52. I'm thinking of trying to temporarily deactivate lua before compiling the port and then activating lua again after compilation is done, but I need to remember how to do that.

This does not sound like a solution to me. How about putting all lua versions in their own subdirectory, i.e., also /opt/local/include/lua-5.3 ? And consequently have no port "lua" at all, but only the (sub-)ports lua50 lua51 lua52 and lua53 that can be installed in their respective subdirectories without conflicting?

Clearly, the ports depending on lua would all need to be modified, but at least this would be a consistent approach. Or how exact is the similar problem dealt with in the case of python?

[mojca (Mojca Miklavec)]

We could certainly remove the main lua port, but that has some drawbacks.

Python is a different beast (you don't get a working python binary for example, you only get python2.7), but wxWidgets are considerably more tricky to use as a consequence that there is no "default" installation for example.

We need some more discussion in any case.

[mojca (Mojca Miklavec)]

And just to clarify: I added the lua52 subport because that was most urgently needed, while I didn't want to delete the lua port itself without some discussion and sufficient testing.

We can certainly discuss whether those subports should be called lua-5.2 instead for example, or if we should at some point remove the main port. But as I mentioned, removing the main port has some drawbacks, so we should try to be careful about what we do.

[dbevans (David B. Evans)]

Although I think updating lua to version 5.3.0 was done without much thought to what might break, trying to support multiple versions of lua is adding complexity that really isn't called for as long as backwards compatibility support for both lua 5.1 and 5.2 is enable in the 5.3 build. While there are a few API items from 5.2 that were dropped from 5.3, the changes are minimal particularly for apps that are 5.2 compatible already and are easily handled as in the case of luaL_checkint as mojca has indicated. See the ​5.3 manual for details of known incompatibilities with 5.2. I don't think 5.1 and earlier is much of an issue as most upstream developers now support 5.2 and several have updated releases for 5.3 where necessary. I think the current lua50 port could be removed as there are no ports that depend on it and lua51 only has one, borwars which hasn't seen much development in a long time and its compatibility problems have to do with 5.1 vs 5.2. So the main issue to my mind is 5.2 verses 5.3.

Finally, note that lua extension modules need to be installed in a given lua's search path to be useful. Typically this is ${prefix}/lib/lua/${lua_version} for binary modules and ${prefix}/share/lua/${lua_version} for lua modules. So to fully support more than one version of lua, we would need to address multiple versions of these as well. Do we really want to get into another multiple version can of worms? I suggest we do as we have decided to do with perl and just support the latest stable version.

I suggest before proliferating multiple lua versios, we make a list of ports that seem to be broken with 5.3 to understand the magnitude of the problem and then see if we can understand what the various issues are. I would much rather work with the upstream developers to fix any 5.3 incompatibilities than try and support multiple lua versions. In short, I favor moving forward rather than standing still or moving backward.

As you are probably aware, the upgrade to 5.3 broke libquvi and libquvi-scripts which in turn broke playlist parsing in the GNOME video player, totem. This is now fixed to work with lua 5.3. Although libquvi and friends claim to only be 5.1 compatible, it took very few changes to get it all working again.

[mojca (Mojca Miklavec)]

I would be happier with a single working version and I support removing lua50 if there is no port that depends on it.

I'm almost sure that luajit is not going to support Lua 5.2 (or 5.3 or later versions for that matter) in any foreseeable future, but in that particular case I would actually suggest compiling lua as part of luajit compilation process. If luajit is the one port you were talking about, we could also get rid of lua51 then.

TeX Live's binary luajittex depends on LuaJIT, but it includes the sources, so there is no need for an external dependency.

I just remembered that one could easily work around the problem with "conflicting" ports by forcing a trace mode during compilation.

[tenomoto (Takeshi Enomoto)]

I'm sorry for committing the update to 5.3 as soon as it is not up-to-date and not maintained, without finding this ticket. I was not aware that Lua is used by a number of ports.

[ryandesign (Ryan Carsten Schmidt)]

lua is one of those languages that seems to break backward compatibility with each version. Not every lua-using program is affected, but it seems like some always are. This is exactly the kind of software that should have multiple simultaneously-installable (sub)ports so the user (or the other program) using lua can choose which version they want. It's the same reason why we offer multiple versions of python, ruby, php, clang, gcc.

Dave, I am definitely in favor of "moving forward" as you said above, just that I have a different definition of "moving forward" than you do. To me, "moving forward" is providing and maintaining the multiple versioned ports of lua, so that even old software can be built with lua support. Removing old lua ports would be moving backward, to me, in that it would break ports that require those old versions of lua. Even if there aren't many ports in MacPorts today that require old versions of lua, it's possible new ports will be added that require old lua, or that some of the ports that currently declare a dependency on just "lua" actually require an older version and have just not been tested with newer versions yet. (One example I'm currently working on: #39074.)

[Schamschula (Marius Schamschula)]

Backward compatibility with 5.2 can be enabled in src/luaconf.h. I manually did this on one of my machines, as lua 5.3.1 broke VLC and VLC-devel.

[m7.thon@…]

Let me get this straight. To enable (partial) backward compatibility with say lua 5.2 two things are required:

• lua 5.3 itself needs to be compiled with backward compatibility enabled by adding -DLUA_COMPAT_5_2 to the CFLAGS (and this is done in the macports version of lua)
• any port that uses deprecated lua 5.2 C-API calls needs to also compile with -DLUA_COMPAT_5_2 to get backwards compatibility (or it can be patched to use the current lua C-API)

Is this correct?

[Schamschula (Marius Schamschula)]

Added a patch to build lua 5.3.1 with 5.2 backward compatibility.

[mp@…]

Cc Me!

[mf2k (Frank Schima)]

lua is currently at version 5.3.5 so I'm closing this ticket.