The Shared Namespace and You

ixokai
Posts: 63
Joined: Fri Dec 25, 2009 7:43 am

Re: The Shared Namespace and You

Post by ixokai »

ixokai wrote:On the mac for example, it will be VERY difficult to support a Lua search path which can easily find any sort of external modules-- I can go into why if you'd like, but its a real issue.
Hm, I retract that claim. See, I had to change the default search paths to:

Code: Select all

#define LUA_ROOT	"../Frameworks/lua/"
#define LUA_LDIR	LUA_ROOT "share/5.1/"
#define LUA_CDIR	LUA_ROOT "lib/5.1/"
#define LUA_PATH_DEFAULT  \
		"./?.lua;"  LUA_LDIR"?.lua;"  LUA_LDIR"?/init.lua;" \
		            LUA_CDIR"?.lua;"  LUA_CDIR"?/init.lua"
#define LUA_CPATH_DEFAULT \
	"./?.dylib;"  LUA_CDIR"?.dylib;" LUA_CDIR"loadall.dylib"
Because its imperative that lua be 'relocatable' on the mac on a whim (e.g., users must be able to move it to a new directory at any point), and look for its required libs not in any of the standard/system places, and it must not require any sort of installation. So the search path has to be solely within the app itself, relative to the executable, or relative to the current user's home directory.

It seems I could, in the batch script which starts Lua, set the LUA_PATH environment variable to replace that all and add "~/Library/Application Support/Mudlet/Scripts", if I wanted to. But having to keep half of my 'system' I'm releasing in one place, and half in another, is still IMHO a vastly worse solution then potential collision, especially when the latter can be mitigated successfully.

--Ix

davidk
Posts: 13
Joined: Wed Dec 09, 2009 7:01 pm
Location: avalon.mud.de

Re: The Shared Namespace and You

Post by davidk »

tobiassjosten wrote:
davidk wrote:Lua modules are tables which contain all the functions and variables, so in the end the result is the same.
No. The distinction is that Lua modules, if done properly, does not pollute the users' namespace.
Read the documentation for the module function. This function creates a table and uses setfenv to set this table as the environment for the file currently run. So everything you define in this file goes into the new table. However, this causes problems when you want to access the global namespace from your module.

If you are careful to put all the stuff into a table on your own and return that table at the end of your file, you get exactly the same result without the trouble caused by changing the environment of the file.

User avatar
demonnic
Posts: 883
Joined: Sat Dec 05, 2009 3:19 pm

Re: The Shared Namespace and You

Post by demonnic »

yeah, I'm using demonnic_blah for my personal items... everything I'm doing is likely to go through a major refactoring soon, though... 1.) to be sure and 2.) because crucible is looking awesome and should be made use of.

tobiassjosten
Posts: 5
Joined: Sat Jan 02, 2010 1:17 am

Re: The Shared Namespace and You

Post by tobiassjosten »

davidk wrote:Read the documentation for the module function. This function creates a table and uses setfenv to set this table as the environment for the file currently run. So everything you define in this file goes into the new table. However, this causes problems when you want to access the global namespace from your module.

If you are careful to put all the stuff into a table on your own and return that table at the end of your file, you get exactly the same result without the trouble caused by changing the environment of the file.
I suggest having a look at the documentation as well. You can easily pull whatever is needed, into the scope of the module by assigning local variables before calling module(). Thereby you can have any functionality or data from the global space, inside your module.

Whatever dependency can't be pulled in, be it from race condition or other reason, can be injected by use of mutators.

As for the table return idea - that still pollutes the user's global scope. Sure, the user can assign it to an arbitrary named variable but you still have the variable you created. Its upstream is also still the global scope, and so if you are planning on using anything but local variables, you are further polluting. While your idea mitigates the symptoms some, it really doesn't solve anything.

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: The Shared Namespace and You

Post by Heiko »

Tobias, I can see the point that you are trying to make. Mudlet would need to compile all Mudlet objects related scripts into the module namespace. In other words, we'd need dynamic plugin loading on demand i.e. when a module is being referenced somewhere in a script, Mudlet needs to load the package xml file again and compile the Mudlet object related scripts of the respective module into the namespace of the module.

Such a change means a good deal of work, but it's certainly possible. Imho the discussion should focus on: Is it *really* necessary, given the relatively small number of available modules of general interest compared to the high flexibility of the currently recommended package naming conventions. If you look at Mushclient the situation is similar. There aren't many plugins of general interest - most likely we'll soon have more Mudlet packages of general interest than Mush plugins, but even then I can't see why package authors shouldn't be able to handle this easily. Even if their preferred package prefix name has been taken, they can still run a 30 seconds regex replace job on the xml file to change the prefix.

Why do we need to have modules?

davidk
Posts: 13
Joined: Wed Dec 09, 2009 7:01 pm
Location: avalon.mud.de

Re: The Shared Namespace and You

Post by davidk »

tobiassjosten wrote:I suggest having a look at the documentation as well.
Ok, here it is: (http://www.lua.org/manual/5.1/manual.html#pdf-module) (Emphasis added by me).
module (name [, ···])

Creates a module. If there is a table in package.loaded[name], this table is the module. Otherwise, if there is a global table t with the given name, this table is the module. Otherwise creates a new table t and sets it as the value of the global name and the value of package.loaded[name]. This function also initializes t._NAME with the given name, t._M with the module (t itself), and t._PACKAGE with the package name (the full module name minus last component; see below). Finally, module sets t as the new environment of the current function and the new value of package.loaded[name], so that require returns t.

If name is a compound name (that is, one with components separated by dots), module creates (or reuses, if they already exist) tables for each component. For instance, if name is a.b.c, then module stores the module table in field c of field b of global a.

This function can receive optional options after the module name, where each option is a function to be applied over the module.

require (modname)

Loads the given module. The function starts by looking into the package.loaded table to determine whether modname is already loaded. If it is, then require returns the value stored at package.loaded[modname]. Otherwise, it tries to find a loader for the module.

To find a loader, require is guided by the package.loaders array. By changing this array, we can change how require looks for a module. The following explanation is based on the default configuration for package.loaders.

First require queries package.preload[modname]. If it has a value, this value (which should be a function) is the loader. Otherwise require searches for a Lua loader using the path stored in package.path. If that also fails, it searches for a C loader using the path stored in package.cpath. If that also fails, it tries an all-in-one loader (see package.loaders).

Once a loader is found, require calls the loader with a single argument, modname. If the loader returns any value, require assigns the returned value to package.loaded[modname]. If the loader returns no value and has not assigned any value to package.loaded[modname], then require assigns true to this entry. In any case, require returns the final value of package.loaded[modname].

If there is any error loading or running the module, or if it cannot find any loader for the module, then require signals an error.
tobiassjosten wrote:As for the table return idea - that still pollutes the user's global scope. Sure, the user can assign it to an arbitrary named variable but you still have the variable you created. Its upstream is also still the global scope, and so if you are planning on using anything but local variables, you are further polluting. While your idea mitigates the symptoms some, it really doesn't solve anything.
As I have quoted from the manual above, the module function creates a new table and puts the table under the given global name. So it "pollutes" the global scope as well. I recommend to return the table as then the require function can put it into package.loaded table (which otherwise the module function would do).

Returning a table and using the module function is almost identical (except for the setfenv). You don't have to believe me, but I guess Mudlet won't be using the default Lua package system for Mudlet packages anyway.

tobiassjosten
Posts: 5
Joined: Sat Jan 02, 2010 1:17 am

Re: The Shared Namespace and You

Post by tobiassjosten »

Heiko wrote:Why do we need to have modules?
You should have modules because this would make it easier to distribute isolated functionality. A queue system, map handling, notification system, anti-illusion helpers, etc. Handy things like the Lua serializer that Nick Gammon wrote and ships with MUSHclient is another example. This will make it possible for Lua programmers to build a toolbox for other system developers to use, and I am certain this will benefit the client greatly in the long run.

Another pro is that I could then easily port my Protea project to Mudlet. ;) It's built around modules to work with as many clients as possible.
davidk wrote:Returning a table and using the module function is almost identical (except for the setfenv). You don't have to believe me, but I guess Mudlet won't be using the default Lua package system for Mudlet packages anyway.
I think you are missing a crucial part of the recipe. If you use module(...), then the module is completely agnostic to its name. It assumes the name you give it and populates the package table accordingly. The difference between this and your proposal is still that your way sets a global, "uncontained" variable AND a user defined one. "My" module(...) way only sets a user defined variable.

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: The Shared Namespace and You

Post by Heiko »

1. You fail to explain *why* we'd need modules.
2. You fail to discuss your module idea in the context of client objects (triggers, aliases, buttons, keys etc.) as they are the most important part of any real world package.
3. We have our own serialize functions. What would Nick Gammon's serialize function offer that isn't covered by our own version?
4. Scanning over your Protea project I wonder why a Mudlet user should use such a system that doesn't make use of any of Mudlet's built in functionality and does its very own low level parsing in Lua to reinvent the wheel in slow motion? How do you think your system's performance would fare against a real Mudlet package that makes proper use of the client's native C++ trigger engine? As soon as your system gets bigger you'll find out what I'm talking about especially in the face of the spreading use of MXP and the resulting extreme overhead of non-print data. IMHO your system should either make proper use of its supported clients' native objects or stick to a black box solution on some proxy client. IMHO a Mudlet, Mush or CMud user has little use for a black box system that he cannot modify with the client's user interface.

tobiassjosten
Posts: 5
Joined: Sat Jan 02, 2010 1:17 am

Re: The Shared Namespace and You

Post by tobiassjosten »

Heiko wrote:1. You fail to explain *why* we'd need modules.
tobiassjosten wrote:You should have modules because this would make it easier to distribute isolated functionality.
Heiko wrote:2. You fail to discuss your module idea in the context of client objects (triggers, aliases, buttons, keys etc.) as they are the most important part of any real world package.
Modules are, IMO, better suited for "isolated functionality". Tools in a toolbox, if you want. Full blown systems might benefit from another distribution model. Allthough that should be pluggable with the modules.

In any case, I'd advocate building a proper API for controlling any aspect of the client with Lua and without having to use an "in client editor".
Heiko wrote:3. We have our own serialize functions. What would Nick Gammon's serialize function offer that isn't covered by our own version?
See that word, example, I used there? Yeah.
Heiko wrote:4. Scanning over your Protea project I wonder why a Mudlet user should use such a system that doesn't make use of any of Mudlet's built in functionality and does its very own low level parsing in Lua to reinvent the wheel in slow motion? How do you think your system's performance would fare against a real Mudlet package that makes proper use of the client's native C++ trigger engine? As soon as your system gets bigger you'll find out what I'm talking about especially in the face of the spreading use of MXP and the resulting extreme overhead of non-print data. IMHO your system should either make proper use of its supported clients' native objects or stick to a black box solution on some proxy client. IMHO a Mudlet, Mush or CMud user has little use for a black box system that he cannot modify with the client's user interface.
Protea does support client specific functionality (adapters). And if there is an aspect you feel is left out in this regard, feel free to submit patches and/or pull requests on GitHub.

As for using Lua to parse I/O. There is little disagreement that C++ is (much) faster than Lua. However, even with tens of thousands of trigger patterns and hundreds of lines to match against per second, this is not a problem. Lua pattern matching handles this just fine.

Parsing in .00002s or .002s sure is a magnitude of 100, but it's still a difference you will never, ever notice. So why over optimize?


This debate has been a blast, really, but this is clearly not going anywhere. So I'll wrap my enagement up with this post. Best of luck people!

User avatar
Heiko
Site Admin
Posts: 1548
Joined: Wed Mar 11, 2009 6:26 pm

Re: The Shared Namespace and You

Post by Heiko »

tobiassjosten wrote: As for using Lua to parse I/O. There is little disagreement that C++ is (much) faster than Lua. However, even with tens of thousands of trigger patterns and hundreds of lines to match against per second, this is not a problem. Lua pattern matching handles this just fine.

Parsing in .00002s or .002s sure is a magnitude of 100, but it's still a difference you will never, ever notice. So why over optimize?
We'll talk again when your system has been finished.

The discussion would have been more productive if you had actually addressed the points made in this thread instead of repeating your mantra.

Post Reply