As I mentioned before in the other thread, I have been having a really hard time trying to figure out how to get Lua modules to load successfully from inside the Lua plugin.
A few of them will load, but most of the time I end up with "undefined reference" errors when the module tries to call some C function from the Lua library.
After trying just about every combination of linker flags (and a few impossible ones) re-compiling, re-installing, and re-starting, again and again, all without any luck, I finally stumbled upon the solution - if I remove the G_MODULE_BIND_LOCAL flag from g_module_open() in plugins.c, everything works just fine.
But it might not be a good idea to let any and all plugins have access to the global namespace like that, so I wonder if there could be some way for a plugin to request this, maybe something like this...
Open the plugin normally, using the G_MODULE_BIND_LOCAL flag, and check for the existence of some symbol, e.g.
make_me_global=TRUE;
If that symbol is NOT found, or if it evaluates to FALSE, then continue on loading the plugin as always. But if the symbol exists and is true, then close the module and re-open it *without* the G_MODULE_BIND_LOCAL flag.
That should keep the performance hit to a minimum, and still keep compatibility with existing plugins.
Of course, plugins should only enable this if they really need it, and the ones that do should take some effort to avoid interfering with each other. ( Using static symbols when possible, and trying to use some hopefully unique naming prefix, etc. )
If you agree to this, I will try to work up a patch.
What do you think?
- Jeff
On Sun, 13 Jan 2008 22:30:51 -0600 "Jeff Pohlmeyer" yetanothergeek@gmail.com wrote:
[...]
After trying just about every combination of linker flags (and a few impossible ones) re-compiling, re-installing, and re-starting, again and again, all without any luck, I finally stumbled upon the solution - if I remove the G_MODULE_BIND_LOCAL flag from g_module_open() in plugins.c, everything works just fine.
But it might not be a good idea to let any and all plugins have access to the global namespace like that, so I wonder if there could be some way for a plugin to request this, maybe something like this...
Open the plugin normally, using the G_MODULE_BIND_LOCAL flag, and check for the existence of some symbol, e.g.
make_me_global=TRUE;
If that symbol is NOT found, or if it evaluates to FALSE, then continue on loading the plugin as always. But if the symbol exists and is true, then close the module and re-open it *without* the G_MODULE_BIND_LOCAL flag.
[...]
We needed to add it in r1734: "Use G_MODULE_BIND_LOCAL for plugins to prevent symbol shadowing by other modules, and to help detect unresolved symbols at loading time."
IIRC without it calling info() from within a plugin when > 1 plugins were loaded without G_MODULE_BIND_LOCAL one of the plugins would get the wrong plugin name.
I don't think we should allow any loading without it, because things can silently go wrong.
Regards, Nick
On Jan 14, 2008 7:50 AM, Nick Treleaven nick.treleaven@btinternet.com wrote:
On Sun, 13 Jan 2008 22:30:51 -0600 "Jeff Pohlmeyer" yetanothergeek@gmail.com wrote:
If that symbol is NOT found, or if it evaluates to FALSE, then continue on loading the plugin as always. But if the symbol exists and is true, then close the module and re-open it *without* the G_MODULE_BIND_LOCAL flag.
We needed to add it in r1734: "Use G_MODULE_BIND_LOCAL for plugins to prevent symbol shadowing by other modules, and to help detect unresolved symbols at loading time."
IIRC without it calling info() from within a plugin when > 1 plugins were loaded without G_MODULE_BIND_LOCAL one of the plugins would get the wrong plugin name.
Yes, I remember that conversation, but I thought we decided there wasn't any real-world reason why a plugin would need to call its own info() function.
I don't think we should allow any loading without it, because things can silently go wrong.
I really wish you would reconsider it, the current situation leaves me with the temptation of trying to cram more and more functionality into the plugin to compensate for its lack of dynamic module support, and it leaves script writers in the position of being stuck with whatever feature set I think they need.
( Not to mention explaining to users why SciTE is able to load Lua modules, but Geany is not. :-)
If we can support dynamic Lua modules, the whole universe opens up -- ready-made support for expat, pcre, libtidy, networking, etc...
- Jeff
On Mon, 14 Jan 2008 08:27:33 -0600 "Jeff Pohlmeyer" yetanothergeek@gmail.com wrote:
On Jan 14, 2008 7:50 AM, Nick Treleaven nick.treleaven@btinternet.com wrote:
We needed to add it in r1734: "Use G_MODULE_BIND_LOCAL for plugins to prevent symbol shadowing by other modules, and to help detect unresolved symbols at loading time."
IIRC without it calling info() from within a plugin when > 1 plugins were loaded without G_MODULE_BIND_LOCAL one of the plugins would get the wrong plugin name.
Yes, I remember that conversation, but I thought we decided there wasn't any real-world reason why a plugin would need to call its own info() function.
But that was just one example of a name conflict. There could be other name conflicts if more than one plugin was loaded without G_MODULE_BIND_LOCAL.
I don't think we should allow any loading without it, because things can silently go wrong.
I really wish you would reconsider it, the current
I'm not saying we definitely can't do it, just that I'm against it. Apart from naming conflicts, G_MODULE_BIND_LOCAL also prevents loading a plugin that tries to access an unresolved symbol. This can easily happen when including a header that has an extern variables or function declarations. Untested code would cause Geany to segfault.
I don't have much experience with dynamic modules. Is there really no other way that the Lua plugin can load those modules for itself only? Surely they shouldn't be shared with Geany and other plugins. Or maybe I'm missing something?
Regards, Nick
On Jan 14, 2008 10:57 AM, Nick Treleaven nick.treleaven@btinternet.com wrote:
But that was just one example of a name conflict. There could be other name conflicts if more than one plugin was loaded without G_MODULE_BIND_LOCAL.
Of course, any module that needs this ability should make some effort to avoid using symbol names that are likely to clash with other plugins.
The main problem is with the plugin interface symbols: cleanup configure geany_callbacks geany_data info init version_check
But I think there is a fairly simple way to avoid this, instead of checking for the existence a symbol like:
make_me_global = TRUE;
We could check for something like this:
symbol_suffix = "_geanylua";
Now, we open the file *with* the G_MODULE_BIND_LOCAL flag, and if we find the symbol_suffix variable, we also check to make sure the normal interface symbol names ( init, info, cleanup, etc ) must *not* exist in this plugin. If it passes all these tests, we store the value of the symbol_suffix variable, close the plugin, and we should now be able to safely re-open it in the global namespace. Only this time, when we initialize the plugin, we append this symbol_suffix string to search for its internal symbol names, e.g. init_geanylua, info_geanylua, cleanup_geanylua, etc.
Apart from naming conflicts, G_MODULE_BIND_LOCAL also prevents loading a plugin that tries to access an unresolved symbol.
But since we first open the plugin with this flag, if that fails, we certainly wouldn't try to re-open it without the flag.
Untested code would cause Geany to segfault.
I'll bet I can think of at least a thousand other ways for an untested plugin to crash Geany. :-)
I don't have much experience with dynamic modules.
That makes two of us.
Is there really no other way that the Lua plugin can load those modules for itself only?
If anyone knows the answer to this, I would love to hear it, but I have already tried everything I could think of, and nothing else worked.
- Jeff
On Mon, 14 Jan 2008 22:40:21 -0600 "Jeff Pohlmeyer" yetanothergeek@gmail.com wrote:
On Jan 14, 2008 10:57 AM, Nick Treleaven nick.treleaven@btinternet.com wrote:
But that was just one example of a name conflict. There could be other name conflicts if more than one plugin was loaded without G_MODULE_BIND_LOCAL.
Of course, any module that needs this ability should make some effort to avoid using symbol names that are likely to clash with other plugins.
[...]
Well, that might work, but it makes the plugin code quite a bit more complex.
Instead of adding complexity to the plugin API with symbol prefixes, maybe we could support this by only allowing one plugin to be loaded without G_MODULE_BIND_LOCAL. Would that work?
Apart from naming conflicts, G_MODULE_BIND_LOCAL also prevents loading a plugin that tries to access an unresolved symbol.
But since we first open the plugin with this flag, if that fails, we certainly wouldn't try to re-open it without the flag.
Good point ;-)
[...]
Is there really no other way that the Lua plugin can load those modules for itself only?
If anyone knows the answer to this, I would love to hear it, but I have already tried everything I could think of, and nothing else worked.
This would be the ideal way, but maybe it's not possible.
Another thing, could you check how other programs with plugins do this? Pidgin has bindings for Perl and maybe some other languages, do they use G_MODULE_BIND_LOCAL? (I'm not sure SciTE is a good example, does it have a plugin interface?)
Regards, Nick
On Tue, 15 Jan 2008 13:01:24 +0000, Nick Treleaven nick.treleaven@btinternet.com wrote:
[...]
Is there really no other way that the Lua plugin can load those modules for itself only?
If anyone knows the answer to this, I would love to hear it, but I have already tried everything I could think of, and nothing else worked.
This would be the ideal way, but maybe it's not possible.
Another thing, could you check how other programs with plugins do this? Pidgin has bindings for Perl and maybe some other languages, do they use G_MODULE_BIND_LOCAL? (I'm not sure SciTE is a
They do. Look at http://developer.pidgin.im/viewmtn/revision/file/2a9420daed1d25d2d80d951ddc9... and look for purple_plugin_probe().
Unfortunately, I don't have any good idea for a solution. @Jeff: do you read any Lua related mailing lists or forums where you could ask for any hints?
Regards, Enrico
I have an idea for something that might work, with little or no changes to Geany itself. But it could be easier to implement than to explain. I will get back here with the results when I find out...
- Jeff
On Jan 16, 2008 4:37 PM, Jeff Pohlmeyer yetanothergeek@gmail.com wrote:
I have an idea for something that might work, with little or no changes to Geany itself.
It worked!
Basically, I just moved most of the code into a second library, so now the geanylua plugin is just a thin wrapper that loads the second lib, bypassing the G_MODULE_BIND_LOCAL flag.
Should have it ready for release soon...
Thanks for your input,
- Jeff