Hello,
I'm (as of now) motivated to implement proxy plugins (to my amusement, until splitwindow2 gets finally reviewed...). Therefore I would like to receive comments on my proposed concept and APIs (further below).
I would also love to discuss about better terms for proxy and "the proxied plugins". I think some adorable internal code/nick names could help us to not get confused in this business.
Note that I have implemented very little yet, so there might be unknown roadblocks and things can easily change. Please also note that I intend to achieve the goals without breaking plugin API/ABI.
The concept:
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions) and a few hooks to call into the plugin when Geany needs it.
Geany will match files against these criterias and, on match, call into the plugin for the first time. This call should probe whether or not the plugin can actually handle the plugin. I think this is needed because a) criterias/file extensions might be ambiguous and b) the plugin might be written against an incompatible version of the proxy (think python 2 vs 3).
Then when Geany generates the plugin list (i.e. when the user opens plugin manager (PM) dialog) it'll call a second time into the proxy. This time the purpose is to load the actual plugin, and install plugin-specific init(), configure(), help() and cleanup() hooks. Unlike in classic plugins these hooks must call into proxy, where they have to be dispatched to call the actual plugin code (this is why we call them proxy right?).
Once loaded, Geany generally doesn't know/care about the difference between classic plugins and proxied plugins. This achieves the ultimate goal: That proxied plugins transparently integrate into the PM dialog and become first-class citizens.
Part of the proposed concept is to add a "fake proxy" for the builtin .so-file support. This enables to use the same code for all plugins (proxied or not). This should greatly reduce the maintenance effort. The fake proxy naturally must be compiled in but otherwise exposes the same APIs as actual proxies.
The APIs:
This propsal adds 3 major functions and alters the 4 existing plugin hooks.
plugin_register_proxy(GeanyPlugin *proxy_plugin, ProxyPluginInfo *info, gpointer user_data);
This is initially called by the proxy and lets Geany know about it. ProxyPluginInfo contains the criterias and pointers to the probe() and load() functions. user_data is proxy-specific and passed to probe().
gboolean (*probe)(const gchar *file, gpointer user_data, gpointer *plugin_data);
This is implemented in the proxy and should return true when it can handle the plugin, it basically replaces the existing utils_str_casecmp(G_MODULE_SUFFIX, ...) call. There is a catch though, while probe() should be as fast as possible, I expect that some proxies still need to load the file half-way or even in full to make that decision. In order to maintain that state, it can set the plugin_data pointer. This will be passed to the load function. This way loading the plugin again can be avoided.
gboolean (*load)(const gchar *file, gpointer user_data, gpointer *plugin_data);
The same function signature, but a different purpose. The proxy should now fully load the plugin specified with file such that is ready to run (it has the same signature in case it needs to load in probe() already, then this function can be the same as probe()).
As for the existing hooks, these need gain a parameter (gpointer plugin_data). Because the hooks can be indirect now, the proxy needs plugin_data to be able to properly dispatch the hooks on a per-plugin basis. I.e. the hooks become:
void (*init) (GeanyData *data, gpointer plugin_data); GtkWidget* (*configure) (GtkDialog *dialog, gpointer plugin_data); void (*configure_single) (GtkWidget *parent, gpointer plugin_data); void (*help) (gpointer plugin_data); void (*cleanup) (gpointer plugin_data);
Because the plugin_data parameter is exclusively for the proxy, the actual plugins do not receive it. They still implement the non-plugin_data variant. The parameter is added to the end so that these can still point to the module symbols for .so file plugins.
That's it so far. I think this proposal should allow to reach our goal (first class plugin scripts) while avoiding to break plugins ABI/API.
Best regards.
On Wed, 07 May 2014 08:40:13 +0200 Thomas Martitz kugel@rockbox.org wrote:
I'm (as of now) motivated to implement proxy plugins (to my amusement, until splitwindow2 gets finally reviewed...). Therefore I would like to receive comments on my proposed concept and APIs (further below).
Feasible, but... why? Looks a bit like GTypePlugin to me, with it's proxied "dynamic" classes, and nearly zero implementations.
Am 07.05.2014 19:59, schrieb Dimitar Zhekov:
On Wed, 07 May 2014 08:40:13 +0200 Thomas Martitz kugel@rockbox.org wrote:
I'm (as of now) motivated to implement proxy plugins (to my amusement, until splitwindow2 gets finally reviewed...). Therefore I would like to receive comments on my proposed concept and APIs (further below).
Feasible, but... why? Looks a bit like GTypePlugin to me, with it's proxied "dynamic" classes, and nearly zero implementations.
I'm not sure what GTypePlugin has to do with it. This isn't about loading new GObject classes dynamically (proxy plugins aren't GObjects). Perhaps you can elaborate a bit?
Best regards
On Wed, 07 May 2014 23:58:11 +0200 Thomas Martitz kugel@rockbox.org wrote:
Am 07.05.2014 19:59, schrieb Dimitar Zhekov:
On Wed, 07 May 2014 08:40:13 +0200 Thomas Martitz kugel@rockbox.org wrote:
I'm (as of now) motivated to implement proxy plugins (to my amusement, until splitwindow2 gets finally reviewed...). Therefore I would like to receive comments on my proposed concept and APIs (further below).
Feasible, but... why? Looks a bit like GTypePlugin to me, with it's proxied "dynamic" classes, and nearly zero implementations.
I'm not sure what GTypePlugin has to do with it. This isn't about loading new GObject classes dynamically (proxy plugins aren't GObjects).
It's not directly technically related, of course, just seems smiliar as a mechanism, and as something without purpose.
Perhaps you can elaborate a bit?
1. KISS.
2. Most plugins have UI. Loading plugins on demand means that the UI will change dynamically, which is hardly a reason for celebrate.
3. Why can't one simply mark the plugins (s)he needs statically? If there are any compatibility or resource usage problems, they should be fixed in the original code, instead of relying that Geany will work better by loading a smaller set of plugins on demand.
4. If we need dynamic loading (though I don't know of any reason for that), why not extend the standard plugin API with criterias, and let the "plugin enabled" checkbox in the PM dialog have a "half-checked" position for plugins with criterias? Why proxiyng?
5. How do we choose which sub-plugins of a proxy plugin should be enabled - a tree structure N levels deep? Obviously, one may not be satisfied with the activation criterias, and may want specific sub-plugins explicitly enabled or disabled.
It seems to me that your idea mixes dynamic loading and proxying. These are not actually related (except for the fact that the criterias, if implemented, should be appliable to the sub-plugins, if implemented).
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions)
Geany will match files against these criterias and, on match, call into the plugin for the first time.
What exactly will it match - the extensions from the files in the current session? So the PM dialog will contain a different list of plugins each time it's open? And yet different list on startup, since plugin_init() is before any files are loaded? Perpahs I don't get your idea at all.
Then when Geany generates the plugin list (i.e. when the user opens plugin manager (PM) dialog) it'll call a second time into the proxy. This time the purpose is to load the actual plugin, and install plugin-specific init(), configure(), help() and cleanup() hooks.
An what happens if the user simply doesn't open the PM dialog - the actual (sub-?)plugin is never loaded?..
I'm not against specific proxies - loading the interpreter or runtime Foo once, instead of a copy for each plugin written in/with Foo, makes sense. But that seems different from what you described, and the sub-plugin search and choosing should be done by the master Foo plugin, using the Foo language and/or runtime. Unless we are trying to build language bridges for the sake of it, there will hardly be more than two or three of those plugins.
Am 08.05.2014 19:25, schrieb Dimitar Zhekov:
On Wed, 07 May 2014 23:58:11 +0200 Thomas Martitz kugel@rockbox.org wrote:
Am 07.05.2014 19:59, schrieb Dimitar Zhekov:
On Wed, 07 May 2014 08:40:13 +0200 Thomas Martitz kugel@rockbox.org wrote:
I'm (as of now) motivated to implement proxy plugins (to my amusement, until splitwindow2 gets finally reviewed...). Therefore I would like to receive comments on my proposed concept and APIs (further below).
Feasible, but... why? Looks a bit like GTypePlugin to me, with it's proxied "dynamic" classes, and nearly zero implementations.
I'm not sure what GTypePlugin has to do with it. This isn't about loading new GObject classes dynamically (proxy plugins aren't GObjects).
It's not directly technically related, of course, just seems smiliar as a mechanism, and as something without purpose.
Perhaps you can elaborate a bit?
KISS.
Most plugins have UI. Loading plugins on demand means that the UI
will change dynamically, which is hardly a reason for celebrate.
- Why can't one simply mark the plugins (s)he needs statically? If
there are any compatibility or resource usage problems, they should be fixed in the original code, instead of relying that Geany will work better by loading a smaller set of plugins on demand.
- If we need dynamic loading (though I don't know of any reason for
that), why not extend the standard plugin API with criterias, and let the "plugin enabled" checkbox in the PM dialog have a "half-checked" position for plugins with criterias? Why proxiyng?
- How do we choose which sub-plugins of a proxy plugin should be
enabled - a tree structure N levels deep? Obviously, one may not be satisfied with the activation criterias, and may want specific sub-plugins explicitly enabled or disabled.
It seems to me that your idea mixes dynamic loading and proxying. These are not actually related (except for the fact that the criterias, if implemented, should be appliable to the sub-plugins, if implemented).
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions) Geany will match files against these criterias and, on match, call into the plugin for the first time.
What exactly will it match - the extensions from the files in the current session? So the PM dialog will contain a different list of plugins each time it's open? And yet different list on startup, since plugin_init() is before any files are loaded? Perpahs I don't get your idea at all.
What???
No, this isn't about loading plugins on demand. This is about implementing APIs so that plugins (probably selected in the PM dialog) can act as a proxy to plugins written in other languages (also selected in the PM dialog). I do not intend to change the user experience w.r.t. to plugins: All plugins, regardless of the language, are user-selected in the PM dialog and loaded only then and on subsequent startups.
Best regards.
On Fri, 09 May 2014 12:29:58 +0200 Thomas Martitz kugel@rockbox.org wrote:
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions) Geany will match files against these criterias and, on match, call into the plugin for the first time.
What???
No, this isn't about loading plugins on demand. This is about implementing APIs so that plugins (probably selected in the PM dialog) can act as a proxy to plugins written in other languages (also selected in the PM dialog). I do not intend to change the user experience w.r.t. to plugins: All plugins, regardless of the language, are user-selected in the PM dialog and loaded only then and on subsequent startups.
So then, the only remaining option seems that Geany will search the plugin paths, and match the found file extensions. It's unclear what call returns the sub-plugin metadata (name, version etc.), but that's solvable.
Unless we are trying to enable scripting in more than a few languages, I see no reason for all these complications. Let the master Foo plugin search for and probe the found .foo files, display a list of them, and save the list of selected ones in it's own configuration, without involving Geany. I'd like to have 100 useful python (lua, js, ...) scripts or macros, and will probably write my own, but certainly don't want them as first class plugins.
As another option, we can declare one scripting language as official, support the language binding for it, and provide the command console. Vim, emacs and gdb have one, so why should the light IDE Geany support N of them? And whoever wants to write a plugin to supports his or her favorite REXX/VBScript/younameit is still free to do so.
Personally I'd choose a language that supports GTK+, *nix and Windows, for obvious reasons.
Like I think Dimitar was saying, adding custom scripts sound like they should be second class plugins.
Sorry to be a broken record here, but my external tools plugin calls whatever language you can make an executable for or call from the command line: https://github.com/sblatnick/geany-plugins/tree/external-tools/external-tool...
I don't see the need for bindings for every language to tap into plugin functionality when you can call scripts and accept the scripts output with some accepted criteria, kinda like an API or like my plugin does. For instance, external-tools lets you output a program to the bottom panel or substitute currently selected text in geany, etc. While this limits the methods the plugin can interact, it supports an easy input and easy output logic.
I haven't been following the entire thread, and I don't mean to come across as continuously advertising my own plugin, but it seems to me that you are discussing integrating other languages to the plugin structure. In my opinion, this plugin already does that to a limited extent, but to an extent that meets my needs as I'm not looking to tweak UI from the plugin.
Just as I think Dimitar was saying, I don't think we want to overly complicate plugins because of wanting to support "everything" or every language for plugins. I hope changes to how we do plugins don't break existing plugins too much, as that was one of the reasons I left gedit. It's annoying rewriting plugins because of the changes.
Thanks,
Steve
On 05/09/2014 11:15 AM, Dimitar Zhekov wrote:
Unless we are trying to enable scripting in more than a few languages, I see no reason for all these complications. Let the master Foo plugin search for and probe the found .foo files, display a list of them, and save the list of selected ones in it's own configuration, without involving Geany. I'd like to have 100 useful python (lua, js, ...) scripts or macros, and will probably write my own, but certainly don't want them as first class plugins.
Am 09.05.2014 19:15, schrieb Dimitar Zhekov:
On Fri, 09 May 2014 12:29:58 +0200 Thomas Martitz kugel@rockbox.org wrote:
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions) Geany will match files against these criterias and, on match, call into the plugin for the first time.
What???
No, this isn't about loading plugins on demand. This is about implementing APIs so that plugins (probably selected in the PM dialog) can act as a proxy to plugins written in other languages (also selected in the PM dialog). I do not intend to change the user experience w.r.t. to plugins: All plugins, regardless of the language, are user-selected in the PM dialog and loaded only then and on subsequent startups.
So then, the only remaining option seems that Geany will search the plugin paths, and match the found file extensions. It's unclear what call returns the sub-plugin metadata (name, version etc.), but that's solvable.
That hasn't been covered in the proposal but should work similar to the 4 other hooks I mentioned.
Unless we are trying to enable scripting in more than a few languages, I see no reason for all these complications. Let the master Foo plugin search for and probe the found .foo files, display a list of them, and save the list of selected ones in it's own configuration, without involving Geany. I'd like to have 100 useful python (lua, js, ...) scripts or macros, and will probably write my own, but certainly don't want them as first class plugins.
So what you are proposing is the status-quo (exactly what geanypy does right now). Apart from the limitations it to those plugins (keybindings) it does not integrate the plugins nicely into the PM dialog.
I accept that the status-quo is fine for you, but not for me and others. IRC discussions repeatedly indicated that proxy plugins are the way to go (the alternative is automagic bindings through gobject introspection which isn't feasible right now). I'd like to see serious plugins in other languages, and not just small scripts/macros. And I'd like to attract new contributors by offering a lower language barrier.
And because the core is more or less in maintenance mode (new features take *ages* to get in, if ever), we need to embrace plugin support even more.
As another option, we can declare one scripting language as official, support the language binding for it, and provide the command console. Vim, emacs and gdb have one, so why should the light IDE Geany support N of them? And whoever wants to write a plugin to supports his or her favorite REXX/VBScript/younameit is still free to do so.
Personally I'd choose a language that supports GTK+, *nix and Windows, for obvious reasons.
I suspect we would support 1 (or at most 2) proxy plugins ourself. Other proxy plugins can be maintained by third parties out of tree or in geany-plugins. I do not want to hard-limit the language choice to our blessed one though.
Best regards
On Fri, 09 May 2014 20:34:18 +0200 Thomas Martitz kugel@rockbox.org wrote:
Am 09.05.2014 19:15, schrieb Dimitar Zhekov:
On Fri, 09 May 2014 12:29:58 +0200 Thomas Martitz kugel@rockbox.org wrote:
Unless we are trying to enable scripting in more than a few languages, I see no reason for all these complications. [...]
I accept that the status-quo is fine for you, but not for me and others. IRC discussions repeatedly indicated that proxy plugins are the way to go (the alternative is automagic bindings through gobject introspection which isn't feasible right now).
I don't know the way to go, but there are two things we should not do:
1. Change the interface radically, gedit-style.
2. Introduce serious code changes that won't benefit the users. Proxy plugins may be beneficial at some future point, or we may end up with several poorly-supported languages, and later with unmaintained proxies we dare not remove because that'll kill their sub-plugins.
I suspect we would support 1 (or at most 2) proxy plugins ourself.
I hope by "we" you mean the core plugins. Otherwise, that's no "support" at all.
Other proxy plugins can be maintained by third parties out of tree or in geany-plugins. I do not want to hard-limit the language choice to our blessed one though.
The other proxy plugins will be DOA, except for mini-plugins (macros, scripts) that suit single users. Why should one bother to write a serious, mass-distributable sub-plugin in a 3rd party language, when there are official ones? To risk the proxy plguin being unmaintained?
Am 10.05.2014 20:02, schrieb Dimitar Zhekov:
On Fri, 09 May 2014 20:34:18 +0200 Thomas Martitz kugel@rockbox.org wrote:
Am 09.05.2014 19:15, schrieb Dimitar Zhekov:
On Fri, 09 May 2014 12:29:58 +0200 Thomas Martitz kugel@rockbox.org wrote:
Unless we are trying to enable scripting in more than a few languages, I see no reason for all these complications. [...]
I accept that the status-quo is fine for you, but not for me and others. IRC discussions repeatedly indicated that proxy plugins are the way to go (the alternative is automagic bindings through gobject introspection which isn't feasible right now).
I don't know the way to go, but there are two things we should not do:
- Change the interface radically, gedit-style.
Do mean the GUI or API? If the latter, that's exactly what I want to avoid if possible. If the former, the proxy plugin work by itself doesn't add GUI changes.
- Introduce serious code changes that won't benefit the users. Proxy
plugins may be beneficial at some future point, or we may end up with several poorly-supported languages, and later with unmaintained proxies we dare not remove because that'll kill their sub-plugins.
Why do you claim there is no benefit for users? Seamless integration of non-C plugins is a clear benefit to me.
I suspect we would support 1 (or at most 2) proxy plugins ourself.
I hope by "we" you mean the core plugins. Otherwise, that's no "support" at all.
Yes.
Other proxy plugins can be maintained by third parties out of tree or in geany-plugins. I do not want to hard-limit the language choice to our blessed one though.
The other proxy plugins will be DOA, except for mini-plugins (macros, scripts) that suit single users. Why should one bother to write a serious, mass-distributable sub-plugin in a 3rd party language, when there are official ones? To risk the proxy plguin being unmaintained?
Why DOA? I don't see that. I also don't see that there will be only small scripts/macro-style plugins.
I expect that plugin authors can make the best pluings if they are able to use their favorite language. So yes, I expect there will be some serious non-C plugins.
Best regards
I sent this email earlier and never saw it come back through the mailing list, so I'll send it a gain to see if it is going through...
...
Like I think Dimitar was saying, adding custom scripts sound like they should be second class plugins.
Sorry to be a broken record here, but my external tools plugin calls whatever language you can make an executable for or call from the command line: https://github.com/sblatnick/geany-plugins/tree/external-tools/external-tool...
I don't see the need for bindings for every language to tap into plugin functionality when you can call scripts and accept the scripts output with some accepted criteria, kinda like an API or like my plugin does. For instance, external-tools lets you output a program to the bottom panel or substitute currently selected text in geany, etc. While this limits the methods the plugin can interact, it supports an easy input and easy output logic.
I haven't been following the entire thread, and I don't mean to come across as continuously advertising my own plugin, but it seems to me that you are discussing integrating other languages to the plugin structure. In my opinion, this plugin already does that to a limited extent, but to an extent that meets my needs as I'm not looking to tweak UI from the plugin.
Just as I think Dimitar was saying, I don't think we want to overly complicate plugins because of wanting to support "everything" or every language for plugins. I hope changes to how we do plugins don't break existing plugins too much, as that was one of the reasons I left gedit. It's annoying rewriting plugins because of the changes.
Thanks,
Steve
On 05/09/2014 11:15 AM, Dimitar Zhekov wrote:
On Fri, 09 May 2014 12:29:58 +0200 Thomas Martitz kugel@rockbox.org wrote:
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions) Geany will match files against these criterias and, on match, call into the plugin for the first time.
What???
No, this isn't about loading plugins on demand. This is about implementing APIs so that plugins (probably selected in the PM dialog) can act as a proxy to plugins written in other languages (also selected in the PM dialog). I do not intend to change the user experience w.r.t. to plugins: All plugins, regardless of the language, are user-selected in the PM dialog and loaded only then and on subsequent startups.
So then, the only remaining option seems that Geany will search the plugin paths, and match the found file extensions. It's unclear what call returns the sub-plugin metadata (name, version etc.), but that's solvable.
Unless we are trying to enable scripting in more than a few languages, I see no reason for all these complications. Let the master Foo plugin search for and probe the found .foo files, display a list of them, and save the list of selected ones in it's own configuration, without involving Geany. I'd like to have 100 useful python (lua, js, ...) scripts or macros, and will probably write my own, but certainly don't want them as first class plugins.
As another option, we can declare one scripting language as official, support the language binding for it, and provide the command console. Vim, emacs and gdb have one, so why should the light IDE Geany support N of them? And whoever wants to write a plugin to supports his or her favorite REXX/VBScript/younameit is still free to do so.
Personally I'd choose a language that supports GTK+, *nix and Windows, for obvious reasons.
Am 09.05.2014 23:07, schrieb Steven Blatnick:
I sent this email earlier and never saw it come back through the mailing list, so I'll send it a gain to see if it is going through...
It came through. As a tip, you can check the mailing list archives[1] before sending the mail another time
[1]: For example here (reached via geany.org): http://lists.geany.org/pipermail/devel/2014-May/date.html
Thanks! That's weird... I still haven't gotten either of the messages myself. I just checked, and they're not in my spam folder. Does the list no longer send the the sender? I used to get my own messages.
On 05/09/2014 03:14 PM, Thomas Martitz wrote:
Am 09.05.2014 23:07, schrieb Steven Blatnick:
I sent this email earlier and never saw it come back through the mailing list, so I'll send it a gain to see if it is going through...
It came through. As a tip, you can check the mailing list archives[1] before sending the mail another time
[1]: For example here (reached via geany.org): http://lists.geany.org/pipermail/devel/2014-May/date.html _______________________________________________ Devel mailing list Devel@lists.geany.org https://lists.geany.org/cgi-bin/mailman/listinfo/devel
On 14-05-09 02:26 PM, Steven Blatnick wrote:
Thanks! That's weird... I still haven't gotten either of the messages myself. I just checked, and they're not in my spam folder. Does the list no longer send the the sender? I used to get my own messages.
I know GMail doesn't, maybe Yahoo started blocking messages back to self also.
Cheers, Matthew Brush
Hi Thomas,
If the goal is really to enable loading plugins written in other languages, we should take a really extensive look at LibPeas before ruling it out and duplicating it ourselves. It seems especially appropriate since you want to eventually get multi-language support into core, and it provides a small library we can have a hard dependency on that (IIUC) itself has soft-dependencies on all the stuff we don't want to require for Geany (language runtimes, interpreters, etc.).
Disclaimer: I'm not a LibPeas expert or even user but from what I've read about it (the docs, tutorials) it sounds like pretty much exactly the end-goal we want for Geany plugins.
My $0.02.
Cheers, Matthew Brush
On 14-05-06 11:40 PM, Thomas Martitz wrote:
Hello,
I'm (as of now) motivated to implement proxy plugins (to my amusement, until splitwindow2 gets finally reviewed...). Therefore I would like to receive comments on my proposed concept and APIs (further below).
I would also love to discuss about better terms for proxy and "the proxied plugins". I think some adorable internal code/nick names could help us to not get confused in this business.
Note that I have implemented very little yet, so there might be unknown roadblocks and things can easily change. Please also note that I intend to achieve the goals without breaking plugin API/ABI.
The concept:
The basic idea is that proxy plugins initially call a Geany API to register themselves as proxies, providing criterias to select potential plugins (for now, this is only a list of file extensions) and a few hooks to call into the plugin when Geany needs it.
Geany will match files against these criterias and, on match, call into the plugin for the first time. This call should probe whether or not the plugin can actually handle the plugin. I think this is needed because a) criterias/file extensions might be ambiguous and b) the plugin might be written against an incompatible version of the proxy (think python 2 vs 3).
Then when Geany generates the plugin list (i.e. when the user opens plugin manager (PM) dialog) it'll call a second time into the proxy. This time the purpose is to load the actual plugin, and install plugin-specific init(), configure(), help() and cleanup() hooks. Unlike in classic plugins these hooks must call into proxy, where they have to be dispatched to call the actual plugin code (this is why we call them proxy right?).
Once loaded, Geany generally doesn't know/care about the difference between classic plugins and proxied plugins. This achieves the ultimate goal: That proxied plugins transparently integrate into the PM dialog and become first-class citizens.
Part of the proposed concept is to add a "fake proxy" for the builtin .so-file support. This enables to use the same code for all plugins (proxied or not). This should greatly reduce the maintenance effort. The fake proxy naturally must be compiled in but otherwise exposes the same APIs as actual proxies.
The APIs:
This propsal adds 3 major functions and alters the 4 existing plugin hooks.
plugin_register_proxy(GeanyPlugin *proxy_plugin, ProxyPluginInfo *info, gpointer user_data);
This is initially called by the proxy and lets Geany know about it. ProxyPluginInfo contains the criterias and pointers to the probe() and load() functions. user_data is proxy-specific and passed to probe().
gboolean (*probe)(const gchar *file, gpointer user_data, gpointer *plugin_data);
This is implemented in the proxy and should return true when it can handle the plugin, it basically replaces the existing utils_str_casecmp(G_MODULE_SUFFIX, ...) call. There is a catch though, while probe() should be as fast as possible, I expect that some proxies still need to load the file half-way or even in full to make that decision. In order to maintain that state, it can set the plugin_data pointer. This will be passed to the load function. This way loading the plugin again can be avoided.
gboolean (*load)(const gchar *file, gpointer user_data, gpointer *plugin_data);
The same function signature, but a different purpose. The proxy should now fully load the plugin specified with file such that is ready to run (it has the same signature in case it needs to load in probe() already, then this function can be the same as probe()).
As for the existing hooks, these need gain a parameter (gpointer plugin_data). Because the hooks can be indirect now, the proxy needs plugin_data to be able to properly dispatch the hooks on a per-plugin basis. I.e. the hooks become:
void (*init) (GeanyData *data, gpointer plugin_data); GtkWidget* (*configure) (GtkDialog *dialog, gpointer plugin_data); void (*configure_single) (GtkWidget *parent, gpointer plugin_data); void (*help) (gpointer plugin_data); void (*cleanup) (gpointer plugin_data);
Because the plugin_data parameter is exclusively for the proxy, the actual plugins do not receive it. They still implement the non-plugin_data variant. The parameter is added to the end so that these can still point to the module symbols for .so file plugins.
That's it so far. I think this proposal should allow to reach our goal (first class plugin scripts) while avoiding to break plugins ABI/API.
Best regards. _______________________________________________ Devel mailing list Devel@lists.geany.org https://lists.geany.org/cgi-bin/mailman/listinfo/devel
Am 08.05.2014 02:33, schrieb Matthew Brush:
Hi Thomas,
If the goal is really to enable loading plugins written in other languages, we should take a really extensive look at LibPeas before ruling it out and duplicating it ourselves. It seems especially appropriate since you want to eventually get multi-language support into core, and it provides a small library we can have a hard dependency on that (IIUC) itself has soft-dependencies on all the stuff we don't want to require for Geany (language runtimes, interpreters, etc.).
Disclaimer: I'm not a LibPeas expert or even user but from what I've read about it (the docs, tutorials) it sounds like pretty much exactly the end-goal we want for Geany plugins.
My $0.02.
Cheers, Matthew Brush
Right, LibPeas was repeatedly suggested already. However, the first reply was always "uhm it would probably require a massive rewrite" which is kind of showstopper given how hard it is to get changes into the core.
Anyway, I have still looked into it and came to the conclusion that it's not a good fit either way, for several reasons. Please correct me if I'm wrong.
a) All API entry points have to be gir-introspectable, .i.e. gobjectified. This is probably the massive rewrite that was suggested. This also implies massive plugin API/ABI breakage b) It would breaks the plugin API in more ways. For example, plugin metadata shall be supplied via an .ini-style file. c) Plugins need to implement and expose a new gobject in order to be even recognized as plugins (PeasEngine recognizes plugins/extensions by their GType)
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
Best regards.
On 14-05-09 01:18 PM, Thomas Martitz wrote:
Am 08.05.2014 02:33, schrieb Matthew Brush:
Hi Thomas,
If the goal is really to enable loading plugins written in other languages, we should take a really extensive look at LibPeas before ruling it out and duplicating it ourselves. It seems especially appropriate since you want to eventually get multi-language support into core, and it provides a small library we can have a hard dependency on that (IIUC) itself has soft-dependencies on all the stuff we don't want to require for Geany (language runtimes, interpreters, etc.).
Disclaimer: I'm not a LibPeas expert or even user but from what I've read about it (the docs, tutorials) it sounds like pretty much exactly the end-goal we want for Geany plugins.
My $0.02.
Cheers, Matthew Brush
Right, LibPeas was repeatedly suggested already. However, the first reply was always "uhm it would probably require a massive rewrite" which is kind of showstopper given how hard it is to get changes into the core.
Anyway, I have still looked into it and came to the conclusion that it's not a good fit either way, for several reasons. Please correct me if I'm wrong.
a) All API entry points have to be gir-introspectable, .i.e. gobjectified. This is probably the massive rewrite that was suggested. This also implies massive plugin API/ABI breakage
I think just the "extension points" need to be GObjects, for example, say we had a "code completion provider" API where plugins would be able to provide better completions for a particular language, we would have a "completion provider extension point" (IIUC). Other examples that we keep talking about being useful in Geany which would each require "tweaking" the existing code in the ways you propose specifically for proxy plugins include:
* Completion providers * Syntax highlighters * Filetype plugins * I had 2 or 3 more that escape me right now
b) It would breaks the plugin API in more ways. For example, plugin metadata shall be supplied via an .ini-style file.
It wouldn't have to severely break the API functions (eg. the stuff in geanyfunctions.h) but it would most certainly not be compatible with the part of the API used for loading/convenience like all the `plugin_*` functions.
c) Plugins need to implement and expose a new gobject in order to be even recognized as plugins (PeasEngine recognizes plugins/extensions by their GType)
This is not only fairly trivial in C but many times more trivial in other languages (well at least in Python it's very simple, not sure about other languages).
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
IMO, it wouldn't be a big deal as a "parallel" plugin system/mechanism, while just leaving the old one in place for some years as a "legacy C plugins API" (most of the API functions are just plain internal Geany functions, not so much work to maintain).
Anyway, I don't really care, but I just think we should consider all the problems with the current plugin API and when implementing something as you're proposing, not just hack on something for the one use-case but maybe think about something more extensible/useful in other contexts.
Cheers, Matthew Brush
On Fri, 09 May 2014 22:18:41 +0200 Thomas Martitz kugel@rockbox.org wrote:
Am 08.05.2014 02:33, schrieb Matthew Brush:
Hi Thomas,
Disclaimer: I'm not a LibPeas expert or even user but from what I've read about it (the docs, tutorials) it sounds like pretty much exactly the end-goal we want for Geany plugins.
Read the documentation. It's only 6 base classes, 2 interfaces and 2 widgets. The widgets form simple plugin manager, but they are completely optional, we can write our own using the current UI.
Right, LibPeas was repeatedly suggested already. However, the first reply was always "uhm it would probably require a massive rewrite" which is kind of showstopper given how hard it is to get changes into the core.
Anyway, I have still looked into it and came to the conclusion that it's not a good fit either way, for several reasons. Please correct me if I'm wrong.
a) All API entry points have to be gir-introspectable, .i.e. gobjectified. This is probably the massive rewrite that was suggested. This also implies massive plugin API/ABI breakage
For C plugins, we just need to pass geany_data and geany_functions. See peas-demo-window.c: peas_extension_set_foreach(), g_signal_connect("extension-added") and the demo plugins.
For python, maybe we can include geanypy/src/*.c in Geany or in a .so library. But if we don't want to end up with more 150+ KB proxies like geanypy, we'd better use introspection. I really admire Matthew and Lex for writing this thing, but that's hardly the best approach, and I don't expect many people to repeat it and maintain such proxy.
b) It would breaks the plugin API in more ways. For example, plugin metadata shall be supplied via an .ini-style file.
Moving a few strings from foo.c to foo.ini will hardly be a problem. :)
c) Plugins need to implement and expose a new gobject in order to be even recognized as plugins (PeasEngine recognizes plugins/extensions by their GType)
For C, it can probably be handled by 2 macros, REGISTER_PLUGIN and REGISTER_PLUGIN_WITH_CONFIGURE. Our current plugins don't support Activate, and the Help in peas is an URL, not a function.
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
peas does does you describe, and provides build-in loaders for Python 2/3 and JavaScript, i.e. standard languages. Please don't throw it away before even trying to adapt it.
Am 10.05.2014 21:06, schrieb Dimitar Zhekov:
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
peas does does you describe, and provides build-in loaders for Python 2/3 and JavaScript, i.e. standard languages. Please don't throw it away before even trying to adapt it.
As you have mentioned, even for C plugins it's a major change, and it requires a lot of changes to Geany (doesn't it?).
So why adapt peas when it requires a lot of changes *too* but also severely breaks plugin API?
I must be missing something, but it essence it appears to me that adapting peas requires no less effort than what I propose (it's not actually that much new code, just a lot of refactoring) but also implies a major plugin API breakage (while my proposal can be implemented without API breakage).
Best regards
On 14-05-10 12:56 PM, Thomas Martitz wrote:
Am 10.05.2014 21:06, schrieb Dimitar Zhekov:
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
peas does does you describe, and provides build-in loaders for Python 2/3 and JavaScript, i.e. standard languages. Please don't throw it away before even trying to adapt it.
As you have mentioned, even for C plugins it's a major change, and it requires a lot of changes to Geany (doesn't it?).
So why adapt peas when it requires a lot of changes *too* but also severely breaks plugin API?
I must be missing something, but it essence it appears to me that adapting peas requires no less effort than what I propose (it's not actually that much new code, just a lot of refactoring) but also implies a major plugin API breakage (while my proposal can be implemented without API breakage).
I don't think it has to break the existing plugin loader interface, we could make a parallel loader that integrates into the current Plugin Manager UI and such. Just leave the existing C hooks in place (ie. plugins.[c], pluginutils.[c], plugindata.h stuff) and maybe recommend people not to use for new plugins. Most of the public API functions could still be shared between core, old-style plugins and new-style plugins (via some manner of bindings for the non-C ones).
Cheers, Matthew Brush
On Sat, 10 May 2014 21:56:30 +0200 Thomas Martitz kugel@rockbox.org wrote:
On Sat, 10 May 2014 14:47:17 -0700 Matthew Brush mbrush@codebrainz.ca wrote:
+1 for some form of introspection/code generation.
Just to be clear: by "introspection", I don't mean *replacing* struct GeanyDocument with GObject GGeanyDocument and so on for the entire API. We may have a GGeanyDocument which represents (a) the GeanyDocument structure, (b) all document_*() functions as geany_document_*() and (c) all "document-" signals as "geany-document" with a GGeanyDocument argument. We can't generate signals for GGeanyDocument "fields" being changed directly from C, but we don't support such signals anyway.
Am 10.05.2014 21:06, schrieb Dimitar Zhekov:
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
peas does does you describe, and provides build-in loaders for Python 2/3 and JavaScript, i.e. standard languages. Please don't throw it away before even trying to adapt it.
As you have mentioned, even for C plugins it's a major change,
I said moving a few strings to .ini, and adding a few macros.
and it requires a lot of changes to Geany (doesn't it?).
It will require some changes, for example GGeanyDocument will not be straight-forward because of our organisation of the document list. Most will be additions though, with few (if any) API breaks.
So why adapt peas when it requires a lot of changes *too* but also severely breaks plugin API?
I must be missing something, but it essence it appears to me that adapting peas requires no less effort than what I propose...
GeanyPy proxy = ~150 KB. 5 language proxies = ~750 KB, code that will have to be written and maintained. For comparision, geany/src/* = 1.9 MB.
peas loader = ~15 KB, language support is in the core, and doesn't change with the new Geany versions, so you don't have to worry about any language(s) becoming unsupported/unmaintained.
We must have Geany API (structures and functions) introspected for all new languages somehow. We can do that with large, manual, bug-prone per-language introspection, or with one GLib introspection.
...but also implies a major plugin API breakage (while my proposal can be implemented without API breakage).
There will be small breakage for the .C plugins, but basicly all API function calls and structures will be the same. For Python, the object names will change. To make an omelette, you have to break the eggs.
Am 11.05.2014 20:31, schrieb Dimitar Zhekov:
On Sat, 10 May 2014 21:56:30 +0200 Thomas Martitz kugel@rockbox.org wrote:
On Sat, 10 May 2014 14:47:17 -0700 Matthew Brush mbrush@codebrainz.ca wrote:
+1 for some form of introspection/code generation.
Just to be clear: by "introspection", I don't mean *replacing* struct GeanyDocument with GObject GGeanyDocument and so on for the entire API. We may have a GGeanyDocument which represents (a) the GeanyDocument structure, (b) all document_*() functions as geany_document_*() and (c) all "document-" signals as "geany-document" with a GGeanyDocument argument. We can't generate signals for GGeanyDocument "fields" being changed directly from C, but we don't support such signals anyway.
That's two GeanyDocument APIs to maintain.
Am 10.05.2014 21:06, schrieb Dimitar Zhekov:
After that I'd say that LibPeas is perhaps something to be considered for new application but not for our existing codebase. I think we want something that enables proxy plugins while maintaining API and ABI stability.
peas does does you describe, and provides build-in loaders for Python 2/3 and JavaScript, i.e. standard languages. Please don't throw it away before even trying to adapt it.
As you have mentioned, even for C plugins it's a major change,
I said moving a few strings to .ini, and adding a few macros.
and it requires a lot of changes to Geany (doesn't it?).
It will require some changes, for example GGeanyDocument will not be straight-forward because of our organisation of the document list. Most will be additions though, with few (if any) API breaks.
So it does require non-trivial changes to the core, too?
So why adapt peas when it requires a lot of changes *too* but also severely breaks plugin API?
I must be missing something, but it essence it appears to me that adapting peas requires no less effort than what I propose...
GeanyPy proxy = ~150 KB. 5 language proxies = ~750 KB, code that will have to be written and maintained. For comparision, geany/src/* = 1.9 MB.
peas loader = ~15 KB, language support is in the core, and doesn't change with the new Geany versions, so you don't have to worry about any language(s) becoming unsupported/unmaintained.
We must have Geany API (structures and functions) introspected for all new languages somehow. We can do that with large, manual, bug-prone per-language introspection, or with one GLib introspection.
Don't be ridiculous. This calculation is severely flawed and you know that.
...but also implies a major plugin API breakage (while my proposal can be implemented without API breakage).
There will be small breakage for the .C plugins, but basicly all API function calls and structures will be the same. For Python, the object names will change. To make an omelette, you have to break the eggs.
No, you don't. This can be implemented without breakage (at least for C plugins, probably for existing python ones too).
On 14-05-13 12:54 AM, Thomas Martitz wrote:
Am 11.05.2014 20:31, schrieb Dimitar Zhekov:
On Sat, 10 May 2014 21:56:30 +0200 Thomas Martitz kugel@rockbox.org wrote: [snip]
So why adapt peas when it requires a lot of changes *too* but also severely breaks plugin API?
I must be missing something, but it essence it appears to me that adapting peas requires no less effort than what I propose...
GeanyPy proxy = ~150 KB. 5 language proxies = ~750 KB, code that will have to be written and maintained. For comparision, geany/src/* = 1.9 MB.
peas loader = ~15 KB, language support is in the core, and doesn't change with the new Geany versions, so you don't have to worry about any language(s) becoming unsupported/unmaintained.
We must have Geany API (structures and functions) introspected for all new languages somehow. We can do that with large, manual, bug-prone per-language introspection, or with one GLib introspection.
Don't be ridiculous. This calculation is severely flawed and you know that.
Well it is true we need this either way for non-C languages to interact with the API. In the case of GeanyLua, GeanyPy and most-likely your new "proxy plugins" it's all whole lot of C code which would probably be too much maintenance burden to be in core. This way is probably the worst approach.
GObject-introspection would probably be the best approach, but for the existing exposed API functions/structs, would probably at the very least require changing doc-comments from Doxygen to GTK-DOC (plus adding some annotations), writing some override files to patch it up, and getting rid of the macros hiding function pointers that are members of structures stuff. This still wouldn't produce a "good" API for non-C languages (or C even), but it would at least make it not completely manual.
Alternatives are such stuff as using SWIG, or having better manually maintained interface files (eg. .vapi file for Vala, cython or ctypes for Python, etc.) or rolling own Gobject-introspection system using Doxygen XML output/libclang/gccxml and a bunch of code generation scripts or something, none of which sounds very good, IMO.
Cheers, Matthew Brush
On 14-05-13 01:43 AM, Matthew Brush wrote:
On 14-05-13 12:54 AM, Thomas Martitz wrote:
Am 11.05.2014 20:31, schrieb Dimitar Zhekov:
On Sat, 10 May 2014 21:56:30 +0200 Thomas Martitz kugel@rockbox.org wrote: [snip]
So why adapt peas when it requires a lot of changes *too* but also severely breaks plugin API?
I must be missing something, but it essence it appears to me that adapting peas requires no less effort than what I propose...
GeanyPy proxy = ~150 KB. 5 language proxies = ~750 KB, code that will have to be written and maintained. For comparision, geany/src/* = 1.9 MB.
peas loader = ~15 KB, language support is in the core, and doesn't change with the new Geany versions, so you don't have to worry about any language(s) becoming unsupported/unmaintained.
We must have Geany API (structures and functions) introspected for all new languages somehow. We can do that with large, manual, bug-prone per-language introspection, or with one GLib introspection.
Don't be ridiculous. This calculation is severely flawed and you know that.
Well it is true we need this either way for non-C languages to interact with the API. In the case of GeanyLua, GeanyPy and most-likely your new "proxy plugins" it's all whole lot of C code which would probably be too much maintenance burden to be in core. This way is probably the worst approach.
GObject-introspection would probably be the best approach, but for the existing exposed API functions/structs, would probably at the very least require changing doc-comments from Doxygen to GTK-DOC (plus adding some annotations), writing some override files to patch it up, and getting rid of the macros hiding function pointers that are members of structures stuff. This still wouldn't produce a "good" API for non-C languages (or C even), but it would at least make it not completely manual.
FWIW, I've been doing some experiments with libpeas as the loader in Geany (in my peas* branches on Github) and for the Python demo plugin (code not pushed to GH yet) I managed to get GObject-Introspection to scan Geany's source code/headers. Most of the work involved just "including what you use" and/or using defined names like `struct GeanyDocument` instead of `GeanyDocument` because the latter is defined off in some unrelated header. Once GI can grok one header at a time (ie. without pre-processing all the source together like C compiler does), it seems to pickup lots of the API (can't vouch for if or how well the bindings actually works yet).
Needs more experimentation/tweaking to be usable still though.
Cheers, Matthew Brush
Am 16.05.2014 11:16, schrieb Matthew Brush:
FWIW, I've been doing some experiments with libpeas as the loader in
Geany (in my peas* branches on Github) and for the Python demo plugin (code not pushed to GH yet) I managed to get GObject-Introspection to scan Geany's source code/headers. Most of the work involved just "including what you use" and/or using defined names like `struct GeanyDocument` instead of `GeanyDocument` because the latter is defined off in some unrelated header. Once GI can grok one header at a time (ie. without pre-processing all the source together like C compiler does), it seems to pickup lots of the API (can't vouch for if or how well the bindings actually works yet).
Needs more experimentation/tweaking to be usable still though.
Very cool that you managed to get GI work (mostly). It'll help us regardless of libpeas, right?
I have a question regarding libpeas. There doesn't seem to be a public (nor documented) API to add loaders. From what I can see the current language support of libpeas is quite poor (only python and seed (that's JS isnt it?)). Also, they don't seem to very committed to maintaining their loaders[1].
I think we want to maintain the ability to add loaders on our own, without depending on a 3rd party project. Especially for potentially creating a compat-loader for our existing plugins. It doesn't seem libpeas readily supports this. Unless I'm missing something.
[1]: https://github.com/GNOME/libpeas/commit/730edb65d6da0ebd0bd7065aeb9435b31d30...
Best regards.
On Fri, 16 May 2014 16:59:17 +0200 Thomas Martitz kugel@rockbox.org wrote:
I have a question regarding libpeas. There doesn't seem to be a public (nor documented) API to add loaders. From what I can see the current language support of libpeas is quite poor (only python and seed (that's JS isnt it?)). Also, they don't seem to very committed to maintaining their loaders[1].
I think we want to maintain the ability to add loaders on our own, without depending on a 3rd party project. Especially for potentially creating a compat-loader for our existing plugins. It doesn't seem libpeas readily supports this. Unless I'm missing something.
The loaders (except for C) are plugins, and the build-in ones are installed in /usr/lib/libpeas-<version>/loaders/ as .so libraries. For example, peas-plugin-loader-python.c contains:
G_MODULE_EXPORT void peas_register_types (PeasObjectModule *module) <-- as a regular plugin { peas_object_module_register_extension_type (module, PEAS_TYPE_PLUGIN_LOADER, PEAS_TYPE_PLUGIN_LOADER_PYTHON); }
And for the next probable question, there is no search for .py files, instead the plugin .ini file (foo.plugin) specifies which loader should be used (C if missing).
Am 16.05.2014 19:32, schrieb Dimitar Zhekov:
On Fri, 16 May 2014 16:59:17 +0200 Thomas Martitz kugel@rockbox.org wrote:
I have a question regarding libpeas. There doesn't seem to be a public (nor documented) API to add loaders. From what I can see the current language support of libpeas is quite poor (only python and seed (that's JS isnt it?)). Also, they don't seem to very committed to maintaining their loaders[1].
I think we want to maintain the ability to add loaders on our own, without depending on a 3rd party project. Especially for potentially creating a compat-loader for our existing plugins. It doesn't seem libpeas readily supports this. Unless I'm missing something.
The loaders (except for C) are plugins, and the build-in ones are installed in /usr/lib/libpeas-<version>/loaders/ as .so libraries. For example, peas-plugin-loader-python.c contains:
G_MODULE_EXPORT void peas_register_types (PeasObjectModule *module) <-- as a regular plugin { peas_object_module_register_extension_type (module, PEAS_TYPE_PLUGIN_LOADER, PEAS_TYPE_PLUGIN_LOADER_PYTHON); }
I found that too. But this is not public and not documented. It's also not installed to /usr/include/libpeas-1.0.
Since we want plugin authors to be able to create loaders we need something that's stable and documented.
Best regards.
And for the next probable question, there is no search for .py files, instead the plugin .ini file (foo.plugin) specifies which loader should be used (C if missing).
I think this could be a problem too. Unless we maintain two loaders (that would be better avoided) we need to find a way to have libpeas find (and load) current plugins. Perhaps we could create a small helper function that creates a suitable .ini (if it doesn't exist) out of the current plugin_set_info() before running libpeas scan, for compatibility purposes.
Best regards.
On 14-05-16 03:03 PM, Thomas Martitz wrote:
Am 16.05.2014 19:32, schrieb Dimitar Zhekov:
On Fri, 16 May 2014 16:59:17 +0200 Thomas Martitz kugel@rockbox.org wrote:
I have a question regarding libpeas. There doesn't seem to be a public (nor documented) API to add loaders. From what I can see the current language support of libpeas is quite poor (only python and seed (that's JS isnt it?)). Also, they don't seem to very committed to maintaining their loaders[1].
It doesn't seem to make much sense to support two JavaScript implementations, although I don't know the specifics of this.
I think we want to maintain the ability to add loaders on our own,
With Peas you can, in fact it's designed with this purpose in mind, unlike the existing C-only loader.
without depending on a 3rd party project. Especially for potentially
Meh, we depend on lots of 3rd party projects, several directly and many indirectly (see `ldd geany`).
Peas seems to be quite popular in the community around our framework/UI toolkit and is used by several (if not dozens) of popular applications, which also means it's readily available in many popular distros' package repositories.
Even if Peas becomes unmaintained/unsupported, we could roll it into our own source tree and maintain it like we do ctags and tagmanager and we still wouldn't have had to write it ourselves :)
creating a compat-loader for our existing plugins. It doesn't seem
After experimenting with this a bit, I believe it doesn't make much sense to make a compatibility layer because a) you still need basically the same code as the existing loader code, b) you would have to change/refactor/re-write lots of it c) you would still have two distinct ways to implement plugin interfaces, d) increases chances of breaking the existing loader/plugins and e) intertwining the two loaders would probably make it harder in the distant future to just drop one of them.
libpeas readily supports this. Unless I'm missing something.
The loaders (except for C) are plugins, and the build-in ones are installed in /usr/lib/libpeas-<version>/loaders/ as .so libraries. For example, peas-plugin-loader-python.c contains:
G_MODULE_EXPORT void peas_register_types (PeasObjectModule *module) <-- as a regular plugin { peas_object_module_register_extension_type (module, PEAS_TYPE_PLUGIN_LOADER, PEAS_TYPE_PLUGIN_LOADER_PYTHON); }
I found that too. But this is not public and not documented. It's also not installed to /usr/include/libpeas-1.0.
Yikes! Free Software not properly documented! :)
Since we want plugin authors to be able to create loaders we need something that's stable and documented.
Then why do you want to write our own libpeas into the existing C plugin loader? It will (at least for some time) be buggier, and less widely tested/used/maintained (ie. only by us), take lots of extra effort, not automate the bridge between core and the plugins' language, probably be less well-documented, and after all, the end result would most-likely provide an inferior experience for non-C plugin authors.
Best regards.
And for the next probable question, there is no search for .py files, instead the plugin .ini file (foo.plugin) specifies which loader should be used (C if missing).
I think this could be a problem too. Unless we maintain two loaders (that would be better avoided) we need to find a way to have libpeas
We wouldn't have to maintain libpeas, it's a separate project with other developers maintaining it for us. Eventually in the long distant future (eg. Geany 2.0), we could drop the old loader and maintain zero loaders :)
find (and load) current plugins. Perhaps we could create a small helper function that creates a suitable .ini (if it doesn't exist) out of the current plugin_set_info() before running libpeas scan, for compatibility purposes.
As mentioned above, this doesn't seem worth the trouble and I don't think it would be as easy/useful as it seems on the surface. We could just leave the existing loader without breaking anything, it's quite stable and works pretty well for plain C plugins that don't want to use GObject-ish stuff. We could document the Peas loader and related work as provisional while we get it all integrated and working well with the GI bindings and still leaving the existing stable C loader in-tact during this time and for the foreseeable future.
Cheers, Matthew Brush
Hi Guys,
Some general observations, irrespective of the implementation.
1. There shall be only one plugin control GUI. The users don't care how a plugin is implemented, or what language its in, they just want to use it. Having to look in two places will often mean one group of plugins is missed. Thats also one of the complaints against the current Geanypy implementation.
2. All current plugins shall continue to work for a considerable period until any new mechanism is in place. Re-compilation is acceptable although maintaining ABI is preferable. It has taken so long to accumulate a set of fairly useful plugins that we can't afford to try to force them to be re-built (quickly) to a new interface to keep working.
3. Any new plugin system shall not require large amounts of boilerplate to write a plugin, in any language. The basic point of this effort is to encourage plugins by making it easier. Hiding boilerplate under a part of the bindings is of course acceptable.
4. A full implementation is required before anything is added to Geany, that includes bindings to all the Geany API for the existing languages (C/C++/Vala/Python). Finding unexpected difficulties with the bindings is so likely (see Geanypy keybindings issues) that there needs to be a clear demonstration that they can be addressed.
Support for autogeneration of the bindings would of course be the cherry on the cake, but thats our issue not the plugin writers.
Cheers Lex
On 14-05-16 07:34 PM, Lex Trotman wrote:
Hi Guys,
Some general observations, irrespective of the implementation.
- There shall be only one plugin control GUI. The users don't care
how a plugin is implemented, or what language its in, they just want to use it. Having to look in two places will often mean one group of plugins is missed. Thats also one of the complaints against the current Geanypy implementation.
+1 I think this is quite important, and if anyone has used the builtin Peas plugin manager UI (as in my peas* branches for testing purposes) they'd probably agree Geany's Plugin Manager is a fair bit nicer. I'm pretty sure this wouldn't be too hard either, as peas provides much the same metadata/concepts as is needed for Geany's PM.
- All current plugins shall continue to work for a considerable
period until any new mechanism is in place. Re-compilation is acceptable although maintaining ABI is preferable. It has taken so long to accumulate a set of fairly useful plugins that we can't afford to try to force them to be re-built (quickly) to a new interface to keep working.
+1 This is why I don't think we should try and munge the existing working C loader into a Peas duplicate and instead use a pre-written one that is used in production, as a parallel/alternative to what exists now.
- Any new plugin system shall not require large amounts of
boilerplate to write a plugin, in any language. The basic point of this effort is to encourage plugins by making it easier. Hiding boilerplate under a part of the bindings is of course acceptable.
+1 C is the only likely suspect, implementing a GObject interface isn't exactly the most brief code to write (but it is at least well structured and easy to read). I'm pretty sure some fairly compatible adapter could be implemented, the GeanyWinCmd interface in my experimental peas* branches is quite similar semantically to the existing plugin interface's plugin_init/plugin_deinit, in practice.
- A full implementation is required before anything is added to
Geany, that includes bindings to all the Geany API for the existing languages (C/C++/Vala/Python). Finding unexpected difficulties with the bindings is so likely (see Geanypy keybindings issues) that there needs to be a clear demonstration that they can be addressed.
-1 This will be dead-on-arrival for any implementation. I would agree if you had said "... before the --enable-alternative-build-option configure flag became the default/defaulted to on" :)
We just aren't setup to maintain a long running disparate branch like this that nobody could humanly review and for sure most people won't test, for final merging into core in some giant commit-bomb designed by one person in their own private/local repository. IMO, we just need to do like we do with --enable-gtk3 configure flag so that the default options don't enable anything provisional/experimental.
To be clear, I don't mean to imply that we should commit random buggy code (like my peas* branches at present) into the core, just that it's too large of a thing not to be worked on in the master development branch by all developers/stakeholders, gradually. By experience we've proven that to do otherwise is anywhere from hard (see GtkBuilder changes), to pointlessly discouraging (see document-messages), to completely impractical to merge into core at the end (see splitwindow2 and any other pending non-trivial core code changes in recent history).
Cheers, Matthew Brush
Note: Sorry for the shalls, I have been reading too many specifications lately and it has worked its way into my brain.
On 17 May 2014 13:04, Matthew Brush mbrush@codebrainz.ca wrote:
On 14-05-16 07:34 PM, Lex Trotman wrote:
Hi Guys,
Some general observations, irrespective of the implementation.
- There shall be only one plugin control GUI. The users don't care
how a plugin is implemented, or what language its in, they just want to use it. Having to look in two places will often mean one group of plugins is missed. Thats also one of the complaints against the current Geanypy implementation.
+1 I think this is quite important, and if anyone has used the builtin Peas plugin manager UI (as in my peas* branches for testing purposes) they'd probably agree Geany's Plugin Manager is a fair bit nicer. I'm pretty sure this wouldn't be too hard either, as peas provides much the same metadata/concepts as is needed for Geany's PM.
Especially the new improved PM :)
I guess it just needs the effort to deprecate libpeas default UI and incorporate it into Geany's.
- All current plugins shall continue to work for a considerable
period until any new mechanism is in place. Re-compilation is acceptable although maintaining ABI is preferable. It has taken so long to accumulate a set of fairly useful plugins that we can't afford to try to force them to be re-built (quickly) to a new interface to keep working.
+1 This is why I don't think we should try and munge the existing working C loader into a Peas duplicate and instead use a pre-written one that is used in production, as a parallel/alternative to what exists now.
That makes sense, especially if the libpeas GUI isn't going to be used (see 1.).
- Any new plugin system shall not require large amounts of
boilerplate to write a plugin, in any language. The basic point of this effort is to encourage plugins by making it easier. Hiding boilerplate under a part of the bindings is of course acceptable.
+1 C is the only likely suspect, implementing a GObject interface isn't exactly the most brief code to write (but it is at least well structured and easy to read). I'm pretty sure some fairly compatible adapter could be implemented, the GeanyWinCmd interface in my experimental peas* branches is quite similar semantically to the existing plugin interface's plugin_init/plugin_deinit, in practice.
I thought you said on IRC it could be hidden under a <shudder> macro or two :)
- A full implementation is required before anything is added to
Geany, that includes bindings to all the Geany API for the existing languages (C/C++/Vala/Python). Finding unexpected difficulties with the bindings is so likely (see Geanypy keybindings issues) that there needs to be a clear demonstration that they can be addressed.
-1 This will be dead-on-arrival for any implementation. I would agree if you had said "... before the --enable-alternative-build-option configure flag became the default/defaulted to on" :)
That means we have decided on an implementation, before any is complete and the full implications are known.
That is not a good way to do something as complex as this. As an example I would point out the misunderstanding already discussed on IRC of how much libpeas actually contributed to the bindings. There will surely be more misunderstandings/lack of knowledge that makes deciding at this point a bad approach.
Certainly if it becomes clear that one approach is markedly superior to the other, and is actually likely to be fully implementable, then its ok to commit a partly working (or even non-working) implementation of new stuff that doesn't break existing functionality.
But we are not at that point yet. So the discussion is how to get to that point.
We just aren't setup to maintain a long running disparate branch like this that nobody could humanly review and for sure most people won't test, for final merging into core in some giant commit-bomb designed by one person in their own private/local repository. IMO, we just need to do like we do with --enable-gtk3 configure flag so that the default options don't enable anything provisional/experimental.
Well, yes, developing it in codebrainz/geany or kugel/geany is unlikely to see it have much adult supervision. :)
Since the switch to git we have not had any long running branches, even though it supports them just fine, in fact the last was probably the build system changes back under SVN. The big advantage of branches under the main repository is that they will let *all* the devs know what you are doing, even if they don't want to know. So there can be input to the process. That was very useful to the build system, even if some of the changes did cause Enrico a heart attack. At least he could then object immediately, not at the end when 1000 more commits depended on the contentious ones.
As you say massive commit bombs from personal forks is a bad approach both technically and socially. Technically its hard to review and understand possible implications of a big change and the developer, having thought about it a lot, has a mental model that none of the reviewers have, and socially the developer has by then got so much personal investment in it, that its hard to respond to criticisms objectively.
That problem is not unique to Geany, the same technical problem occurs even with the Kernel when some company drops in a whole new driver or something, and they have many more resources than we do. But things like new schedulers or other big changes internal to the kernel must progress through long-term branches before being included in the mainline.
To be clear, I don't mean to imply that we should commit random buggy code (like my peas* branches at present) into the core, just that it's too large of a thing not to be worked on in the master development branch by all developers/stakeholders, gradually. By experience we've proven that to do otherwise is anywhere from hard (see GtkBuilder changes), to pointlessly discouraging (see document-messages), to completely impractical to merge into core at the end (see splitwindow2 and any other pending non-trivial core code changes in recent history).
Well, all of those have been personal development commit-bombs.
To be fair, document-messages is waiting on a GTK version hike, precisely because it *has not* got lots of #ifdefs to turn it off on 2.16. Why we can't manage to make the GTK jump is not relevant to this discussion.
Splitwindow2 did have some testing during development, and I am happy to have it committed, but Colomban has expressed reservations for the above mentioned technical reasons IIRC.
Cheers Lex
Cheers, Matthew Brush
Devel mailing list Devel@lists.geany.org https://lists.geany.org/cgi-bin/mailman/listinfo/devel
On 14-05-16 06:43 PM, Matthew Brush wrote:
[snip]
As mentioned above, this doesn't seem worth the trouble and I don't think it would be as easy/useful as it seems on the surface. We could just leave the existing loader without breaking anything, it's quite stable and works pretty well for plain C plugins that don't want to use GObject-ish stuff. We could document the Peas loader and related work as provisional while we get it all integrated and working well with the GI bindings and still leaving the existing stable C loader in-tact during this time and for the foreseeable future.
Just to be clear, I don't mean to imply that we shouldn't make any effort towards providing a simple way to adapt existing plugins to use the new loader with minimal changes, just that I don't think that this mechanism or some compatibility layer for the loader needs to be a blocker/priority towards providing an alternative parallel loader that eases working in other languages and paves the way for more specific and powerful extension points with more idiomatic interfaces and APIs in multiple languages.
Cheers, Matthew Brush
Am 17.05.2014 03:43, schrieb Matthew Brush:
On 14-05-16 03:03 PM, Thomas Martitz wrote:
I think we want to maintain the ability to add loaders on our own,
With Peas you can, in fact it's designed with this purpose in mind, unlike the existing C-only loader.
If it's designed for that, why doesn't it expose that to its users? peas-plugin-loader.h is neither documented nor actually shipped, which makes me think they want to keep their plugin architecture private (subject to change). Given the GNOME community reputation that really worries me because we totally want to add loaders to libpeas.
without depending on a 3rd party project. Especially for potentially
Meh, we depend on lots of 3rd party projects, several directly and many indirectly (see `ldd geany`).
Yes, but these generally don't affect the plugin API. When we make promises about the stability of the plugin API but depend on a 3rd party to fullfill that promise it's not an ideal situation. Especially when depending on the GNOME community as with libpeas.
Peas seems to be quite popular in the community around our framework/UI toolkit and is used by several (if not dozens) of popular applications, which also means it's readily available in many popular distros' package repositories.
Even if Peas becomes unmaintained/unsupported, we could roll it into our own source tree and maintain it like we do ctags and tagmanager and we still wouldn't have had to write it ourselves :)
Yes I think that would be an acceptable solution once problems arise. libpeas is pretty tiny isnt it?
creating a compat-loader for our existing plugins. It doesn't seem
After experimenting with this a bit, I believe it doesn't make much sense to make a compatibility layer because a) you still need basically the same code as the existing loader code, b) you would have to change/refactor/re-write lots of it c) you would still have two distinct ways to implement plugin interfaces, d) increases chances of breaking the existing loader/plugins and e) intertwining the two loaders would probably make it harder in the distant future to just drop one of them.
The problem is that libpeas demands a different method of making plugins available (.ini vs known symbols in the .so file). We definitely want only one method, even if it means transitioning the symbol method to the .ini file.
I don't think b) is true, and for e) it would make it actually simpler if the one loader is just a plugin of the other one.
libpeas readily supports this. Unless I'm missing something.
The loaders (except for C) are plugins, and the build-in ones are installed in /usr/lib/libpeas-<version>/loaders/ as .so libraries. For example, peas-plugin-loader-python.c contains:
G_MODULE_EXPORT void peas_register_types (PeasObjectModule *module) <-- as a regular plugin { peas_object_module_register_extension_type (module, PEAS_TYPE_PLUGIN_LOADER, PEAS_TYPE_PLUGIN_LOADER_PYTHON); }
I found that too. But this is not public and not documented. It's also not installed to /usr/include/libpeas-1.0.
Yikes! Free Software not properly documented! :)
It's not just a case of bad docs here. In this case it means it's not documented because it's private to libpeas, i.e. an implementation detail. For the same reason the peas-plugin-loader.h is not installed/distributed.
While it has a plugin architecture for loaders it doesn't seem designed to allow for 3rd party loaders.
Since we want plugin authors to be able to create loaders we need something that's stable and documented.
Then why do you want to write our own libpeas into the existing C plugin loader? It will (at least for some time) be buggier, and less widely tested/used/maintained (ie. only by us), take lots of extra effort, not automate the bridge between core and the plugins' language, probably be less well-documented, and after all, the end result would most-likely provide an inferior experience for non-C plugin authors.
I do not want to actually write that. However, I'm not convinced of libpeas yet (especially because if the .ini aspect), and then there's no alternative left. It only has to work well for us so I think the maintainance effort is manageable (and we'd have more control).
But if libpeas can actually help us I'm happy to adopt it. I really mean it.
On 14-05-18 03:07 PM, Thomas Martitz wrote:
Am 17.05.2014 03:43, schrieb Matthew Brush:
On 14-05-16 03:03 PM, Thomas Martitz wrote:
I think we want to maintain the ability to add loaders on our own,
With Peas you can, in fact it's designed with this purpose in mind, unlike the existing C-only loader.
If it's designed for that, why doesn't it expose that to its users? peas-plugin-loader.h is neither documented nor actually shipped, which makes me think they want to keep their plugin architecture private (subject to change). Given the GNOME community reputation that really worries me because we totally want to add loaders to libpeas.
Because I guess they assume that if you're going to write another language loader you're going to contribute it back to them. It would be like adding a Scintilla lexer into Geany's source tree without submitting it back upstream.
without depending on a 3rd party project. Especially for potentially
Meh, we depend on lots of 3rd party projects, several directly and many indirectly (see `ldd geany`).
Yes, but these generally don't affect the plugin API. When we make promises about the stability of the plugin API but depend on a 3rd party to fullfill that promise it's not an ideal situation. Especially when depending on the GNOME community as with libpeas.
This doesn't affect the plugin *API*. The only part that depends on how peas does it is in the .plugin ini files stuff. Nothing in the plugin's code even really has to know about Peas except maybe calling some type registration function, it's just GObject interfaces after that.
Peas seems to be quite popular in the community around our framework/UI toolkit and is used by several (if not dozens) of popular applications, which also means it's readily available in many popular distros' package repositories.
Even if Peas becomes unmaintained/unsupported, we could roll it into our own source tree and maintain it like we do ctags and tagmanager and we still wouldn't have had to write it ourselves :)
Yes I think that would be an acceptable solution once problems arise. libpeas is pretty tiny isnt it?
I appears to be after some brief looking at the source code.
creating a compat-loader for our existing plugins. It doesn't seem
After experimenting with this a bit, I believe it doesn't make much sense to make a compatibility layer because a) you still need basically the same code as the existing loader code, b) you would have to change/refactor/re-write lots of it c) you would still have two distinct ways to implement plugin interfaces, d) increases chances of breaking the existing loader/plugins and e) intertwining the two loaders would probably make it harder in the distant future to just drop one of them.
The problem is that libpeas demands a different method of making plugins available (.ini vs known symbols in the .so file). We definitely want only one method, even if it means transitioning the symbol method to the .ini file.
Yeah, this is actually a bonus for both C and non-C plugins. For one, it means Geany doesn't have to actually load the .so files for C plugins to read the plugin info, and additionally for not-C languages, it means the engine can get metadata about the plugin without involving the loaders (ie. no need to PyInitialize() the interpreter just to get metadata of unloaded plugin in Plugin Manager GUI).
That being said, if we make existing plugin loader use .INI stuff it will break all plugins, so maybe we could make some "opt-in" way that doesn't.
I don't think b) is true, and for e) it would make it actually simpler if the one loader is just a plugin of the other one.
For b) I mean mostly just shuffling around/re-factoring code. You're right about e), re-factoring it this way probably would improve its encapsulation/isolation.
libpeas readily supports this. Unless I'm missing something.
The loaders (except for C) are plugins, and the build-in ones are installed in /usr/lib/libpeas-<version>/loaders/ as .so libraries. For example, peas-plugin-loader-python.c contains:
G_MODULE_EXPORT void peas_register_types (PeasObjectModule *module) <-- as a regular plugin { peas_object_module_register_extension_type (module, PEAS_TYPE_PLUGIN_LOADER, PEAS_TYPE_PLUGIN_LOADER_PYTHON); }
I found that too. But this is not public and not documented. It's also not installed to /usr/include/libpeas-1.0.
Yikes! Free Software not properly documented! :)
It's not just a case of bad docs here. In this case it means it's not documented because it's private to libpeas, i.e. an implementation detail. For the same reason the peas-plugin-loader.h is not installed/distributed.
While it has a plugin architecture for loaders it doesn't seem designed to allow for 3rd party loaders.
As above, I presume the idea is that new loaders should be contributed back upstream.
Since we want plugin authors to be able to create loaders we need something that's stable and documented.
Then why do you want to write our own libpeas into the existing C plugin loader? It will (at least for some time) be buggier, and less widely tested/used/maintained (ie. only by us), take lots of extra effort, not automate the bridge between core and the plugins' language, probably be less well-documented, and after all, the end result would most-likely provide an inferior experience for non-C plugin authors.
I do not want to actually write that. However, I'm not convinced of libpeas yet (especially because if the .ini aspect), and then there's no alternative left. It only has to work well for us so I think the maintainance effort is manageable (and we'd have more control).
I think you're under-estimating the effort required, even not including the proxy plugin changes you want to add in as discussed originally. The (reverse) bindings stuff to call out to the non-C plugin languages, the work involved in setting up their interpreters, and only loading them when actually needed, a way to find meta-info about them without loading them (and their interpreters), and probably much more (I guess just look at libpeas source, most of it would have to be written, presumably).
But if libpeas can actually help us I'm happy to adopt it. I really mean it.
I think it's a pretty good way forward without breaking existing loader and it also happens work nicely with adapting GObject-Introspection for auto-API-bindings, since that will auto-bind the peas extension points (reverse of API bindings), making it actually usable from non-C/GObject without extra work.
Cheers, Matthew Brush
[...]
After experimenting with this a bit, I believe it doesn't make much sense to make a compatibility layer because a) you still need basically the same code as the existing loader code, b) you would have to change/refactor/re-write lots of it c) you would still have two distinct ways to implement plugin interfaces, d) increases chances of breaking the existing loader/plugins and e) intertwining the two loaders would probably make it harder in the distant future to just drop one of them.
The problem is that libpeas demands a different method of making plugins available (.ini vs known symbols in the .so file). We definitely want only one method, even if it means transitioning the symbol method to the .ini file.
Given the choice I would go for the .ini (or similar) method, I've always worried that Geany is exposed to a crash risk by loading the .so files (and so running the module init code) just to fill in the PM. Reading a text file is much safer. And creating a text file for existing plugins should be simpler than converting any of their code.
I don't think b) is true, and for e) it would make it actually simpler if the one loader is just a plugin of the other one.
libpeas readily supports this. Unless I'm missing something.
The loaders (except for C) are plugins, and the build-in ones are installed in /usr/lib/libpeas-<version>/loaders/ as .so libraries. For example, peas-plugin-loader-python.c contains:
G_MODULE_EXPORT void peas_register_types (PeasObjectModule *module) <-- as a regular plugin { peas_object_module_register_extension_type (module, PEAS_TYPE_PLUGIN_LOADER, PEAS_TYPE_PLUGIN_LOADER_PYTHON); }
I found that too. But this is not public and not documented. It's also not installed to /usr/include/libpeas-1.0.
Yikes! Free Software not properly documented! :)
It's not just a case of bad docs here. In this case it means it's not documented because it's private to libpeas, i.e. an implementation detail. For the same reason the peas-plugin-loader.h is not installed/distributed.
While it has a plugin architecture for loaders it doesn't seem designed to allow for 3rd party loaders.
Since we want plugin authors to be able to create loaders we need something that's stable and documented.
Then why do you want to write our own libpeas into the existing C plugin loader? It will (at least for some time) be buggier, and less widely tested/used/maintained (ie. only by us), take lots of extra effort, not automate the bridge between core and the plugins' language, probably be less
Just to be clear, libpeas only automates the bridge to the *loader* and the plugins, it does nothing for the interface between the Geany/Scintilla functions and the plugins (and neither does Thomas' proposal).
That binding is independent of the loader and should be the same effort either way so long as the libpeas loaded plugins can get the plugindata pointer somehow. Then for C/C++/Vala its already done for both options, and mostly done for Python2 by reusing geanypy's bindings.
Automated bindings to Geany/Scintilla would be a huge step forward to supporting other languages, but it is a separate problem from the loader.
well-documented, and after all, the end result would most-likely provide an inferior experience for non-C plugin authors.
I do not want to actually write that. However, I'm not convinced of libpeas yet (especially because if the .ini aspect), and then there's no alternative left. It only has to work well for us so I think the maintainance effort is manageable (and we'd have more control).
The only questions I have on libpeas is:
1. what changes it requires in Geany to make Geany keybindings usable from libpeas loaded plugins, and 2. the effort to incorporate it into the existing PM.
IIUC Thomas' proposal would use the existing methods so it doesn't require any changes in these areas.
Cheers Lex
But if libpeas can actually help us I'm happy to adopt it. I really mean it.
Devel mailing list Devel@lists.geany.org https://lists.geany.org/cgi-bin/mailman/listinfo/devel
On 14-05-18 10:10 PM, Lex Trotman wrote:
[snip] 2. the effort to incorporate it into the existing PM.
I don't think this will be very difficult, even from me who loathes GtkTreeView :)
IIUC Thomas' proposal would use the existing methods so it doesn't require any changes in these areas.
But it requires writing own libpeas (ie. scanners, loaders+runtimes, +gobject-introspection-bindings, etc).
Cheers, Matthew Brush
On 19 May 2014 19:58, Matthew Brush mbrush@codebrainz.ca wrote:
On 14-05-18 10:10 PM, Lex Trotman wrote:
[snip]
- the effort to incorporate it into the existing PM.
I don't think this will be very difficult, even from me who loathes GtkTreeView :)
IIUC Thomas' proposal would use the existing methods so it doesn't require any changes in these areas.
But it requires writing own libpeas (ie. scanners,
Unless I have misunderstood Thomas (have I?) he is using the existing scanner.
loaders+runtimes,
again existing
+gobject-introspection-bindings, etc).
As said in my previous post, bindings have nothing to do with the loader.
The main difference between the two options is that libpeas can provide loaders for new languages simply (as shown by Matthew loading Python3) but at the cost of having to integrate something very different to the current system, whereas Thomas' proposal is a simple extension of the existing system to fix Geanypy problems but doesn't offer loading of any extra languages without further effort.
To my mind both approaches have their good points, and both have their downsides.
As I said before, I am not able to pick a best solution at this point.
To my mind the important question is "how hard is it to change Geany to allow libpeas loaded plugins to register keybindings?" with the secondary question of the PM as well (its more the interface from peas to the PM code than the treeview, since it has to work with existing plugins as well). If those potential problems are solved then libpeas may provide a faster path to more languages, though only C (so C++ & Vala I guess), Python, javascript and something called seed are currently available. I would really like to know what plans there are for others?
Cheers Lex
Cheers, Matthew Brush
Devel mailing list Devel@lists.geany.org https://lists.geany.org/cgi-bin/mailman/listinfo/devel
Am 19.05.2014 13:13, schrieb Lex Trotman:
On 19 May 2014 19:58, Matthew Brush mbrush@codebrainz.ca wrote:
On 14-05-18 10:10 PM, Lex Trotman wrote:
[snip]
- the effort to incorporate it into the existing PM.
I don't think this will be very difficult, even from me who loathes GtkTreeView :)
IIUC Thomas' proposal would use the existing methods so it doesn't require any changes in these areas.
But it requires writing own libpeas (ie. scanners,
Unless I have misunderstood Thomas (have I?) he is using the existing scanner.
Yes, you got this right. And I have already done it locally with +260/-160 LOCs.
loaders+runtimes,
again existing
Only geanypy, not javascript. And it'll require some more work to adapt to the new core infrastructure.
+gobject-introspection-bindings, etc).
As said in my previous post, bindings have nothing to do with the loader.
That's right. The bindings question is orthogonal to the loader-infrastructure one. But I think Matthew is working on autogenerating bindings which is great (I'll try to help where I can). Then we can focus on the loader question more easily.
The main difference between the two options is that libpeas can provide loaders for new languages simply (as shown by Matthew loading Python3) but at the cost of having to integrate something very different to the current system, whereas Thomas' proposal is a simple extension of the existing system to fix Geanypy problems but doesn't offer loading of any extra languages without further effort.
Another key difference is that in my proposal the loaders are also normal plugins (I call them pluxys) that use the same infrastructure (and therefore appear in the PM dialog). The pluxys can therefore offer more that just proxying (though I would not recommend it) and can easily be maintained by a third party. With libpeas, they come with libpeas and are separately enabled/disabled (though we could probably work the peas loaders into the PM gui somehow). On the other hand, it might be preferable to separately manage pluxys.
Yet another key difference is the .ini file approach used by libpeas vs. fixed symbols in the .so files used by Geany. I too think the .ini approach is superior but unfortunately it's a break for existing plugins.
To my mind both approaches have their good points, and both have their downsides.
As I said before, I am not able to pick a best solution at this point.
To my mind the important question is "how hard is it to change Geany to allow libpeas loaded plugins to register keybindings?" with the secondary question of the PM as well (its more the interface from peas to the PM code than the treeview, since it has to work with existing plugins as well). If those potential problems are solved then libpeas may provide a faster path to more languages, though only C (so C++ & Vala I guess), Python, javascript and something called seed are currently available. I would really like to know what plans there are for others?
seed == javascript, IIUC
I don't know what the plans are but libpeas doesn't appear to be actively developed. The last mailing list activity is from may 2012. It was a question by an ubuntu developer, and it remained unanswered. The git log is only slightly more promising.
So I absolutely do not expect that libpeas will add new languages in the short-mid term (or change at all), so (from our POV) it only adds javascript to the pool. Adding more language, which I would love to see, will require effort from our plugin authors. And I expect we would need to clone/fork libpeas (similar to scintalla) because the APIs to add language loaders are not public and presumably unstable (but again, I don't expect libpeas to change at all due to inactivity).
Sorry if I sound negative towards libpeas. I'm not fundamentally against it. I try to objectively evaluate it, and it doesn't appear to be the holy grail either, especially with our strong focus on maintaining the plugin ABI/API and exposing the current API via bindings.
Best regards.
Le 19/05/2014 14:17, Thomas Martitz a écrit :
[...]
Sorry if I sound negative towards libpeas. I'm not fundamentally against it. I try to objectively evaluate it, and it doesn't appear to be the holy grail either, especially with our strong focus on maintaining the plugin ABI/API and exposing the current API via bindings.
Amen to that. I'm not saying using Peas is not a good idea, I'm just saying we should step a little back and see things in perspective, and for what they really are. If then Peas still seems to be the most sensible, then great.
Cheers, Colomban
On 14-05-21 06:45 AM, Colomban Wendling wrote:
Le 19/05/2014 14:17, Thomas Martitz a écrit :
[...]
Sorry if I sound negative towards libpeas. I'm not fundamentally against it. I try to objectively evaluate it, and it doesn't appear to be the holy grail either, especially with our strong focus on maintaining the plugin ABI/API and exposing the current API via bindings.
Amen to that. I'm not saying using Peas is not a good idea, I'm just saying we should step a little back and see things in perspective, and for what they really are. If then Peas still seems to be the most sensible, then great.
What makes you think (some of us) haven't done that? It's not my fault people had fantasies about all the wonderment it could provide and are let down to find out that it only saves us rolling one large piece of the non-C language plugin puzzle :)
But I agree, it would be fantastic if everyone was on the same page with regards to what libpeas provides and how much of it we'll have to re-write ourselves into the existing loader if we don't use it, and what the dependencies Geany would have afterwards, and how we'd have to manually write language bridges to those hook into those languages (ex. geanypy), etc.
Cheers, Matthew Brush
Le 19/05/2014 07:10, Lex Trotman a écrit :
[...]
Just to be clear, libpeas only automates the bridge to the *loader* and the plugins, it does nothing for the interface between the Geany/Scintilla functions and the plugins (and neither does Thomas' proposal).
That binding is independent of the loader and should be the same effort either way so long as the libpeas loaded plugins can get the plugindata pointer somehow. Then for C/C++/Vala its already done for both options, and mostly done for Python2 by reusing geanypy's bindings.
Automated bindings to Geany/Scintilla would be a huge step forward to supporting other languages, but it is a separate problem from the loader.
I think that's two key points that should be clearly understood by all parts in the discussion: no matter how we load, we need a binding.
An automated binding (e.g. GI) would benefit any loading approaches, and probably is the only sensible way to go, especially for more that one non-C language.
Apart that, I basically +1 everything Lex said, and don't have much to add.
Regards, Colomban
On Mon, 19 May 2014 15:10:27 +1000 Lex Trotman elextr@gmail.com wrote:
The only questions I have on libpeas is:
- what changes it requires in Geany to make Geany keybindings usable
from libpeas loaded plugins
For the current C plugins, replacing PLUGIN_KEY_GROUP() [deprecated since 0.19] with calls to plugin_set_key_group() and keybindings_set_item(). That's 9 plugins (including my geanyinsertnum, gee), trivial change, doesn't break anything.
For geanypy, you should know better than me. The documentation mentions that "Currently using keybindings requires [...] hackery, due to Geany expecting all plugins to be shared libraries written in C".
For the plugins to come, it's a matter of how we pass keyboard callbacks to the core, not how we load them.
- the effort to incorporate it into the existing PM.
Depends on what you call "incorporate". Replacing the loader is probably a matter of two days at most. After that, plugin_help() should be replaced with automatic helper (peas plugins have help URL), and the other g_module_symbol()-loaded functions should be replaced with function calls from the plugin, or with peas extensions (g_module_symbol() won't work for non-C languages anyway).
On Tue, 13 May 2014 09:54:09 +0200 Thomas Martitz kugel@rockbox.org wrote:
Am 11.05.2014 20:31, schrieb Dimitar Zhekov:
On Sat, 10 May 2014 21:56:30 +0200 Thomas Martitz kugel@rockbox.org wrote:
On Sat, 10 May 2014 14:47:17 -0700 Matthew Brush mbrush@codebrainz.ca wrote:
+1 for some form of introspection/code generation.
Just to be clear: by "introspection", I don't mean *replacing* struct GeanyDocument with GObject GGeanyDocument and so on for the entire API. We may have a GGeanyDocument which represents (a) the GeanyDocument structure, (b) all document_*() functions as geany_document_*() and (c) all "document-" signals as "geany-document" with a GGeanyDocument argument. We can't generate signals for GGeanyDocument "fields" being changed directly from C, but we don't support such signals anyway.
That's two GeanyDocument APIs to maintain.
You don't want to maintain a mapping of GeanyDocument, but are have not problems with maintaining 5 manually written introspections for it?..
and it requires a lot of changes to Geany (doesn't it?).
It will require some changes, for example GGeanyDocument will not be straight-forward because of our organisation of the document list. Most will be additions though, with few (if any) API breaks.
So it does require non-trivial changes to the core, too?
Anything except manually written introspections for each language requires changes in the core. I suggested a way to minimize them.
GeanyPy proxy = ~150 KB. 5 language proxies = ~750 KB, code that will have to be written and maintained. For comparision, geany/src/* = 1.9 MB.
peas loader = ~15 KB, language support is in the core, and doesn't change with the new Geany versions, so you don't have to worry about any language(s) becoming unsupported/unmaintained.
Don't be ridiculous. This calculation is severely flawed and you know that.
I don't know anything like that. 150 KB per proxy is the best estimate we have at the moment.
To make an omelette, you have to break the eggs.
No, you don't. This can be implemented without breakage (at least for C plugins, probably for existing python ones too).
Since you have the enthusiasm to write additional language support, not me, and you are not willing to accept any suggestions, this discussion is pointless.
On 14-05-10 12:06 PM, Dimitar Zhekov wrote:
[snip] For python, maybe we can include geanypy/src/*.c in Geany or in a .so library. But if we don't want to end up with more 150+ KB proxies like geanypy, we'd better use introspection. I really admire Matthew and Lex for writing this thing, but that's hardly the best approach, and I don't expect many people to repeat it and maintain such proxy.
You can say it, it's actually the worst approach :)
It's works OK though, doesn't require additional tools (eg. SWIG/Cython/etc) and is probably more efficient/smaller than using FFI stuff from ctypes module, although it probably doesn't really matter here.
+1 for some form of introspection/code generation.
Cheers, Matthew Brush
Hi Thomas,
A somewhat similar method that I had a (very rough) prototype for, used a .so wrapper for each non-native plugin (I was using python and used most of geanypy) providing the API wrapper for that language. There was a "language manager" plugin for the language which did the initialising for the language (in Python that was PyInitialize).
Note that none of the communication to the plugin went through the language manager, so it was no longer a proxy, each plugin was a full strength first class plugin with all the capabilities (and responsibilities). The wrapper could provide as thin or as thick an adapter as the language needed.
This seemed to work, and so long as the language manager didn't Pyfinalise the plugins kept working after it was closed. What I hadn't gotten around to doing was your proposed communication between the language manager and Geany so the plugins could be in the plugin manager list.
Unfortunately I can't find the code ATM, and may have lost it in one of the disk crashes (backups??) but hopefully the information that it is possible will be useful.
Cheers Lex
[...]
Sorry for the double post, but Murphys law says I found the code just *after* posting :) More details below.
On 8 May 2014 13:03, Lex Trotman elextr@gmail.com wrote:
Hi Thomas,
A somewhat similar method that I had a (very rough) prototype for, used a .so wrapper for each non-native plugin (I was using python and used most of geanypy) providing the API wrapper for that language. There was a "language manager" plugin for the language which did the initialising for the language (in Python that was PyInitialize).
Note that none of the communication to the plugin went through the language manager, so it was no longer a proxy, each plugin was a full strength first class plugin with all the capabilities (and responsibilities). The wrapper could provide as thin or as thick an adapter as the language needed.
This seemed to work, and so long as the language manager didn't Pyfinalise the plugins kept working after it was closed. What I hadn't gotten around to doing was your proposed communication between the language manager and Geany so the plugins could be in the plugin manager list.
In fact I got it to appear in the Geany plugin manager by:
1. the Python language manager plugin scanned the Python plugins directory and copied a numbered .so into the Geany plugins directory where the Geany plugin manager would see it.
2. when the numbered .so is loaded it asks the language manager (using its number) which python plugin it should load.
3. it then g_module_opens the Python bindings, which load and run the Python plugin returning the information to the Geany PM.
This is pretty tricky but it makes the python plugins appear in the geany plugin manager as if they are C ones so they have full capabilities (and especially can have keybindings, something that doesn't work right in Geanypy) and it doesn't need any modifications to Geany itself, so the API is unchanged.
Cheers (again) Lex
Unfortunately I can't find the code ATM, and may have lost it in one of the disk crashes (backups??) but hopefully the information that it is possible will be useful.
Cheers Lex
[...]