[Geany-Devel] [FT-plugins] Proposed "Features"
kugel at xxxxx
Wed Aug 31 11:22:09 UTC 2016
Am 31.08.2016 um 13:03 schrieb Jiří Techet:
> On Tue, Aug 30, 2016 at 11:29 PM, Thomas Martitz <kugel at rockbox.org
> <mailto:kugel at rockbox.org>> wrote:
> Am 30.08.2016 um 21:10 schrieb Jiří Techet:
> Geany would then merge the tags, perhaps giving the plugin
> ones more
> weight, and store it in TM.
> I think you underestimate how many tags we're talking
> here. The
> example libclang ft-plugin would have to re-walk the
> entire AST
> (which is absolutely massive, particularly for C++),
> convert it to
> TM tag structures, and then Geany/TM would have to perform
> merging logic, would would be more complicated than now if
> it was
> to support C++ properly, every single re-parse. My
> intuition tells
> me that just won't be fast enough, Clang already jumps through
> hoops and uses tricks to just build its own AST in-time.
> I think it would be a disaster performance-wise. The number of
> AST nodes can be easily 100x more than the amount of tags we
> have from ctags (we get a single tag for a function now and
> AST will contain complete tree for the function body) so just
> this might cost 100x more. In addition all the necessary
> copies to TM internal representation, having to maintain the
> tree structure (in TM we use GPtrArrays for everything which
> are very efficient and during tag merge we try to eliminate
> even pointer dereferences because those start getting
> expensive when managing many tags) etc.
> Let's not outright reject possible solutions based on performance
> assumptions and guestimations. Performance can be evaluated on an
> actual implementation. Until then it's simply an invalid argument
> for this discussion. But FWIW, I don't think performance is the
> driving aspect.
> No, performance is a very valid point. Tag updates don't happen in a
> background thread in Geany but rather on the main thread (and changing
> this would require lots of modifications as neither ctags, nor TM nor
> Geany are thread-safe) and all updates have to happen in a really
> short time period - you cannot make the GUI freeze while the user is
> typing so you have 100ms at most without any noticeable delay.
I'm saying we can't evaluate performance at this time, because no
implementation is available. So saying now "uhm I fear it might be too
slow my guess is that the other one is 100x faster" is just a wild
assumption that doesn't help except spraying FUD. And outright rejecting
a proposal based on such assumptions is invalid.
Please lets evaluate solutions, implement them, and then have an eye on
> What's needed from the AST is tags (as you and I mentioned
> elsewhere, AST is the complete code representation, so much more
> than what's required currently). Those can be extracted in one
> pass. I don't see that tags need to be converted back to the
> original AST.
> As Matthew said, tags are insufficient for good autocompletion (e.g.
> consider dynamic languages like Python where you have to infer
> variable type from the right-hand side of an assignment and based on
> that generate an autocompletion list).
Tags with incomplete information are insufficient. One solution can be
to have tags hold sufficient information.
FWIW, unless you actually compile the stuff (for C++), any source code
analysis will be insufficient for some cases. I was under the impression
that we do *not* want mandatory compilation, which drags in build system
implications, for stuff like auto completion.
But if a smart ft-plugin does this I don't care. I only care about
if/how Geany - and other non-ft-plugins - can use the data from such an
> For any successful plugin operation, the AST has to be generated
> (and probably re-generated regularly) and traversed at least once.
> Creating tags in a traversal that's happening done anyway probably
> isn't even going to add much overhead, if any.
> But I assume AST is created in a background thread - as I said, all TM
> code runs on the main thread and everything has to happen fast enough
> between keystrokes so users don't experience any delays while typing.
I don't care how the ft-plugin that provides tags to Geany generates
it's internal AST. It can surely use a background thread. This is quite
> As you say, we use a GPtrArray of tags because it's very efficient
> for sorting and merging. I think any ft-plugin will also have to
> sort (at least) for showing auto-completion and symbols tree - if
> it shows them itself. So it may even have to create such
> array/lists that TM uses anyway (you can't sort AST directly). So
> it might as well pass them to TM for sorting.
> Whatever you do with tiny lists (say 1000 items) of tags which appear
> as a result of autocompletion doesn't matter. I'm talking about
> workspace array of tags which may contain all tags from a project with
> hundreds of thousands or millions of tags and which have to be updated
> as the user types and these updates have to be handled in a very
> efficient way.
Wait, a AST comprising a million nodes doesn't result in a million
autocompletion candidates. Anyway, how does the number relate to our
> And even if we did this, I don't know how we could handle ASTs
> of different languages in a generic way because these will
> differ significantly.
> One more time, seems I wasn't clear enough yet: I'm *not*
> suggesting that we create generic code inside Geany that handles
> any kind of AST. What I suggest is that plugins that use AST
> internally (or not) pass tag-like information to Geany, extracted
> and flattened from its internal AST (or whatever it uses). Then
> Geany and other plugins can use that information for their own
> Again, for smart autocompletion you will need things from AST.
> The biggest problem of current scope completion in Geany is the lack
> of information about local variables which we might get with the new
> cxx ctags parser and add them to TM. We'll get better results then but
> still AST-based autocompletion will be able to do smarter things.
Again, using AST is fine, within the ft-plugin. Use whatever you need
from the AST for smart completion, and pass it in a generic format to
Geany. This way Geany *will use* AST-based information just fine.
More information about the Devel