[Geany-Devel] [FT-plugins] Proposed "Features"

[Jiří Techet]

On Tue, Aug 30, 2016 at 11:29 PM, Thomas Martitz <kugel at rockbox.org> wrote:

> Am 30.08.2016 um 21:10 schrieb Jiří Techet:
>
>         Geany would then merge the tags, perhaps giving the plugin
>>         ones more
>>         weight, and store it in TM.
>>
>>
>>     I think you underestimate how many tags we're talking here. The
>>     example libclang ft-plugin would have to re-walk the entire AST
>>     (which is absolutely massive, particularly for C++), convert it to
>>     TM tag structures, and then Geany/TM would have to perform some
>>     merging logic, would would be more complicated than now if it was
>>     to support C++ properly, every single re-parse. My intuition tells
>>     me that just won't be fast enough, Clang already jumps through
>>     hoops and uses tricks to just build its own AST in-time.
>>
>>
>> I think it would be a disaster performance-wise. The number of AST nodes
>> can be easily 100x more than the amount of tags we have from ctags (we get
>> a single tag for a function now and AST will contain complete tree for the
>> function body) so just this might cost 100x more. In addition all the
>> necessary copies to TM internal representation, having to maintain the tree
>> structure (in TM we use GPtrArrays for everything which are very efficient
>> and during tag merge we try to eliminate even pointer dereferences because
>> those start getting expensive when managing many tags) etc.
>>
>
>
> Let's not outright reject possible solutions based on performance
> assumptions and guestimations. Performance can be evaluated on an actual
> implementation. Until then it's simply an invalid argument for this
> discussion. But FWIW, I don't think performance is the driving aspect.
>

No, performance is a very valid point. Tag updates don't happen in a
background thread in Geany but rather on the main thread (and changing this
would require lots of modifications as neither ctags, nor TM nor Geany are
thread-safe) and all updates have to happen in a really short time period -
you cannot make the GUI freeze while the user is typing so you have 100ms
at most without any noticeable delay.


>
> What's needed from the AST is tags (as you and I mentioned elsewhere, AST
> is the complete code representation, so much more than what's required
> currently). Those can be extracted in one pass. I don't see that tags need
> to be converted back to the original AST.
>

As Matthew said, tags are insufficient for good autocompletion (e.g.
consider dynamic languages like Python where you have to infer variable
type from the right-hand side of an assignment and based on that generate
an autocompletion list).


>
> For any successful plugin operation, the AST has to be generated (and
> probably re-generated regularly) and traversed at least once. Creating tags
> in a traversal that's happening done anyway probably isn't even going to
> add much overhead, if any.
>

But I assume AST is created in a background thread - as I said, all TM code
runs on the main thread and everything has to happen fast enough between
keystrokes so users don't experience any delays while typing.


>
> As you say, we use a GPtrArray of tags because it's very efficient for
> sorting and merging. I think any ft-plugin will also have to sort (at
> least) for showing auto-completion and symbols tree - if it shows them
> itself. So it may even have to create such array/lists that TM uses anyway
> (you can't sort AST directly). So it might as well pass them to TM for
> sorting.


Whatever you do with tiny lists (say 1000 items) of tags which appear as a
result of autocompletion doesn't matter. I'm talking about workspace array
of tags which may contain all tags from a project with hundreds of
thousands or millions of tags and which have to be updated as the user
types and these updates have to be handled in a very efficient way.


>
>
>
>
>> And even if we did this, I don't know how we could handle ASTs of
>> different languages in a generic way because these will differ
>> significantly.
>>
>>
> One more time, seems I wasn't clear enough yet: I'm *not* suggesting that
> we create generic code inside Geany that handles any kind of AST. What I
> suggest is that plugins that use AST internally (or not) pass tag-like
> information to Geany, extracted and flattened from its internal AST (or
> whatever it uses). Then Geany and other plugins can use that information
> for their own purposes.


Again, for smart autocompletion you will need things from AST.

The biggest problem of current scope completion in Geany is the lack of
information about local variables which we might get with the new cxx ctags
parser and add them to TM. We'll get better results then but still
AST-based autocompletion will be able to do smarter things.

Cheers,

Jiri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geany.org/pipermail/devel/attachments/20160831/0dd15595/attachment.html>