[Geany-Devel] [FT-plugins] Proposed "Features"

[Lex Trotman]

[...]>>
>> std::vector has a second parameter, which has a default value so its
>> legal to omit it, but the type you will get from an accurate parser to
>> store in TM will be the complete type which will be something like
>> std::vector<int, std::allocator<int>> not just plain std::vector<int>.
>> This is the type you see printed in g++ and clang++ error messages for
>> template expansions, very messy.
>>
>> So the way TM does a pure textual comparison of the prefix in the
>> source code `std::vector<int>` simply won't work to find
>> `std::vector<int,std::allocator<int>>`
>>
>> It also won't work if there are spaces as I showed in a previous post
>> `std::vector< int >` is perfectly legal C++ but won't match
>> `std::vector<int>`.  And before you ask, yes lots of code has the
>> spaces so that nested angle bracket closes (as in my
>> ...allocator<int>> above) look like > > not the >> operator.  Being
>> able to omit the space and not have it seen as a shift right operator
>> is only a recent addition to the standard.
>
>
> OK, so we'd have to have a canonical representation of types (how this looks
> like is defined by the ft-plugin, e.g. whitespaces stripped, default
> parameters expanded) to compare against, and a method to find that tag when
> looking up std::vector< int >. It seems be complicated indeed, but doable.
> Most of the complexity will be in the ft-plugin providing canonicalized,
> tag-like data.

There is no guarantee that a gcc based library (assuming RMS allows it
one day) would use the same canonical form as libclang, in fact it
probably deliberately would not, so its hard to swap them, so now you
not only have language specific code in Geany, but FT-plugin specific
code.  And for other languages FT-plugins too.

And it still doesn't address the default template parameter issue above.

>
>>
>> And for template functions declared as `template<class T>T& f(T& t){
>> return t<<1; }` they are used as simply `int a = f(1);` not an angle
>> bracket in sight, and a textual comparison to find the declaration for
>> a calltip isn't going to work because no such explicit declaration
>> exists. So how can TM answer whats legal here `f(std::cout).` (answer
>> all of std::ostream) versus here `f(1).` (answer nothing).
>
>
> I would say that a calltip for a templated function should show the generic
> function since you don't know what the user is going to type, regardless of
> the design (except perhaps if the specialized function can be inferred from
> the scope). Which specialized function is to be called changes as the user
> types. As I said in the other mail to Matthew, trying to infer from the lhs
> of the assignment is dangerous and impractical too (I would not want this
> behavior), especially if the lhs is determined only by the rhs (auto x =
> foo(1)).

Yes, in C++ you can't infer from left to right, but there are
languages where this happens, Haskel, O'Caml that I immediately think
of.  But my example was just to point out that the template parameter
is inferred from the type of the function parameter, the user does not
specify `f<int>(1)` sorry for the confusion.

And yes, as I showed in another post, accurate calltips can change as
parameters are added.

But the last two examples above are not calltips, they are
autocomplete, the function call is finished.

Both end in a dot accessing members of the return from the function.
To get accurate type info it is necessary to first recognise that a
template is needed for `f` (there are no <>s in the expressions) then
recognise that its return type matches its parameter type, then you
can suggest what members can come after the dot.

And just to note neither the return or the function parameter need to
be exactly the template parameter, for example `template<class T> T&
f(T* a)` will happily deduce a `T` from a `T*` and make the return
type `T&`.  And several other type expressions work.

>
> Perhaps you can show specialized calltips once user has entered a few
> parameters but that's still guessing. I don't think this is a good idea.
>

But the FT-plugin may do this (at least IIUC libclang will) so its
fine if Geany doesn't, but it has to ask the FT-plugin not use TM.

> But plain textual comparison against the function name seems to work best
> here to me.
>

Depends what you are coding, as the car manufacturers say, "your
mileage may vary".

>
>> [...]
>>>>
>>>> Good point, the FT-Plugin API needs functions for goto declaration and
>>>> goto definition.  Unlike TM, the plugin can take language and scope
>>>> rules into account when it answers, so you get the right declaration,
>>>> continuing the theme above, where would you go to find the declaration
>>>> of std::vector<int> ?
>>>>
>>>> Certainly not the line:
>>>>
>>>> std::vector<int> list_of_ints;
>>>>
>>>> thats the declaration of `list_of_ints` but thats the only place that
>>>> `std::vector<int>` exists in the code.
>>>
>>>
>>>
>>> Ideally the header which defines std::vector<A> would be found (and maybe
>>> opened) right? Currently this isn't possible with TMTag (as var_type can
>>> only contain either vector<A> or vector<int>) but it shouldn't be too
>>> hard
>>> to make it work.
>>
>> But now you are encoding language specific semantics into TM, and as I
>> noted above its more complicated than just that example.
>
>
> Generic ways to deal with language specific properties isn't quite the same.
> Also, this doesn't apply to just C++, other languages have complex meta
> programming / generic types too (Vala, Java, C#, etc.).
>
> And even then, I'm totally fine with TM knowing language specifics.

I keep saying Geany as well, not just TM, the language specifics to
storing stuff is one thing, the language specifics for using that
stored stuff is another more complex thing.  And, if you have to ask
the plugin for the more complex questions, there is no point of
storing all the gory details in TM, because the plugin isn't going to
use it.

> TM can
> hold every information needed to make smart language specific features work,
> preferably in generic ways. That doesn't mean TM has to semantically
> understand the language's source code, if the information is provided by
> external parsers. In fact, it doesn't have to understand source code itself
> at all.
>

So as originally proposed, Geany has to ask the plugin first.

> There may be a point we want to limit the smartness anyway, if requires very
> complex solutions for little gain, extremely rare use cases or they it can
> become annoying to the user (e.g. if it requires to much setup).
>

To emphasise I am not saying don't make TM and Geany smarter, but I am
saying don't make a `c.c` like mess of language specific code in Geany
or TM, and don't say "this is good enough for me so it should be good
enough for you" and prevent the use of smarter plugins for those who
want them.  That means the plugin has to bypass the code in TM/Geany
to bypass its restrictions.

>> Please don't get the idea from simple examples posted here that they
>> cover even a fraction of the craziness of languages (and not just C++
>> either, its simply the one I'm most familiar with).
>>
>>   Like adding a generic_type field which is set for each
>>>
>>> template instance (containing vector<A>), then instead of looking up the
>>> decl of generic_type if it's not NULL, otherwise var_type.
>>>
>>> Of course, the TMTags associated with vector<A> and vector<int> would
>>> have
>>> to provided by the ft-plugin too (maybe there needs to be a new TMTagType
>>> for vector<A> too).
>>
>> And don't forget that when trying to look up
>> `std::vector<int>::value_type` to see whats next, or to find the
>> declaration, either Geany or TM has to parse it and break it into the
>> elements `std`, `vector<int>` and `value_type`, ie has to know the
>> language syntax for template specialisations and namespaces,
>> `vector<int>` is not a legal name.  And as I said before, that syntax
>> varies between languages, so again you are encoding language specifics
>> into Geany or TM.
>
>
> I don't understand that. TM doesn't need to know that. It just has some
> strings contained in tags that are matched against other strings. TM doesn't
> break A::B::X, the parser does (currently this is split into A::B and X). TM
> already has the concept of scope for that. It works as of now, without
> knowing langauge specific syntax.
>

ATM Geany does that breakup of the expression in the source code and
asks TM for each element.

As I said above, if the plugin parser is doing it, the plugin parser
will look it up in its accurate language specific symbol table as
well, it won't use TM.  So no need to store language specifics in TM
if you are using such a library.

To go back to one of your very initial comments "don't want to move
everything from TM/Geany to the plugin" my intention in all these
examples is to show that an accurate plugin MUST bypass TM/Geany
because they will always have limitations without effectively having
the same code as the plugin, ie a compiler.

>
>>
>> And in a final real piece of evil C++ for the day, variadic templates
>> with variable numbers of parameters (like C variadic functions with
>> ...).
>>
>> std::get<1>(std::make_tuple(1 ,std::string("abc"), 3)).size(
>>
>> ok, lets lookup the return type of std::make_tuple
>> http://en.cppreference.com/w/cpp/utility/tuple/make_tuple thats
>> simple-ish
>> and the type of std::get<1>
>> http://en.cppreference.com/w/cpp/utility/tuple/get hmmmm
>>
>> And yep its all completely legal and idiomatic C++ and totally static
>> typed so should be no problem analysing it.
>>
>> Sorry Colomban you didn't want see half the craziness of C++ I know :)
>>
>
> I think we the above suggestions this should work. Sure it must be parsed by
> a ft-plugin.
>
> You're heavy on templates which require compilation. I'm interested to see
> how you want realize this (passing CFLAGS etc.) in a user friendly way on a
> per-ft-plugin basis.
>

Yes, to get the benefit of the plugins wisdom, the price is that it
needs to know how the software is to be built (at least for libclang).
But thats the plugins problem, not Geanys.  Its a trade off.  It might
not be a worthwhile trade off for you, but don't disallow usage by
others for whom the tradeoff is different.  And as its a plugin you
can decide to not use it.

>>>>>> Of course moving the problem to plugin space doesn't mean the plugin
>>>>>> can't use Geany facilities where their capabilities fit the
>>>>>> requirement.  But we should not try to expand Geany to handle all
>>>>>> types of functionality.
>>>>>>
>>>>>> Like your TM query interface, the plugins should answer the questions
>>>>>> like "whats the autocomplete here", "what calltips are relevant here"
>>>>>> with a flat list of data relevant to the question.
>>>>>
>>>>>
>>>>> My TM query interface wants to return all matching tags, including
>>>>> those
>>>>> found by ft-plugins. Can this be done?
>>>>
>>>> Only if your query plugin queries the FT plugin.
>>>>
>>> So that means
>>>
>>> 1) My plugin contains code specifically for each known ft-plugin (i.e.
>>> not
>>> the plugin API)
>>
>> No, but the implementation inside Geany of the query interface that
>> you have added in your PR may have to know to call the FT-plugin to
>> answer the query, but the plugin using the query interface doesn't
>> need to know about that.
>
>
> Okay. That's at least one more good reason to have the query interface :-)

I havn't looked in detail at your query interface, but the approach of
providing general wrapper APIs for external use, instead of exposing
the guts of Geany, I very much support.  Remember waaaay back when we
were doing the build system, I proposed a set of functions to expose
it to the plugin interface in a generic manner, but they were vetoed
by Nick.


Cheers
Lex

>
> Best regards
>
> _______________________________________________
> Devel mailing list
> Devel at lists.geany.org
> https://lists.geany.org/cgi-bin/mailman/listinfo/devel