[Geany-Devel] [FT-plugins] Proposed "Features"

[Thomas Martitz]

Am 01.09.2016 um 15:00 schrieb Lex Trotman:
> [...]
>>> It is of course always "possible", after all its a mere matter of
>>> programming :)
>>>
>>> And of course an instantiated class template is just a type, but what
>>> is the name of std::vector<int> and std::vector<A> so it can be looked
>>> up?
>>
>> vector<int> and vector<A>, with scope being std (I think trailing scope
>> delimiters are not saved in the scope string of TMTag, just intermediate
>> ones).
>>
> Sorry Thomas, I was a little mean and laid a trap.


No problem. The devil is always in the details and we can only define 
requirements when looking and complicated cases.


>
> std::vector has a second parameter, which has a default value so its
> legal to omit it, but the type you will get from an accurate parser to
> store in TM will be the complete type which will be something like
> std::vector<int, std::allocator<int>> not just plain std::vector<int>.
> This is the type you see printed in g++ and clang++ error messages for
> template expansions, very messy.
>
> So the way TM does a pure textual comparison of the prefix in the
> source code `std::vector<int>` simply won't work to find
> `std::vector<int,std::allocator<int>>`
>
> It also won't work if there are spaces as I showed in a previous post
> `std::vector< int >` is perfectly legal C++ but won't match
> `std::vector<int>`.  And before you ask, yes lots of code has the
> spaces so that nested angle bracket closes (as in my
> ...allocator<int>> above) look like > > not the >> operator.  Being
> able to omit the space and not have it seen as a shift right operator
> is only a recent addition to the standard.

OK, so we'd have to have a canonical representation of types (how this 
looks like is defined by the ft-plugin, e.g. whitespaces stripped, 
default parameters expanded) to compare against, and a method to find 
that tag when looking up std::vector< int >. It seems be complicated 
indeed, but doable. Most of the complexity will be in the ft-plugin 
providing canonicalized, tag-like data.

>
> And for template functions declared as `template<class T>T& f(T& t){
> return t<<1; }` they are used as simply `int a = f(1);` not an angle
> bracket in sight, and a textual comparison to find the declaration for
> a calltip isn't going to work because no such explicit declaration
> exists. So how can TM answer whats legal here `f(std::cout).` (answer
> all of std::ostream) versus here `f(1).` (answer nothing).

I would say that a calltip for a templated function should show the 
generic function since you don't know what the user is going to type, 
regardless of the design (except perhaps if the specialized function can 
be inferred from the scope). Which specialized function is to be called 
changes as the user types. As I said in the other mail to Matthew, 
trying to infer from the lhs of the assignment is dangerous and 
impractical too (I would not want this behavior), especially if the lhs 
is determined only by the rhs (auto x = foo(1)).

Perhaps you can show specialized calltips once user has entered a few 
parameters but that's still guessing. I don't think this is a good idea.

But plain textual comparison against the function name seems to work 
best here to me.


> [...]
>>> Good point, the FT-Plugin API needs functions for goto declaration and
>>> goto definition.  Unlike TM, the plugin can take language and scope
>>> rules into account when it answers, so you get the right declaration,
>>> continuing the theme above, where would you go to find the declaration
>>> of std::vector<int> ?
>>>
>>> Certainly not the line:
>>>
>>> std::vector<int> list_of_ints;
>>>
>>> thats the declaration of `list_of_ints` but thats the only place that
>>> `std::vector<int>` exists in the code.
>>
>>
>> Ideally the header which defines std::vector<A> would be found (and maybe
>> opened) right? Currently this isn't possible with TMTag (as var_type can
>> only contain either vector<A> or vector<int>) but it shouldn't be too hard
>> to make it work.
> But now you are encoding language specific semantics into TM, and as I
> noted above its more complicated than just that example.

Generic ways to deal with language specific properties isn't quite the 
same. Also, this doesn't apply to just C++, other languages have complex 
meta programming / generic types too (Vala, Java, C#, etc.).

And even then, I'm totally fine with TM knowing language specifics. TM 
can hold every information needed to make smart language specific 
features work, preferably in generic ways. That doesn't mean TM has to 
semantically understand the language's source code, if the information 
is provided by external parsers. In fact, it doesn't have to understand 
source code itself at all.

There may be a point we want to limit the smartness anyway, if requires 
very complex solutions for little gain, extremely rare use cases or they 
it can become annoying to the user (e.g. if it requires to much setup).

> Please don't get the idea from simple examples posted here that they
> cover even a fraction of the craziness of languages (and not just C++
> either, its simply the one I'm most familiar with).
>
>   Like adding a generic_type field which is set for each
>> template instance (containing vector<A>), then instead of looking up the
>> decl of generic_type if it's not NULL, otherwise var_type.
>>
>> Of course, the TMTags associated with vector<A> and vector<int> would have
>> to provided by the ft-plugin too (maybe there needs to be a new TMTagType
>> for vector<A> too).
> And don't forget that when trying to look up
> `std::vector<int>::value_type` to see whats next, or to find the
> declaration, either Geany or TM has to parse it and break it into the
> elements `std`, `vector<int>` and `value_type`, ie has to know the
> language syntax for template specialisations and namespaces,
> `vector<int>` is not a legal name.  And as I said before, that syntax
> varies between languages, so again you are encoding language specifics
> into Geany or TM.

I don't understand that. TM doesn't need to know that. It just has some 
strings contained in tags that are matched against other strings. TM 
doesn't break A::B::X, the parser does (currently this is split into 
A::B and X). TM already has the concept of scope for that. It works as 
of now, without knowing langauge specific syntax.


>
> And in a final real piece of evil C++ for the day, variadic templates
> with variable numbers of parameters (like C variadic functions with
> ...).
>
> std::get<1>(std::make_tuple(1 ,std::string("abc"), 3)).size(
>
> ok, lets lookup the return type of std::make_tuple
> http://en.cppreference.com/w/cpp/utility/tuple/make_tuple thats
> simple-ish
> and the type of std::get<1>
> http://en.cppreference.com/w/cpp/utility/tuple/get hmmmm
>
> And yep its all completely legal and idiomatic C++ and totally static
> typed so should be no problem analysing it.
>
> Sorry Colomban you didn't want see half the craziness of C++ I know :)
>

I think we the above suggestions this should work. Sure it must be 
parsed by a ft-plugin.

You're heavy on templates which require compilation. I'm interested to 
see how you want realize this (passing CFLAGS etc.) in a user friendly 
way on a per-ft-plugin basis.

>>>>> Of course moving the problem to plugin space doesn't mean the plugin
>>>>> can't use Geany facilities where their capabilities fit the
>>>>> requirement.  But we should not try to expand Geany to handle all
>>>>> types of functionality.
>>>>>
>>>>> Like your TM query interface, the plugins should answer the questions
>>>>> like "whats the autocomplete here", "what calltips are relevant here"
>>>>> with a flat list of data relevant to the question.
>>>>
>>>> My TM query interface wants to return all matching tags, including those
>>>> found by ft-plugins. Can this be done?
>>> Only if your query plugin queries the FT plugin.
>>>
>> So that means
>>
>> 1) My plugin contains code specifically for each known ft-plugin (i.e. not
>> the plugin API)
> No, but the implementation inside Geany of the query interface that
> you have added in your PR may have to know to call the FT-plugin to
> answer the query, but the plugin using the query interface doesn't
> need to know about that.

Okay. That's at least one more good reason to have the query interface :-)

Best regards