[Geany-devel] In-memory tagmanager parsing

[Colomban Wendling]

Enrico Tröger a écrit :
> On Sun, 18 Apr 2010 15:48:14 +0200, Colomban wrote:
> 
>> Hi,
>>
>> I took a look at the tagmanager in an attempt to make it work
>> in-memory, since it is not yet completely the case.
>> I found why and how to fix the (C?) function arguments (attached
>> patch), and it seems to work pretty well for C (and Python and PHP,
>> though I did only a few tests with them).
> 
> What exactly do you mean?
How, yep I see my sentence was quite confusing... sorry.
> Do you refer to the broken function signature parsing when the
> in-memory parsing is enabled? But how is this related to Python and PHP
> parsers as those parsers work completely differently?
Yes I refer to the function signature parsing, which affects the C parser.
I spoke of Python and PHP only because I enabled the
update_tags_from_buffer() #if0'ed code and they seemed to work quite
well with it. But it seems from what you says below that it was a very
fast and wrong assumption that it means it actually use in-memory parsing...

>> For now the only defect I saw (with C) is with anonymous enumerations
>> and so (that are named anon_enum_NUM) with which the NUM increases at
>> each re-parsing (I use document_update_tag_list(doc, TRUE) to
>> re-parse); but I suppose it wouldn't it too difficult to fix, would
>> it?.
> 
> No idea. I don't know the C parser that well (tagmanager/c.c).
> I always tried to touch it as less than possible as it is very complex
> and very easy to break it :(.
Yep, true. Anyway hum, what to say of a code that use longjump()? :-'
Not to blame anyone, I never felt it.
And for my original question, I think I'll just... see?

>> Then, I would like to know what else didn't work, what need to be
>> fixed, etc.; because I'd really love to have it working.
> 
> The parsers...
Haha! I must admit I hoped a more precise answer :D

>> Enrico Tröger wrote: (on thread Function Definition)
>>> Some time ago, I started working on this but it never really worked
>>> and additionally, it could work currently only for a few parsers
>>> (some of those are C, Fortran, SQL IIRC). To get it working
>>> reliably, some more work is needed and we would have to adjust *all*
>>> existing parsers which is by no means an easy task.
>> Do you know which parsers would *not* work? And hum, if each and every
> 
> All except C, Fortran, SQL and JavaScript.
> I searched my personal mail archives, the Geany and Geany-devel mailing
> list archives and everything but I didn't find any correspondance where
> I mentioned that experiemental code I committed :(.
> Though I was very sure (until I didn't find any related mail) that I
> talked to Nick about this, either via personal mail or via a mailing
> list. Either my memories kid on me or I lost those mails.
> 
> Nick do you remember any conversion with me about this and maybe still
> have the relevant mails?
> 
> Anyway, best documentation is the source, haha. The whole stuff have
> been added in SV r3184
> (http://geany.svn.sf.net/viewvc/geany?view=rev&revision=3184).
Ow. Well, indigestible but still instructive, thanks.
But I'll wait hoping for Nick's remembrance, still.

> I don't remember all details (sigh, this is long time ago :( ), but I
> think the main problem was that most parsers read the data from the
> file so that it could not be easily converted to read from a buffer.
> Only the few above mentioned parsers did it so that we could change the
> underlying data to be a buffer instead of a real file.
Hum, I don't fully understand. By "it" in the last sentence you mean
read.[ch]?

>> parser must care about buffer VS file, wouldn't it be good to abstract
>> this a little more? (with e.g. a little I/O layer - I already started a
>> small library to check if it would be easy to emulate file I/O on
>> buffer, and it seems not to be too hard)
> 
> Yeah, that would be a clean and proper solution and would probably
> solve a lot of problems.
> But before doing this, we need to decide whether we want to stay as
> compatible as possible with the CTags project(which makes very slow
> progress, almost dead) as we did before or whether we would spend time
> on modernising the parsers and adjust them to work more like we need it
> (not sure how many differences there would be at all though).
> 
> Once this decision is made, we can think about your question above
> about an I/O abstracting layer.
Yeah. I might say if it is so dead, it is hope not so far from blindness
to wait for updates and fixes from it.
But OTHO I completely understand that the simple idea of being the
maintainer of it might be quite... scary.

> I guess when we really
> get to get it working, it's being a great thing for users, i.e. us and
> many more :D.
Definitely!

Regards,
Colomban