[Geany-devel] Request: multithreaded tag generation?

[Colomban Wendling]

Hi,

Le 07/11/2011 16:35, Harold Aling a écrit :
> Dear Geany Devs,
> 
> I recently switched from GeanyPRJ to Gproject. Since Gproject doesn't
> support multiple open projects I have to switch between projects, but
> it takes up to 4 minutes to close one project and open another. A
> project consists of roughly 1000-2000 php-related files.
> 
> The "Generate tags for all project files" causes this massive delay,
> but I really need that feature.
> 
> At work I have a 2-core CPU, where 1 is completely idle and on my
> desktop at home there are 5 cores are doing nothing while generating
> tags. Can't they be utilized to speed up the tag generation?

TL;DR: it's really not that easy.

Long story:

The tag parsers cannot use more than one thread, but we could imagine
running more than one at the same time in different threads.  Not sure
what is Geany's and GProject deal here, so I'll speak about Geany; Jiřì
could probably answer about the GProject part.

Well so, running the parser in it own thread.  It's a good idea I
already tries to implement for real-time tag parsing (for it to be not
intrusive in the parsing part), but it's not easy.
We need two things:

1) A function that can run the parser and notify when done.
2) Collect some return value and data
3) Make sure nothing is accessed at the same time between more than one
single thread (for the young ppl that never tried, without this
everything goes wild :D)

1 alone is easy.  2 alone is easy.  But combining 1, 2 and 3 isn't,
because currently we could do 1) in Geany side, but 2 needs to call a TM
function which wouldn't know about 3.
What is needed then is to implement all in the TM side, and this needs
quite a few changes not to access tm_file->tags during the update.


Well, that's just rough answer mostly take from my memory at trying to
implement it once, maybe I missed some parts.

To conclude, yes it'd be good to have this at some point, but it's far
from being trivial to implement, so it will need somebody to dig deep[2].

Regards,
Colomban


[1] One could argue we should drop it in favor of custom code because TM
is very hard to understand (to me), has a few annoying bugs and is
overkill, but we do use it ATM…  and nobody yet tried to get rid of it
(lot of changes to do for probably only more bugs in the early times).

[2] and in the TM code, ouch :D