[Geany-Devel] Geany performance

[Matthew Brush]

On 13-09-29 12:23 AM, Павел Рощин wrote:
>
>> IIUC what you are seeing here is Scintilla laying out the window for the
>> first time, not just the goto.  That still will be done if the
>> editor_goto_pos is not called, just from a delayed idle callback IIUC.  If
>> its in the order of 3 seconds then I'm not sure delaying it is really going
>> to improve the user experience (even though it moves the action out of the
>> function you are measuring :)
>
> Profiling shows that loading time was reduced dramatically. And I feel that
> difference as user.
>
>> This now ignores the cl_options settings since it doesn't call
>> set_cursor_position() any more :(
>
> It doesn't _only_ if pos == 0. Go to zero position? Imagine I'm trying to load
> project with 1000 files. For each document set_cursor_position(0) will be
> called. Why?
>
>> Possibly, but you have to ensure that the actions actually happen at some
>> point.  Sure you don't have to layout the hidden windows immediately, but
>> does that mean that the first time you swap to a hidden file is slow?
>
> This is the thing called interface responsiveness. For user it's _much_ better
> to wait 0.02 sec 1000 times when clicking on document than wait 20 sec on
> loading. Most JIT compilers based on this trick: JIT compiler "delays" hot
> loops preferring execution over compiling.
>

Agreed. But here's the thing: every single file that is loaded needs to be:

1) Loaded from disk (slow)
   - Malloc a C string for whole document
2) Copied into scintilla buffer
   - Freed whole document string
3) Completely lexed by Scintilla lexer (blocking for huge docs)
4) Grabbing the raw char* of the buffer from Scintilla.
5) ** Does it take a whole other copy here? **
6) Passing through CTags and TM (blocking for huge docs).
7) Updating the user CTags into Scintilla keywords. (SCI_SETKEYWORDS 
with Ctags results)
8) Updating the Symbols tree view with new parsed tags.
9) Focusing the GTK+ notebook page
10) Setting caret pos to 0 which should be the natural position of the 
caret but who knows what magic Scintilla does when moving caret.

At present, most of these steps can't be delayed, they all kind of 
depend on the others due to Geany assuming a single thread on a single 
core and sharing a bunch of the loaded data globally between modules.

>> It is difficult, it has been discussed many times, but neither Geany, nor
>> Ctags, nor tagmanager, nor GTK are thread safe and it would take a major
>> re-design to use them from multiple threads.
>
> I see, but it could be a great improvement. Maybe even make co-routines but in
> multicore world it sounds surprisingly.
>

And more surprising in plain C99 :)

>
>> Basically the same problem as number 1), plus the difficulty of maintaining
>> which tabs are loaded and laid out.
>
> Same answer: responsiveness and again responsiveness. I'm human, if computer
> thinks 0.1 sec - I will even don't notice, if interface holds for 10 sec - it
> makes me nervous. Maintaining should simple: just add flag "delayed_load" and
> toggle it first time user clicked on document. Of course if there are no
> architecture problems.
>

This doesn't mention all of the background tasks and such that would 
need to be spawned and updating somehow into the main loop in an 
application that is down to its core a single threaded app. Not 
impossible but not as trivial as adding a "delayed_load" flag alone.

>> Sad as I am to say it, but maybe those with 1000s of tabs need a different
>> editor/ide, Geany's target is "small lightweight and simple" and it is
>> important that it maintains its focus on that.
>
> I tried Eclipse. Without plugins it's functionality is similar to Notepad++ but
> it eats 0.5 Gb of memory even without any project. To support all languages I
> need  for I should download thousands of plugins and now it eatch 1.5 Gb.
> QtCreator is a good but... C++ only. Recently Python was added.
>
> I need Geany for my work - it's huge project and I should handle over 10 million
> of files with different languages - from TeX to C++. Either I have to open 100-s

If you have a single project with 10M files, surely you are doing 
something wrong :)

> of IDE or I can use Geany. I know there are much to do, I'm sure I could help
> with developing but let's start with this simple improvement? I'll make PR if
> it's easier for you.
>

Indeed we would encourage any contributions here, but just remember it's 
existing software with certain hardcoded assumptions that have to be 
lived with. It's easy to bikeshed about how stuff should be, it's a 
whole different story to actually update the gnarly code to make it a 
reality and not break everyone's IDE in the process :)

Cheers,
Matthew Brush