[Geany-devel] Geany memory behavior "suspected memory leak - ID: 3415254"

[Colomban Wendling]

Le 14/10/2011 05:09, Lex Trotman a écrit :
> Hi All,

Hey,

I ran Valgrind quite a lot on Geany lately -- though I never ran it in
production because it is slooooow --, and thus will comment with what I
know and have seen.

> I have been investigating bug report "suspected memory leak - ID:
> 3415254" and have gathered the following data (before my patience ran
> out).  The tests reported here are with Git 55646df8 unless otherwise
> noted.
> 
> Geany left all day without any activity (with files open) did not
> increase in memory usage (and the same with 0.20, the reported
> version).
> 
> I then tried opening and closing geany/src/* repeatedly.  The
> following table shows Geany memory usage (in Mb) when all files are
> closed and when all open and the difference.  The last column shows
> that the first two times extra memory was used to tab through all the
> open files, but after that no more was used.
> 
> closed open delta tabbing
>  5.1   27.9  22.8   1.1
> 28.8   31.2   0.4   1.1
> 32.1   33.0   0.9   0.0
> 32.8   34.8   2.0   0.0
> 34.6   35.9   1.3   0.0
> 35.7   36.4   0.7   0.0
> 36.2   37.4   1.2   0.0
> 37.2   38.1   0.9   0.0
> 37.9   41.8   2.9   0.0
> 41.6   42.3   0.7   0.0
> 42.0   42.8   0.8   0.0
> 42.6   43.3   0.7   0.0
> 43.1   44.3   1.2   0.0
> 44.1   44.7   0.4   0.0
> 44.5   45.7   1.2   0.0
> 
> To me the key points are:
> 
> 1. each time all files are closed only 200k is returned to the OS, but
> since the first open uses 22.8Mb but later ones less, Geany isn't
> leaking all of it, it is being held by the allocators.

Geany at least keeps all the GeanyDocuments alive and tries to re-use
them (document_create() at line 564).  This avoids having to re-allocate
the document struct everytime, though it'll then never release this
memory.  However, either the user won't open so much files at once or
she's likely to keep having a large amount of open files, so I don't
think it's a (the?) problem -- and anyway I guess the allocation would
be quite small.
However, we'd probably better use GSlice here (for fragmentation and
maybe caching), and maybe even release the memory to GSlice so it does
all the management, and could even release it to the system at some
point if it makes sense to it.

Also note that we never release the global tags, so it's likely the
first run loads those and be part of the 22Mb.  However, on my
installation the C global tags only use something like 400k in memory.

> 2. ignoring the first open, the amount of extra memory used to open
> all the files varied from 0.4Mb to 2.9Mb.  Average 1.1Mb or 5% of the
> initial 22.8Mb and standard deviation of 0.665Mb or 3% of the 22.8Mb.
> 
> 3. Simple leaks are unlikely to cause the large deviation in the
> memory increase, each open of the same set of files would tend to leak
> the same amount.  It is therefore likely that fragmentation effects
> cause the large deviation, but it may not be the whole cause of the
> increase.

We are not the only possible source of leaks.  At least Valgrind pointed
GTK leaks quite a lot in the file chooser dialog.

Also, it seems FontConfig, Pango or Scintilla (either can misuse the one
down) have a few leaks -- I think it's not (much of) Scintilla's fault,
but I don't know it very well to debug it.  Some looks like one-time
leaks, so probably unfreed initializations, some other seems to happen
more often.

Since I only speak of "definitely lost" leaks (as Valgrind calls them),
this should not include too much false-positives, though it still can --
we had a few "definitely lost" leaks on Geany that I have/could fix but
that would not reduce the real memory usage since they would only be
freed at closing time anyways.

> Since Geany uses three different allocators, each of which has
> differing policies for holding onto memory, it is going to be
> difficult to separate real leaks from allocator effects.

You could easily reduce this to 2 by using the G_SLICE=always-malloc
environment variable when running the tests.  This makes the GSlice API
simply wrap plain malloc/free for easier debugging ;)

> For the bug reporter to have accumulated 300Mb of memory over "a few"
> days would have needed about 500 file opens per day, but maybe
> somewhat less as editing increases the memory usage.

That's the problem of profiling and testing, we generally do this on
unreal situations... either by lack of time or because it has other
constraints, but heh, still not real.

> So I don't think we have to worry excessively that we have a major
> leak, but keep an eye open for any possible problems.

Agreed, but keeping a eye on memory usage is always a good thing :)

> Note that all my testing was with no plugins and only a couple of
> filetypes.  There may be leaks and other effects in plugins and/or
> other Scintilla lexers.  If the OP posts more information we may know
> more.

I also mostly tested C and python, not much others.


Just a few thoughts/points.

Cheers,
Colomban