I have a large Java file (4.8MB). If I insert a new <b><big>{</big></b> near the start of this file, the run time for the parser becomes very long (5 minutes or so) - while the parser is running the editor is unusable. Is there some way to prevent this from happening? By parser I mean the code that determines the coloration of the keywords, strings, etc. <b>Thanks!</b>
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791
Hum, that's odd. 4.8M doesn't look very large, and certainly nothing that would require 5 minutes to process. Could you provide the file so we can check what's going on?
Also, which version of Geany are you using, and on which OS?
Anyway, to work around this you can try a few things: * Disable real-time symbol parsing (set *Edit → Preferences → Editor → Completions → Symbol list update frequency* to 0). This would prevent updating the Symbols pane while typing, so if it's the bottleneck it should help a lot. Note however that the symbols will still be extracted when saving the file. * If the above helped but it's still too slow when saving the file, you can disable symbol parsing for the Java filetype by setting the `tag_parser` `settings` option to an empty value in *filetypes.java*:
```config [settings] tag_parser= ``` This is kind of a hack but it should disable all symbol parsing for Java files. * The nuclear option, disabling all filetype-specific features (highlighting, symbol parsing, etc.), is to use the filetype None for Java files.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162575825
If it's 5MB source code file, I'm guessing it's auto-generated (or else Java is _way_more verbose than I imagined). If so, it could be all on one line which would make it really slow. If it's not all on one line but has many long lines and long-line wrapping is on, it could also take a long time.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162704446
It is mainly hand written and so most of the lines are 40-120 chars long. Line wrapping is off. However there are some generated lines included: the longest of which is 4205 chars long. Java Lint takes 6 seconds. The parser black-out when it occurs typically takes about 5 minutes to recover. If I understand you correctly, you are suggesting breaking up the generated lines or better removing them to a separate file? Geany built on or after 2015-07-13.
I appreciate your time and thoughts on this matter.
Thanks,
Phil
Philip R Brenan
On Mon, Dec 7, 2015 at 11:29 PM, Matthew Brush notifications@github.com wrote:
If it's 5MB source code file, I'm guessing it's auto-generated (or else Java is _way_more verbose than I imagined). If so, it could be all on one line which would make it really slow. If it's not all on one line but has many long lines and long-line wrapping is on, it could also take a long time.
— Reply to this email directly or view it on GitHub https://github.com/geany/geany/issues/791#issuecomment-162704446.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162710941
If I understand you correctly, you are suggesting breaking up the generated lines or better removing them to a separate file?
Well Scintilla is known not to handle really long lines well (ex. minified JS), but also "word wrapping" causes a major performance hit. I would try to disable the latter, if you enabled it, before messing with the code.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162712496
@philiprbrenan did you try the suggestions by @b4n to allow identifying if its the symbol parser or highlighting lexer?
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162716459
Following your suggestions:
*Disable real-time symbol parsing* prevents the problem from occurring.
* Disable symbol parsing at save:* I saved with an extraneous { in place and the save was very slow. However this is less of a problem because I can control when a save occurs and just not save when the extra the bracket is outstanding.
*Disabling all file type-specific features* works as suggested - this was how I was getting around the problem earlier but it takes time to set and unset and one forgets to do it in advance and so this is an error prone process.
I appreciate your help!
Thanks,
Phil
Philip R Brenan
On Mon, Dec 7, 2015 at 4:23 PM, Colomban Wendling notifications@github.com wrote:
Hum, that's odd. 4.8M doesn't look very large, and certainly nothing that would require 5 minutes to process. Could you provide the file so we can check what's going on?
Also, which version of Geany are you using, and on which OS?
Anyway, to work around this you can try a few things:
- Disable real-time symbol parsing (set *Edit → Preferences → Editor →
Completions → Symbol list update frequency* to 0). This would prevent updating the Symbols pane while typing, so if it's the bottleneck it should help a lot. Note however that the symbols will still be extracted when saving the file.
If the above helped but it's still too slow when saving the file, you can disable symbol parsing for the Java filetype by setting the tag_parser settings option to an empty value in *filetypes.java*:
[settings] tag_parser=
This is kind of a hack but it should disable all symbol parsing for Java files.
- The nuclear option, disabling all filetype-specific features
(highlighting, symbol parsing, etc.), is to use the filetype None for Java files.
— Reply to this email directly or view it on GitHub https://github.com/geany/geany/issues/791#issuecomment-162575825.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162717799
[Test.java.zip](https://github.com/geany/geany/files/54802/Test.java.zip)
When I save the attached Java file of 2K lines it takes about 30 seconds to save during which time Geany is unresponsive. Lint of course finds lots of errors very quickly and bails out after less than a second. The brackets match correctly, but class test1 is defined multiple times.
If I shorten the file the parse blackout problem disappears at around 600 lines - although the is still a noticeable pause before the file is reported as saved in the Messages tab.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162725385
Looks like the particular file is encountering pathological worst case performance of the ctags parser or symbol handling software.
Improvements are welcome.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162729251
It's not that slow here, maybe 0.5-0.75 seconds to save, but my computer is really fast. It seems to perform better if replacing `{` with `\n{\n` so that the lines aren't so long (or so that fold points are more spread out, not sure).
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-162759579
Maybe one more suggestion - do you experience the same slowness if you switch from the "Symbols" tab in the sidebar to something else? IMO the parser should be fast in this case but I suspect the symbols tree generation is the slow one here.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163949100
I have tried clicking in the Symbols window and then clicking in the Java source and back again: this does not seem to cause any problems, nor does expanding or collapsing the symbols tree. The effect appear to be non linear - when I save 1K lines the delay is perhaps 2 seconds, but when I try to save 2K lines it is 30 seconds. Checking the system monitor shows 100% CPU, memory appears to be fine.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163960095
Sorry, maybe I said it in a confusing way - in the sidebar select e.g. Documents instead of Symbols. The thing is that when the Symbols tree isn't shown, it isn't rebuilt when you type. I believe this might fix the problem you mentioned in your first post about the freeze when typing { at the beginning of the file. But I doubt it will have any effect on saving.
Could you provide some bigger file for testing? The file you provided is just 100 lines and is insufficient to trigger the issue for me.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163961757
OK nevermind, I can reproduce it when copy-pasting the lines in your file several times. I'll have a look at the profiler output if I can see something.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163962387
Yes - that is what I have been doing to see where the effect kicks in.
Whether the Symbols/Documents tabs is open or not does not seem to have any effect - the blackout effect is easily triggered by doing a save regardless.
The fact that I am reusing the same names over and over again might be part of the problem. If helpful I could construct a more meaningful example using unique names?
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163964070
Alright, just tried with the profiler and the big part of the problem should be solved by the patch here:
https://github.com/techee/geany/commit/a11f82b7bca2697b41cbf01cf3151aad608f4...
It doesn't fix the slowness completely but at least it should fix the non-linear part of it. With about 4000 lines from your example about 75% of time was spent by rehighlighting the document. The remaining 25% were spent by the parser (I'm afraid we cannot do much with this).
If you are able to recompile, could you try the patch if it helps?
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163970875
This is absolutely marvellous - I will try to do this - as this is my first attempt at patching and compiling it will take a day or two as I will have to discover the exact procedure.
One of the (many ) reasons I use Geany for Java in preference to using the standard Java tools is that Geany scales much better on large files: it responds quickly and predictably with far fewer distracting glitchs and unexpected slow downs. I greatly appreciate your efforts to eliminate this problem - it makes a big difference.
Thanks,
Phil
Philip R Brenan
On Fri, Dec 11, 2015 at 3:51 PM, Jiří Techet notifications@github.com wrote:
Alright, just tried with the profiler and the big part of the problem should be solved by the patch here:
techee@a11f82b https://github.com/techee/geany/commit/a11f82b7bca2697b41cbf01cf3151aad608f4e63
It doesn't fix the slowness completely but at least it should fix the non-linear part of it. With about 4000 lines from your example about 75% of time was spent by rehighlighting the document. The remaining 25% were spent by the parser (I'm afraid we cannot do much with this).
If you are able to recompile, could you try the patch if it helps?
— Reply to this email directly or view it on GitHub https://github.com/geany/geany/issues/791#issuecomment-163970875.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163974909
By the way, I've been playing with the file a bit more and I can see quite some time spent in the symbol tree building too (switch to the Documents tab, edit the file so some symbols get added/removed and switch back to the Symbols tree - it takes quite some time to rebuild it).
In the past I was suggesting we should limit the number of entries in the tree to some sane number, say 10000 entries (your file contains 234 entries per line, with 1000 lines it becomes 234000 entries), because the current implementation doesn't scale very well:
https://github.com/geany/geany/pull/475#issuecomment-98514145
I think we should introduce some limit.
Yes, scaleability is one of my favorite Geany features too (not only for big files but also for big projects with thousands of files). So I'm definitely interested in improving any code that doesn't scale well.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163983610
@philiprbrenan By the way, the patch removes a single line from the code so instead of pulling the patch you can just get Geany from master and comment-out the single line which might be easier for you.
If you are on Debian, just run
``` apt-get build-dep geany ```
which should install all the dependencies and then run
``` ./autogen.sh --disable-html-docs make sudo make install ```
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-163984982
The symbol tree slowness may also be caused by this:
https://github.com/geany/geany/issues/577#issuecomment-122903681
The repeated symbol names make things worse for the tree generation - this might be fixable though.
Note to self: learn the difference between multiplication and addition: in the numbers above there are just 35 tags per line.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-164024924
A huge improvement - down from 10's of seconds to too fast to notice. Thank you very much!
On Fri, Dec 11, 2015 at 4:36 PM, Jiří Techet notifications@github.com wrote:
@philiprbrenan https://github.com/philiprbrenan By the way, the patch removes a single line from the code so instead of pulling the patch you can just get Geany from master and comment-out the single line which might be easier for you.
If you are on Debian, just run
apt-get build-dep geany
which should install all the dependencies and then run
./autogen.sh --disable-html-docs make sudo make install
— Reply to this email directly or view it on GitHub https://github.com/geany/geany/issues/791#issuecomment-163984982.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-164035388
A bit off-topic, but ... @techee are you using the XCode profiler, GNU grof, or other? I tried to do a profile build to test this previously, but I was unable to get gprof to produce any output in the report, I suspect because of splitting libgeany out of the main app. All I did was to put `./configure [some options] CFLAGS="-pg" CXXFLAGS="-pg" && make install` (I also tried `-pg` in the `LDFLAGS`) and then run Geany and produce a report from gmon.out. Is there anything else that needs to be done to make it work (assuming you're using gprof)?
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-164088102
@codebrainz I guess you missed the announcement of the wiki page about profiling Geany:
http://article.gmane.org/gmane.editors.geany.devel/9439
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-164158289
@philiprbrenan Great to hear! Even though the original patch wasn't quite right as Colomban noticed, the updated version should fix the performance issue too.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-164158340
I guess you missed the announcement of the wiki page about profiling Geany
Yep, I missed that. Will give it a read, thanks.
--- Reply to this email directly or view it on GitHub: https://github.com/geany/geany/issues/791#issuecomment-164160737
Closed #791.
Closing as the issue seems to be resolved according to the comments. Feel free to reopen if it is still an issue.
github-comments@lists.geany.org