Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default.
On 09/22/2011 09:30 AM, Nick Treleaven wrote:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
Well under 1 second to load geany.html here, and editing it is quite smooth as well.
Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable. I didn't spend much time troubleshooting, but I suspect turning down the symbol update frequency might help.
But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default.
IMO it would be best not to disabling automatic updating of the symbols, it's a really good feature.
Cheers, Matthew Brush
On 23 September 2011 10:34, Matthew Brush mbrush@codebrainz.ca wrote:
On 09/22/2011 09:30 AM, Nick Treleaven wrote:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
Well under 1 second to load geany.html here, and editing it is quite smooth as well.
Ditto, but of course it depends on the machine.
Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable.
IIUC the tags files are not re-loaded so the problem here suggests symbol lookup problems rather than parsing problems.
I didn't spend much time
troubleshooting, but I suspect turning down the symbol update frequency might help.
If it is a parsing problem sure.
But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default.
Of course if we turn it off by default a lot of people won't turn it on and won't know about the feature. IMHO its better to have it on by default and field a few ML/IRC questions about it being slow for big files.
Cheers Lex
IMO it would be best not to disabling automatic updating of the symbols, it's a really good feature.
Cheers, Matthew Brush _______________________________________________ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Le 23/09/2011 03:09, Lex Trotman a écrit :
On 23 September 2011 10:34, Matthew Brush mbrush@codebrainz.ca wrote:
On 09/22/2011 09:30 AM, Nick Treleaven wrote:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
Well under 1 second to load geany.html here, and editing it is quite smooth as well.
Ditto, but of course it depends on the machine.
Same here, I don't see any kind of lag with that file. But yeah, depends a lot on the machine's power.
Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable.
IIUC the tags files are not re-loaded so the problem here suggests symbol lookup problems rather than parsing problems.
Actually, IIRC Tagmanager updates an array holding all workspace tags every time an update is done, so having many tags file or many open files of a same filetype *may* (?) reduce the performances. No guarantees on that though, there are too many dark corners in Tagmanager (:-')
Regards, Colomban
On 24 September 2011 20:33, Colomban Wendling lists.ban@herbesfolles.org wrote:
Le 23/09/2011 03:09, Lex Trotman a écrit :
On 23 September 2011 10:34, Matthew Brush mbrush@codebrainz.ca wrote:
On 09/22/2011 09:30 AM, Nick Treleaven wrote:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
Well under 1 second to load geany.html here, and editing it is quite smooth as well.
Ditto, but of course it depends on the machine.
Same here, I don't see any kind of lag with that file. But yeah, depends a lot on the machine's power.
Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable.
IIUC the tags files are not re-loaded so the problem here suggests symbol lookup problems rather than parsing problems.
Actually, IIRC Tagmanager updates an array holding all workspace tags every time an update is done, so having many tags file or many open files of a same filetype *may* (?) reduce the performances. No guarantees on that though, there are too many dark corners in Tagmanager (:-')
Are you saying that it re-reads all open tags files every time it re-parses a source file? That would certainly make it slow.
And if it is just an array, unless the array is indexed or sorted then searching it would be slow for computing the auto-completions.
Cheers Lex
Regards, Colomban _______________________________________________ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Le 24/09/2011 12:39, Lex Trotman a écrit :
On 24 September 2011 20:33, Colomban Wendling lists.ban@herbesfolles.org wrote:
Le 23/09/2011 03:09, Lex Trotman a écrit :
On 23 September 2011 10:34, Matthew Brush mbrush@codebrainz.ca wrote:
On 09/22/2011 09:30 AM, Nick Treleaven wrote:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
Well under 1 second to load geany.html here, and editing it is quite smooth as well.
Ditto, but of course it depends on the machine.
Same here, I don't see any kind of lag with that file. But yeah, depends a lot on the machine's power.
Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable.
IIUC the tags files are not re-loaded so the problem here suggests symbol lookup problems rather than parsing problems.
Actually, IIRC Tagmanager updates an array holding all workspace tags every time an update is done, so having many tags file or many open files of a same filetype *may* (?) reduce the performances. No guarantees on that though, there are too many dark corners in Tagmanager (:-')
Are you saying that it re-reads all open tags files every time it re-parses a source file? That would certainly make it slow.
And if it is just an array, unless the array is indexed or sorted then searching it would be slow for computing the auto-completions.
1) tm_workspace keeps a sorted array of all non-global tags, that is rebuilt on each reparse [1]. This needs to rebuild and sort an array holding all tags (I suspect that this may be "slow" with many tags), but don't need reparsing anything.
2) As Matthew said in another mail, tm_tags_find() uses bsearch() [2] so it's not that slow upon search. Note however that scope completion is a complete other topic -- BTW it's still broken... :/
Cheers, Colomban
[1] tm_workspace.c:tm_workspace_update() [2] tm_tag.c:tm_tags_find()
Am 22.09.2011 18:30, schrieb Nick Treleaven:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default.
How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is.
Best regards.
Le 23/09/2011 19:48, Thomas Martitz a écrit :
Am 22.09.2011 18:30, schrieb Nick Treleaven:
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default.
How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is.
I like the idea.
Having a per-document setting (enabled/disabled) that would be automagically turned off if a configured performance impact is reached would probably be a correct compromise.
Something like:
(arbitrary values) max-reparse-duration: 500ms max-reparse-duration-exceeding: 3
if (reparse_duration >= max_reparse_duration) { /* give heavier weight to failures */ doc->n_reparse_duration_exceeding += 2; if (doc->n_reparse_duration_exceeding >= (max_reparse_duration_exceeding * 2)) { doc->real_time_reparse = FALSE; } } else { /* give lighter weight to success */ doc->n_reparse_duration_exceeding -= 1; }
max-reparse-duration-exceeding is useful not to suffer of false-positives like a reparsing happening when the machine is under heavy load or any other temporary problem. And it is always possible to set it to <= 1 to disable that heuristic.
Opinions, ideas?
Regards, Colomban
[...]
How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is.
I like the idea.
Hi Colomban, Thomas,
I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session.
Although I have a great idea for an animated paper clip in the bottom right corner that can suggest it every couple of seconds ... :)
Colomban is right in trying to evaluate where Nicks problem is first, thats far more likely to be productive, this sort of thing is the last option not the first.
Cheers Lex
Am 24.09.2011 13:32, schrieb Lex Trotman:
Hi Colomban, Thomas,
I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session.
I too think there should be a notification. But I think turning it of per-file (without touching the actual setting) is also good and the user will appreciate that.
Note that it would be turned off in the case symbol generation takes a long time, so it wouldn't go unnoticed if it's deactivated.
Colomban is right in trying to evaluate where Nicks problem is first, thats far more likely to be productive, this sort of thing is the last option not the first.
I agree.
Best regards.
[..]
I too think there should be a notification. But I think turning it of per-file (without touching the actual setting) is also good and the user will appreciate that.
Per file is a good idea.
Note that it would be turned off in the case symbol generation takes a long time, so it wouldn't go unnoticed if it's deactivated.
I'm always surprised what users can overlook, but if we notify them, it doesn't matter how unobservant they are :)
Colomban is right in trying to evaluate where Nicks problem is first, thats far more likely to be productive, this sort of thing is the last option not the first.
I agree.
Also Matthew's perceived slowness with lots of tags loaded needs checking, is it reloading them all the time (as Colomban seemed to suggest) or does the symbol table used by tagmanager have O(1), O(log), O(N) or worse performance.
Cheers Lex
On 09/24/2011 05:49 AM, Lex Trotman wrote:
Also Matthew's perceived slowness with lots of tags loaded needs checking, is it reloading them all the time (as Colomban seemed to suggest) or does the symbol table used by tagmanager have O(1), O(log), O(N) or worse performance.
I'm too afraid to go back into the TagManager code, but IIRC it uses a sorted GPtrArray. I think for each new tag added it qsort()s the array and also removes duplicates by looping over the whole array. Lookups are done with bsearch() on the array I think.
Not sure if that matters or is what you asked :)
Cheers, Matthew Brush
Le 24/09/2011 13:32, Lex Trotman a écrit :
[...]
How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is.
I like the idea.
Hi Colomban, Thomas,
Hey Lex, everyone,
I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session.
Hum, right, automagic might not be good, asking the user is always better.
I've got 2 WIP patches, though they might not be appropriate to commit before release since they add strings... anyway, here they are:
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
Both misses docs, up to come when they are ready.
So, I'm soliciting your impressions, opinions, remarks, etc. Also, is this whole thing important enough to break string freeze less than a week before release? (read: to have some strings untranslated)
Cheers, Colomban
PS:
Although I have a great idea for an animated paper clip in the bottom right corner that can suggest it every couple of seconds ... :)
Hehe, I like the idea... :D
Colomban is right in trying to evaluate where Nicks problem is first, thats far more likely to be productive, this sort of thing is the last option not the first.
BTW, after Nick's answer, it looks very weird to me, like if regexes was rrrrrrrreeeeeeeeeeaaaaaaaaaalllllllllyyyyyyy slow on Windows (or everywhere?)
Hey Colomban,
[...]
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
This looks like a good idea.
For the fixme, It shouldn't apply per document settings here, they are not in the prefs so they won't have changed. Just leave it out.
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
Havn't had time to look at this, but as I said its a last resort.
Both misses docs, up to come when they are ready.
So, I'm soliciting your impressions, opinions, remarks, etc. Also, is this whole thing important enough to break string freeze less than a week before release? (read: to have some strings untranslated)
I'm not Nick, but I would suggest leaving it until after release, thats not far away, and then as soon as it is committed Nick will have built a new version :)
Cheers Lex
Le 27/09/2011 02:03, Lex Trotman a écrit :
Hey Colomban,
[...]
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
This looks like a good idea.
For the fixme, It shouldn't apply per document settings here, they are not in the prefs so they won't have changed. Just leave it out.
There is no prefs, but if autocompletion_update_freq is changed to or from 0 (e.g. disabled or enabled), then we need to update the setting for all documents where the user haven't done a choice. Otherwise the setting would only change for newly opened documents, which is no good IMHO.
BTW, I just see I actually add a user_choice flag in 0002, so the info is here:
foreach_document(i) { if (! documents[i]->priv->tag_list_update.user_choice) documents[i]->priv->tag_list_update.user_choice = editor_prefs.autocompletion_update_freq > 0; }
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
Havn't had time to look at this, but as I said its a last resort.
Yeah, but if the user don't make a choice by accident just because she was typing something else, I think it's better :)
Both misses docs, up to come when they are ready.
So, I'm soliciting your impressions, opinions, remarks, etc. Also, is this whole thing important enough to break string freeze less than a week before release? (read: to have some strings untranslated)
I'm not Nick, but I would suggest leaving it until after release, thats not far away, and then as soon as it is committed Nick will have built a new version :)
Agreed, apart that if it is a real problem, Nick won't be the only one suffering of it. But Matthew tried on a Windows XP and didn't see any issue, so it at least doesn't affect everybody... /me still wonders what's happening.
Cheers, Colomban
[...]
There is no prefs, but if autocompletion_update_freq is changed to or from 0 (e.g. disabled or enabled), then we need to update the setting for all documents where the user haven't done a choice. Otherwise the setting would only change for newly opened documents, which is no good IMHO.
Oops forgot about the global setting. But it shouldn't override a per document setting chosen by the user. I think your below ensures it doesn't?
BTW, I just see I actually add a user_choice flag in 0002, so the info is here:
foreach_document(i) { if (! documents[i]->priv->tag_list_update.user_choice) documents[i]->priv->tag_list_update.user_choice = editor_prefs.autocompletion_update_freq > 0; }
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
Havn't had time to look at this, but as I said its a last resort.
[...]
I'm not Nick, but I would suggest leaving it until after release, thats not far away, and then as soon as it is committed Nick will have built a new version :)
Agreed, apart that if it is a real problem, Nick won't be the only one suffering of it. But Matthew tried on a Windows XP and didn't see any issue, so it at least doesn't affect everybody... /me still wonders what's happening.
I suspect we would have heard if it was a common problem, thats why I think its ok to wait till after release. (I'm trying to save you work :)
Cheers Lex
On 26/09/2011 22:38, Colomban Wendling wrote:
So, I'm soliciting your impressions, opinions, remarks, etc. Also, is this whole thing important enough to break string freeze less than a week before release? (read: to have some strings untranslated)
Well, it's not just untranslated strings - it's best not to introduce new features close to a release in case we introduce bugs.
On 26/09/2011 22:38, Colomban Wendling wrote:
Le 24/09/2011 13:32, Lex Trotman a écrit :
[...]
How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is.
I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session.
Hum, right, automagic might not be good, asking the user is always better.
I've got 2 WIP patches, though they might not be appropriate to commit before release since they add strings... anyway, here they are:
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
Not sure about the use case:
1. Very big files > x MB - I don't think we need to support that case specially, Geany is a programmer's editor. The user can just open as filetype none if they only want basic editing.
2. Slow tag parsers - a per-filetype setting might be more practical than per-document?
I see it might be useful but I'm not convinced it's worth it.
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
I think this is overkill - who would want editing to have > ~0.2s delays? Auto-disable should be fine with a status bar message.
Am 26.09.2011 23:38, schrieb Colomban Wendling:
Hum, right, automagic might not be good, asking the user is always better.
I've got 2 WIP patches, though they might not be appropriate to commit before release since they add strings... anyway, here they are:
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
I think that one isn't needed. I don't think anyone wants to set it off per document, except if one parses slowly in which case we can turn it of automatically (with notification). When I said per-document setting I meant internally, not one actually accessible in the menus.
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
This is more what I meant. I'm not so sure an extra dialog is needed, but if people want one that's ok. Would a "don't ask me again" checkbox be a good idea (if it doesn't already).
Best regards.
Nick, Thomas,
[...]
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
@Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad.
@Thomas, and since the problem is with a specific large file, Colomban is right it is per document, since you only need to turn off the one that is too large.
@Nick, therefore not per filetype either
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
This is more what I meant. I'm not so sure an extra dialog is needed, but if people want one that's ok. Would a "don't ask me again" checkbox be a good idea (if it doesn't already).
@Thomas, While I still think automatic changing of settings is a last resort, and as Nick said what performance number is acceptable? If we do add it, the user must be notified, but a "don't tell me again" is ok (for the session)
@Nick no one reads the status bar, sadly
Cheers Lex
On 28/09/2011 00:45, Lex Trotman wrote:
Nick, Thomas,
[...]
0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :)
@Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad.
You edited out my reply which addressed that:
On 27/09/2011 17:17, Nick Treleaven wrote:
- Very big files > x MB - I don't think we need to support that case
specially, Geany is a programmer's editor. The user can just open as filetype none if they only want basic editing.
In the open dialog, choose filetype None and you can easily open big files, plus it will be faster as syntax highlighting won't be enabled.
@Thomas, and since the problem is with a specific large file, Colomban is right it is per document, since you only need to turn off the one that is too large.
@Nick, therefore not per filetype either
For large files an auto-disable reparsing feature would work fine.
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK.
This is more what I meant. I'm not so sure an extra dialog is needed, but if people want one that's ok. Would a "don't ask me again" checkbox be a good idea (if it doesn't already).
@Thomas, While I still think automatic changing of settings is a last resort, and as Nick said what performance number is acceptable? If we do add it, the user must be notified, but a "don't tell me again" is ok (for the session)
Perhaps I was wrong with 0.2s, maybe 0.5s (I didn't check Colomban's code to see what he used).
@Nick no one reads the status bar, sadly
A dialog is too intrusive. Consider opening 10 slow files and having to click the dialog for each one. Status bar is fine - as you sometimes say programmers don't need to be treated like newbies. Not having tag reparsing enabled will mean a few invalid bug reports at worst.
On 28/09/2011 12:03, Nick Treleaven wrote:
@Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad
Sorry, I missed the bit where you said no highlighting is bad. But in that case, auto-disable works fine. In any case, there is no xxx Gb maximum, I just meant that we shouldn't have a document menu option that is only for disabling reparsing when it can be done automatically.
Also I'm not sure whether Scintilla handles very big files well. I don't think we need to be concerned about that.
On 28 September 2011 21:12, Nick Treleaven nick.treleaven@btinternet.com wrote:
On 28/09/2011 12:03, Nick Treleaven wrote:
@Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad
Sorry, I missed the bit where you said no highlighting is bad. But in that case, auto-disable works fine.
Actually I should make it clear that I am not talking about editing huge files, but displaying them and showing all the navigational assistance that we can provide. So I guess so long as the tags generated by the first run that took too long and triggered the disable are going to be still there, then disabling it is ok.
Actually that leads to the question, why is it reparsing an unchanged file after the first time anyway?
In any case, there is no xxx Gb maximum, I
just meant that we shouldn't have a document menu option that is only for disabling reparsing when it can be done automatically.
Also I'm not sure whether Scintilla handles very big files well. I don't think we need to be concerned about that.
Like molasses :) if you try to edit it that is.
Cheers Lex
Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
On 28/09/2011 12:56, Lex Trotman wrote:
Actually that leads to the question, why is it reparsing an unchanged file after the first time anyway?
It isn't, AFAICT. I was editing doc/geany.html (python not installed yet).
On 28 September 2011 22:01, Nick Treleaven nick.treleaven@btinternet.com wrote:
On 28/09/2011 12:56, Lex Trotman wrote:
Actually that leads to the question, why is it reparsing an unchanged file after the first time anyway?
It isn't, AFAICT. I was editing doc/geany.html (python not installed yet).
So you was docutils :) no wonder it was annoying when it was slow.
I know Colomban hasn't done the docs yet, we need to be careful to explain this well, there is no other automatic preference overrideing feature in Geany.
Needs to explain that tags will stay there but won't update until save (they will still update on save right?) so the user can figure out how to realign tags and file after, for example, they delete a whole chapter of geany.html and make most of their tags useless.
The statusbar message needs to be logged to the status window as well, status bar messages can be overwritten at any time.
Cheers Lex
Le 22/09/2011 18:30, Nick Treleaven a écrit :
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
I think 5s is weird in the first place. I know geany.html is 6.5k-lines file, and that it may have say, 150 tags, but 5s seems way too much IMHO.
Maybe we should try to find out why this is so slow on your machine, there might show a Geany bug, performance problem or something else.
* What kind of machine did you run (basically, CPU's MHz) * Had you many HTML files open, with many tags each? * Had you many non-HTML files open?
...and any information that might be useful (:-')
(BTW, I've seen HTML tag parser is regex-based, maybe that's particularly slow?)
But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default.
I like Thomas' idea, see other mail.
Regards, Colomban
On 24/09/2011 11:59, Colomban Wendling wrote:
Le 22/09/2011 18:30, Nick Treleaven a écrit :
Hi, Whilst opening doc/geany.html I found it takes>5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
I think 5s is weird in the first place. I know geany.html is 6.5k-lines file, and that it may have say, 150 tags, but 5s seems way too much IMHO.
Maybe we should try to find out why this is so slow on your machine, there might show a Geany bug, performance problem or something else.
I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds. It might help to update it from CTags (we had a patch from Jiri Techet but it didn't look like it would build on Windows, i.e. wasn't from CTags).
- What kind of machine did you run (basically, CPU's MHz)
Pentium 4, 1.9 GHz, 512 MB ram.
- Had you many HTML files open, with many tags each?
- Had you many non-HTML files open?
Just that one file.
Le 26/09/2011 13:48, Nick Treleaven a écrit :
On 24/09/2011 11:59, Colomban Wendling wrote:
Le 22/09/2011 18:30, Nick Treleaven a écrit :
Hi, Whilst opening doc/geany.html I found it takes>5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved.
I think 5s is weird in the first place. I know geany.html is 6.5k-lines file, and that it may have say, 150 tags, but 5s seems way too much IMHO.
Maybe we should try to find out why this is so slow on your machine, there might show a Geany bug, performance problem or something else.
I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe yeah, but I find it a little weird you see *so much* slowness...
For comparison, on my machine [1] parsing and updating the list takes less than 60ms for geany.html.
It might help to update it from CTags (we had a patch from Jiri Techet but it didn't look like it would build on Windows, i.e. wasn't from CTags).
Maybe we could use GRegex now we have a GLib recent enough?
Anyway tagamanger/regex.c looks like a few glibc source file concatenated with a 0xff separator, I'd then guess any POSIX RE implementation (e.g. today's glibc's one) would do, wouldn't it?
- What kind of machine did you run (basically, CPU's MHz)
Pentium 4, 1.9 GHz, 512 MB ram.
Not any kind of "slow" machine... Could anybody else try the same on another Windows machine, just to see if it is Windows-related or something? (Enrico, Matthew?)
- Had you many HTML files open, with many tags each?
- Had you many non-HTML files open?
Just that one file.
OK... with a fresh configdir?
Regards, Colomban
[1] OK, it is a little more powerful (i3 2.93GHz), but still
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe yeah, but I find it a little weird you see *so much* slowness...
For comparison, on my machine [1] parsing and updating the list takes less than 60ms for geany.html.
Just to be sure, are you seeing the 'built-in regex' debug message:
12:42:07: (null) INFO : GTK 2.22.0, GLib 2.26.0, built-in regex
It might help to update it from CTags (we had a patch from Jiri Techet but it didn't look like it would build on Windows, i.e. wasn't from CTags).
Maybe we could use GRegex now we have a GLib recent enough?
Anyway tagamanger/regex.c looks like a few glibc source file concatenated with a 0xff separator, I'd then guess any POSIX RE implementation (e.g. today's glibc's one) would do, wouldn't it?
That's probably what Jiri did, but I noticed CTags had some modifications to build on Windows. Perhaps glibc would work, but I think taking it from ctags is safer.
I don't know if GRegex would work, CTags parsers may use non-POSIX features - are \w \s part of POSIX?
OK... with a fresh configdir?
Just tried that, same problem. Task manager showed I still had about 230MB memory free whilst having the problem - the freeze caused 100% CPU as expected.
On Tue, Sep 27, 2011 at 14:17, Nick Treleaven nick.treleaven@btinternet.com wrote:
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe yeah, but I find it a little weird you see *so much* slowness...
For comparison, on my machine [1] parsing and updating the list takes less than 60ms for geany.html.
Just to be sure, are you seeing the 'built-in regex' debug message:
12:42:07: (null) INFO : GTK 2.22.0, GLib 2.26.0, built-in regex
It might help to update it from CTags (we had a patch from Jiri Techet but it didn't look like it would build on Windows, i.e. wasn't from CTags).
Maybe we could use GRegex now we have a GLib recent enough?
Anyway tagamanger/regex.c looks like a few glibc source file concatenated with a 0xff separator, I'd then guess any POSIX RE implementation (e.g. today's glibc's one) would do, wouldn't it?
That's probably what Jiri did, but I noticed CTags had some modifications to build on Windows. Perhaps glibc would work, but I think taking it from ctags is safer.
Basically I took the (then) latest versions of the regex files plus the makefile from ctags. The patch still applies by the way. I've just diffed the result of my patch with the ctags regex and there aren't big differences. What I've forgotten include into my patch is a conditional include under windows in regex_internal.h so if you want to use my patch, substitute this file with the ctags one. I haven't tested it under Windows myself so I don't know what happens.
Cheers,
Jiri
On 27/09/2011 13:17, Nick Treleaven wrote:
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe we could use GRegex now we have a GLib recent enough?
I don't know if GRegex would work, CTags parsers may use non-POSIX features - are \w \s part of POSIX?
I'm now working on porting lregex.c to GRegex, I think it might be compatible enough. I'll commit it to unstable if it goes OK.
On 30/09/2011 14:37, Nick Treleaven wrote:
On 27/09/2011 13:17, Nick Treleaven wrote:
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe we could use GRegex now we have a GLib recent enough?
I don't know if GRegex would work, CTags parsers may use non-POSIX features - are \w \s part of POSIX?
Actually I was wrong, our parsers don't use \w or \s.
I'm now working on porting lregex.c to GRegex, I think it might be compatible enough. I'll commit it to unstable if it goes OK.
Now done. This fixes my performance issue with doc/geany.html. Needs testing though. Regex parsers are:
actionscript cobol html php R
On 30/09/2011 17:32, Nick Treleaven wrote:
On 30/09/2011 14:37, Nick Treleaven wrote:
On 27/09/2011 13:17, Nick Treleaven wrote:
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe we could use GRegex now we have a GLib recent enough?
I'm now working on porting lregex.c to GRegex, I think it might be compatible enough. I'll commit it to unstable if it goes OK.
Now done. This fixes my performance issue with doc/geany.html. Needs testing though. Regex parsers are:
actionscript cobol html php R
Link: http://geany.svn.sourceforge.net/viewvc/geany?view=revision&revision=597...
I think Colomban's idea of using GRegex is great in that all systems will then have the same regex behaviour. The old situation was a support headache.
GRegex is more reliable besides being more powerful, based on PCRE. The Windows version (at least) of GNU regex we had didn't understand Mac line endings for ^, $.
On Sun, Oct 2, 2011 at 16:34, Nick Treleaven nick.treleaven@btinternet.com wrote:
On 30/09/2011 17:32, Nick Treleaven wrote:
On 30/09/2011 14:37, Nick Treleaven wrote:
On 27/09/2011 13:17, Nick Treleaven wrote:
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe we could use GRegex now we have a GLib recent enough?
I'm now working on porting lregex.c to GRegex, I think it might be compatible enough. I'll commit it to unstable if it goes OK.
Now done. This fixes my performance issue with doc/geany.html. Needs testing though. Regex parsers are:
actionscript cobol html php R
Link: http://geany.svn.sourceforge.net/viewvc/geany?view=revision&revision=597...
I think Colomban's idea of using GRegex is great in that all systems will then have the same regex behaviour. The old situation was a support headache.
GRegex is more reliable besides being more powerful, based on PCRE. The Windows version (at least) of GNU regex we had didn't understand Mac line endings for ^, $.
Cool! By the way, are the #ifdef HAVE_REGEX needed? From glib sources it seems GRegex either uses system pcre or built-in pcre so you should have always regex support.
Cheers,
Jiri
On 02/10/2011 20:16, Jiří Techet wrote:
On Sun, Oct 2, 2011 at 16:34, Nick Treleaven
Now done. This fixes my performance issue with doc/geany.html. Needs testing though. Regex parsers are:
actionscript cobol html php R
Link: http://geany.svn.sourceforge.net/viewvc/geany?view=revision&revision=597...
I think Colomban's idea of using GRegex is great in that all systems will then have the same regex behaviour. The old situation was a support headache.
GRegex is more reliable besides being more powerful, based on PCRE. The Windows version (at least) of GNU regex we had didn't understand Mac line endings for ^, $.
Cool! By the way, are the #ifdef HAVE_REGEX needed? From glib sources it seems GRegex either uses system pcre or built-in pcre so you should have always regex support.
I left HAVE_REGEX in lregex.c so less code is different from CTags. I think it doesn't cause any harm.
BTW:
Le 02/10/2011 16:34, Nick Treleaven a écrit :
On 30/09/2011 17:32, Nick Treleaven wrote:
On 30/09/2011 14:37, Nick Treleaven wrote:
On 27/09/2011 13:17, Nick Treleaven wrote:
On 26/09/2011 23:00, Colomban Wendling wrote:
Le 26/09/2011 13:48, Nick Treleaven a écrit :
I guess the reason is the regex code that is really old which is used for Windows builds.
Maybe we could use GRegex now we have a GLib recent enough?
I'm now working on porting lregex.c to GRegex, I think it might be compatible enough. I'll commit it to unstable if it goes OK.
Now done. This fixes my performance issue with doc/geany.html.
Great!
Needs testing though. Regex parsers are:
actionscript cobol html php R
Link: http://geany.svn.sourceforge.net/viewvc/geany?view=revision&revision=597...
I think Colomban's idea of using GRegex is great in that all systems will then have the same regex behaviour. The old situation was a support headache.
+1 :)
GRegex is more reliable besides being more powerful, based on PCRE. The Windows version (at least) of GNU regex we had didn't understand Mac line endings for ^, $.
Hehe, even better :) Now we should probably port all our regexes usage to GRegex -- and drop the included one when done, but there's no hurry.
Thanks for having done the hardest part: the try & check :) Colomban
On 17/10/2011 18:18, Colomban Wendling wrote:
GRegex is more reliable besides being more powerful, based on PCRE. The
Windows version (at least) of GNU regex we had didn't understand Mac line endings for ^, $.
Hehe, even better:) Now we should probably port all our regexes usage to GRegex -- and drop the included one when done, but there's no hurry.
Yes, I added a TODO to this effect ;-) I'm planning on porting the find & replace regex code sometime, unless someone beats me to it. The increased power for users should be nice.
On 11-10-17 10:40 AM, Nick Treleaven wrote:
On 17/10/2011 18:18, Colomban Wendling wrote:
GRegex is more reliable besides being more powerful, based on PCRE. The
Windows version (at least) of GNU regex we had didn't understand Mac line endings for ^, $.
Hehe, even better:) Now we should probably port all our regexes usage to GRegex -- and drop the included one when done, but there's no hurry.
Yes, I added a TODO to this effect ;-) I'm planning on porting the find & replace regex code sometime, unless someone beats me to it. The increased power for users should be nice.
IIRC there's at least one bug open on this subject that could be closed afterwards, yay!
Cheers, Matthew Brush
Le 27/09/2011 01:44, Lex Trotman a écrit :
[...]
Pentium 4, 1.9 GHz, 512 MB ram.
I'm no windows expert, but isn't 512mb considered kinda marginal, 1gb better. Whats your general windows performance like?
I doubt that, even on Windows, it would so drastically reduce the performances. Was your RAM completely full so it needed to swap?
Cheers, Colomban
On 27/09/2011 01:53, Colomban Wendling wrote:
Le 27/09/2011 01:44, Lex Trotman a écrit :
[...]
Pentium 4, 1.9 GHz, 512 MB ram.
I'm no windows expert, but isn't 512mb considered kinda marginal, 1gb better. Whats your general windows performance like?
I doubt that, even on Windows, it would so drastically reduce the performances. Was your RAM completely full so it needed to swap?
Windows performance is fine, I run Firefox with tons of tabs easily.
I have Windows XP BTW, so 512MB is not unusually low.