On Tue, 4 Jan 2011 19:43:23 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
Dnia wtorek, 4 stycznia 2011 o 18:22:42 Dimitar Zhekov napisał(a):
As a crude estimate, under Linux you can try to malloc(minimal presumable required memory). It won't be actually allocated, unless you memset() it or something, but if the result is NULL, a warning is justified.
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
malloc fails at 01 << 040 at my place.
And so? On the machine I'm currently at: kernel 2.6.32, glibc-2.1, 512MB RAM + 512MB virtual, single malloc fails at ~640MB.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
Dnia wtorek, 4 stycznia 2011 o 20:19:55 Dimitar Zhekov napisał(a):
On Tue, 4 Jan 2011 19:43:23 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
Dnia wtorek, 4 stycznia 2011 o 18:22:42 Dimitar Zhekov napisał(a):
As a crude estimate, under Linux you can try to malloc(minimal presumable required memory). It won't be actually allocated, unless you memset() it or something, but if the result is NULL, a warning is justified.
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory. GNU malloc uses mmap so it is all virtual.
malloc fails at 01 << 040 at my place.
And so? On the machine I'm currently at: kernel 2.6.32, glibc-2.1, 512MB RAM + 512MB virtual, single malloc fails at ~640MB.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
I have 1 GB of physical memory. (01 << 040) is the first integer that does not fit into unsigned int.
Chris
2011/1/5 Krzysztof Żelechowski giecrilj@stegny.2a.pl:
Dnia wtorek, 4 stycznia 2011 o 20:19:55 Dimitar Zhekov napisał(a):
On Tue, 4 Jan 2011 19:43:23 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
Dnia wtorek, 4 stycznia 2011 o 18:22:42 Dimitar Zhekov napisał(a):
As a crude estimate, under Linux you can try to malloc(minimal presumable required memory). It won't be actually allocated, unless you memset() it or something, but if the result is NULL, a warning is justified.
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory. GNU malloc uses mmap so it is all virtual.
And on modern kernels virtual memory can be overcommitted significantly, so it may not work & is expensive since the page tables have to be constructed even if the memory is not used.
Also this could cause fragmentation of the virtual address space that may limit how many times it will work.
If you really want to do something then a warning "this is a big file are you sure you want to open it" using a hidden pref is the most sensible, with a pref value of zero meaning no check so people like me with 4G can turn it off.
Cheers Lex
malloc fails at 01 << 040 at my place.
And so? On the machine I'm currently at: kernel 2.6.32, glibc-2.1, 512MB RAM + 512MB virtual, single malloc fails at ~640MB.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
I have 1 GB of physical memory. (01 << 040) is the first integer that does not fit into unsigned int.
Chris _______________________________________________ Geany mailing list Geany@uvena.de http://lists.uvena.de/cgi-bin/mailman/listinfo/geany
On Wed, 5 Jan 2011 11:08:18 +1100 Lex Trotman elextr@gmail.com wrote:
2011/1/5 Krzysztof Żelechowski giecrilj@stegny.2a.pl:
Dnia wtorek, 4 stycznia 2011 o 20:19:55 Dimitar Zhekov napisał(a):
On Tue, 4 Jan 2011 19:43:23 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
Dnia wtorek, 4 stycznia 2011 o 18:22:42 Dimitar Zhekov napisał (a):
As a crude estimate, under Linux you can try to malloc(minimal presumable required memory). It won't be actually allocated, unless you memset() it or something, but if the result is NULL, a warning is justified.
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory. GNU malloc uses mmap so it is all virtual.
And on modern kernels virtual memory can be overcommitted significantly, so it may not work & is expensive since the page tables have to be constructed even if the memory is not used.
Also this could cause fragmentation of the virtual address space that may limit how many times it will work.
If you really want to do something then a warning "this is a big file are you sure you want to open it" using a hidden pref is the most sensible, with a pref value of zero meaning no check so people like me with 4G can turn it off.
I think this shouldn't be done as a hidden pref (in case of we really want to add such a thing which I doubt is very useful at all) as I'm sure a lot of people having issues with to big files and not reading this list will just not read manual etc. and therefor not be able to find the switch to either turn it on or off. Also the maximum is hardly depending on plattform and filetype ... at least from my experience open up huge XML files on Windows. I suggest to add a bool "Warn at huge files" im combination with an input element configuring at which point we are talking about a big file.
Cheers + just my 2ct. Frank
On Wed, 5 Jan 2011 18:30:06 +0100 Frank Lanitz frank@frank.uvena.de wrote:
If you really want to do something then a warning "this is a big file are you sure you want to open it" using a hidden pref is the most sensible, with a pref value of zero meaning no check so people like me with 4G can turn it off.
I think this shouldn't be done as a hidden pref (in case of we really want to add such a thing which I doubt is very useful at all) as I'm
I think it is useful if it can prevent someone's Geany freezing and possibly making the whole system unresponsive.
sure a lot of people having issues with to big files and not reading this list will just not read manual etc. and therefor not be able to find the switch to either turn it on or off.
I think confirming loading of a big file is not a big problem for the user. The dialog could mention that the limit is configurable.
Also the maximum is hardly depending on plattform and filetype ... at least from my experience open up huge XML files on Windows. I suggest to add a bool "Warn at huge files" im combination with an input element configuring at which point we are talking about a big file.
As Lex suggested, a hidden pref could accept a value of zero to disable the confirmation completely.
The question is what should the default be, and personally I think some kind of limit is better than no limit.
Regards, Nick
Le 05/01/2011 18:57, Nick Treleaven a écrit :
On Wed, 5 Jan 2011 18:30:06 +0100 Frank Lanitz frank@frank.uvena.de wrote:
If you really want to do something then a warning "this is a big file are you sure you want to open it" using a hidden pref is the most sensible, with a pref value of zero meaning no check so people like me with 4G can turn it off.
I think this shouldn't be done as a hidden pref (in case of we really want to add such a thing which I doubt is very useful at all) as I'm
I think it is useful if it can prevent someone's Geany freezing and possibly making the whole system unresponsive.
I agree. Once, I opened by accident a ~1GB file with Geany, and regretted my mistake for minutes -- because I didn't want to kill Geany, I had unsaved changes. I wish I had such a confirmation dialogue ^^
Moreover, are such over-sized files so common in a text editor? I'd bet no.
sure a lot of people having issues with to big files and not reading this list will just not read manual etc. and therefor not be able to find the switch to either turn it on or off.
I think confirming loading of a big file is not a big problem for the user. The dialog could mention that the limit is configurable.
I could even have a "don't ask me anymore"-like checkbox like in some apps (I think of Firefox but there are others) that would: 1) show there is a preference 2) allow annoyed user to easily get rid of the warning
Also the maximum is hardly depending on plattform and filetype ... at least from my experience open up huge XML files on Windows. I suggest to add a bool "Warn at huge files" im combination with an input element configuring at which point we are talking about a big file.
As Lex suggested, a hidden pref could accept a value of zero to disable the confirmation completely.
The question is what should the default be, and personally I think some kind of limit is better than no limit.
I personally almost never edit files > 5Mo, so I'd suggest about 30/50Mo limit by default. But allowing to modify the value is probably a good thing if the default size doesn't match user needs (e.g. someone that often edit 100Mo files but won't make Geany hang with 1Gb files).
My 2 cents.
Regards, Colomban
On Wed, 5 Jan 2011 17:57:15 +0000 Nick Treleaven nick.treleaven@btinternet.com wrote:
The question is what should the default be, and personally I think some kind of limit is better than no limit.
50MB file size IMHO. Most users will never need it, and most machines will be able to handle it, even for XML. Personally I work with larger files, but prefer another editor for them.
As of the setting being hidden or configurable, won't we include the Various ("hidden") preferences editor after 0.20? If so, the question becomes simpler: Does the threshold belong to a specific tab, for example "File", or just leave it in "Various"?
On 6 January 2011 06:25, Dimitar Zhekov dimitar.zhekov@gmail.com wrote:
On Wed, 5 Jan 2011 17:57:15 +0000 Nick Treleaven nick.treleaven@btinternet.com wrote:
The question is what should the default be, and personally I think some kind of limit is better than no limit.
50MB file size IMHO. Most users will never need it, and most machines will be able to handle it, even for XML. Personally I work with larger files, but prefer another editor for them.
No number will suit everyone :-) , but as we are only talking about a default value then to avoid the argument I'd second a value in this range.
As of the setting being hidden or configurable, won't we include the Various ("hidden") preferences editor after 0.20? If so, the question becomes simpler: Does the threshold belong to a specific tab, for example "File", or just leave it in "Various"?
Good point, I'd leave it in the various/hidden/advanced/i promise I'll be careful/... tab.
Cheers Lex
-- E-gards: Jimmy _______________________________________________ Geany mailing list Geany@uvena.de http://lists.uvena.de/cgi-bin/mailman/listinfo/geany
On Wed, 5 Jan 2011 21:25:38 +0200 Dimitar Zhekov dimitar.zhekov@gmail.com wrote:
The question is what should the default be, and personally I think some kind of limit is better than no limit.
50MB file size IMHO. Most users will never need it, and most machines will be able to handle it, even for XML. Personally I work with larger files, but prefer another editor for them.
I think 50MB is reasonable too.
As of the setting being hidden or configurable, won't we include the Various ("hidden") preferences editor after 0.20?
I'm planning to, but I want to make some changes to the code in a branch first.
For one thing, I'm not sure it's safe to allow some hidden prefs to change after startup, as the code that reads them might assume the value stays constant.
If so, the question becomes simpler: Does the threshold belong to a specific tab, for example "File", or just leave it in "Various"?
I think it probably isn't needed enough to go in the File tab.
Regards, Nick
On Thu, 6 Jan 2011 17:08:44 +0000 Nick Treleaven nick.treleaven@btinternet.com wrote:
As of the setting being hidden or configurable, won't we include the Various ("hidden") preferences editor after 0.20?
I'm planning to, but I want to make some changes to the code in a branch first.
Some fields were added to the dialog, so the -0.5.diff will not compile any more because of conflicting identifiers (label248, vbox54 etc.) The latest .diff I have is for r5484.
But now that 0.20 is out, I'll renumber the fields, this time using gaps in geany.glade numbering if possible for future compatibility.
For one thing, I'm not sure it's safe to allow some hidden prefs to change after startup, as the code that reads them might assume the value stays constant.
I can do such a check, and mail at least the prefs which are obviously safe, and why. Of course, they must be verifyed by at least one more developer.
Hi,
Attached is a various prefs editor which applies against 0.20. Now a few words about the individual prefs:
[current] - the current pref value is always used, and there are no dependencies on the previous value AFAICT.
[restart] - requires restart to take effect.
allow_always_save - If changed from false to true and File -> Save was disabled, it remains disabled after the file is changed (the shortcut works properly). Easily fixable, but why do we enable Save All if always_save is true? If we expect each Save All to always save all files, I'd better be sure to never enable this. [restart], or a ui_save_buttons_toggle() call when updating the preferences.
brace_match_ltgt - [current]
compiler_tab_autoscroll - [current]
complete_snippets_whilst_editing - [current]
find_selection_type - [current]
gio_unsafe_save_backup - [current]
indent_hard_tab_width - [restart]. And a big, fat warning. Best of all, kill it.
msgwin_[a-z]+_visible - Safe to change. [restart], or a call to msgwin_show_hide_tabs() when updating the preferences.
new_document_after_close - [current]
number_[a-z_]+_items - Safe to change. [restart].
show_editor_scrollbars - Safe to change, affects the newly opened files. For the files already open, [restart] or foreach document sci_set_scrollbar_mode().
show_symbol_list_expanders - TODO
statusbar_template - [current]
use_gtk_word_boundaries - TODO
use_safe_file_saving - [current]. I think utils_write_file() should be based on write_data_to_disk(), and should display a popup message like document_save_file(). The second worst thing after losing a document is to lose a project/config/... file, possibly without even noticing it.
Am 05.01.2011 20:25, schrieb Dimitar Zhekov:
On Wed, 5 Jan 2011 17:57:15 +0000 Nick Treleaven nick.treleaven@btinternet.com wrote:
The question is what should the default be, and personally I think some kind of limit is better than no limit.
50MB file size IMHO. Most users will never need it, and most machines will be able to handle it, even for XML. Personally I work with larger files, but prefer another editor for them.
I just tried to open a 42MB diff on emacs and he did ask me, whether I really want to open. Not sure whether it was about the 42 or really about the size :D Just as some kind of follow up here :)
Cheers, Frank
On Wed, 5 Jan 2011 11:08:18 +1100 Lex Trotman elextr@gmail.com wrote:
And on modern kernels virtual memory can be overcommitted significantly so it may not work
I haven't said that a successful alloc guarantees everything will be fine. :) But with the kernel defaults, you'll at least catch the most obvious cases.
is expensive since the page tables have to be constructed even if the memory is not used.
CPU cost: that's only once per file, memory cost: < 1%
Also this could cause fragmentation of the virtual address space that may limit how many times it will work.
Yes, or some other problems. It's a hack, and should only be used if we can't agree on anything better. Personally I'm for a preference.
---
On Tue, 4 Jan 2011 22:05:52 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory.
What a strange idea... Unless your CPU lacks MMU, of course.
GNU malloc uses mmap so it is all virtual.
mmap for large blocks and heap for small blocks IIRC, and naturally, the heap itself is mmap-ed.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
I have 1 GB of physical memory. (01 << 040) is the first integer that does not fit into unsigned int.
So then, using more than 4GB _will_ be a problem on your system, QED. I pointed out it's a rough check.
But if your malloc() fails on 4GB, and not below that, you either have an unusually large swap file, or your kernel is set to always overcommit. With the defaults, the check works much better.
On 6 January 2011 06:14, Dimitar Zhekov dimitar.zhekov@gmail.com wrote:
On Wed, 5 Jan 2011 11:08:18 +1100 Lex Trotman elextr@gmail.com wrote:
And on modern kernels virtual memory can be overcommitted significantly so it may not work
I haven't said that a successful alloc guarantees everything will be fine. :)
Understand, I just think that overcommit capability means that it errs on the unsafe side which is the wrong way to avoid lockups.
But with the kernel defaults, you'll at least catch the most
obvious cases.
IIUC the default is to overcommit, but I guess it depends on what your distro sets, I no longer bother to compile kernels.
is expensive since the page tables have to be constructed even if the memory is not used.
CPU cost: that's only once per file, memory cost: < 1%
Also this could cause fragmentation of the virtual address space that may limit how many times it will work.
Yes, or some other problems. It's a hack, and should only be used if we can't agree on anything better. Personally I'm for a preference.
Agree, I'm just trying to point out some risks in support of the preference option.
On Tue, 4 Jan 2011 22:05:52 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory.
What a strange idea... Unless your CPU lacks MMU, of course.
GNU malloc uses mmap so it is all virtual.
mmap for large blocks and heap for small blocks IIRC, and naturally, the heap itself is mmap-ed.
But what does windows do?? Don't want more differences between versions to maintain.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
I have 1 GB of physical memory. (01 << 040) is the first integer that does not fit into unsigned int.
So then, using more than 4GB _will_ be a problem on your system, QED. I pointed out it's a rough check.
But if your malloc() fails on 4GB, and not below that, you either have an unusually large swap file, or your kernel is set to always overcommit. With the defaults, the check works much better.
And you have a 64 bit kernel.
Cheers Lex
-- E-gards: Jimmy _______________________________________________ Geany mailing list Geany@uvena.de http://lists.uvena.de/cgi-bin/mailman/listinfo/geany
Dnia czwartek, 6 stycznia 2011 o 00:35:16 Lex Trotman napisał(a):
On Tue, 4 Jan 2011 22:05:52 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory.
What a strange idea... Unless your CPU lacks MMU, of course.
GNU malloc uses mmap so it is all virtual.
mmap for large blocks and heap for small blocks IIRC, and naturally, the heap itself is mmap-ed.
But what does windows do?? Don't want more differences between versions to maintain.
Windows uses HeapAlloc or VirtualAlloc. However, what you need to know probably is what your run-time library does, not what Microsoft Windows does. The situation with RTL under Microsoft Windows is that there are various vendors and we cannot expect much consistency among them. Of course you can say Geany should be compiled by MinGW — but I would rather not require that.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
I have 1 GB of physical memory. (01 << 040) is the first integer that does not fit into unsigned int.
So then, using more than 4GB _will_ be a problem on your system, QED. I pointed out it's a rough check.
But if your malloc() fails on 4GB, and not below that, you either have an unusually large swap file, or your kernel is set to always overcommit. With the defaults, the check works much better.
And you have a 64 bit kernel.
Guilty as charged :-)
Cheers, Chris
Dnia środa, 5 stycznia 2011 o 20:14:00 Dimitar Zhekov napisał(a):
On Wed, 5 Jan 2011 11:08:18 +1100 Lex Trotman elextr@gmail.com wrote:
And on modern kernels virtual memory can be overcommitted significantly so it may not work
I haven't said that a successful alloc guarantees everything will be fine. :) But with the kernel defaults, you'll at least catch the most obvious cases.
is expensive since the page tables have to be constructed even if the memory is not used.
CPU cost: that's only once per file, memory cost: < 1%
Also this could cause fragmentation of the virtual address space that may limit how many times it will work.
Yes, or some other problems. It's a hack, and should only be used if we can't agree on anything better. Personally I'm for a preference.
On Tue, 4 Jan 2011 22:05:52 +0100 Krzysztof Żelechowski giecrilj@stegny.2a.pl wrote:
I do not know where you got it from, but the last OS I know that behaved like that was MacOS 7. [cut]
Like what?..
Like necessarily allocating a contiguous block of physical memory.
What a strange idea... Unless your CPU lacks MMU, of course.
GNU malloc uses mmap so it is all virtual.
mmap for large blocks and heap for small blocks IIRC, and naturally, the heap itself is mmap-ed.
If by "01 << 040" you mean 4GB, how much real and virtual memory do you have?
I have 1 GB of physical memory. (01 << 040) is the first integer that does not fit into unsigned int.
So then, using more than 4GB _will_ be a problem on your system, QED. I pointed out it's a rough check.
But if your malloc() fails on 4GB, and not below that, you either have an unusually large swap file, or your kernel is set to always
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 1 0 324092 35424 17904 283900 7 38 191 77 479 810 16 5 76 3 0
MemTotal: 1004948 kB SwapTotal: 1510040 kB VmallocTotal: 34359738367 kB VmallocChunk: 34359546964 kB
Filename Type Size Used Priority /dev/sda5 partition 1510040 276772 -1
overcommit. With the defaults, the check works much better.
overcommit_memory = 0 overcommit_ratio = 50 CommitLimit: 2012512 kB
vmalloc is not set in /proc/cmdline.
I think this setup is quite typical.
Chris