Hi,
I have been recently playing around handling binary files with Geany. At least Notepad++, which is also Scintilla-based, is able to do that (opens and saves binaries correctly). This feature looks very promising for me, as there are sometimes files with mixed content (e.g. Outlook forms, resource files or malformed e-mails).
First, I've found a problem (seems that was partially due to my charset handling patch http://lists.uvena.de/geany-devel/2009-October/001480.html): when opening a binary (or malformed) file Geany would hang eating 100% CPU. My 1st patch is to fix that (tested for some cases, all worked... but should be checked once more).
Then, I tried to make Geany open binaries correctly (the 2nd patch). What I did was: - Made encodings_convert_to_utf8 attempt to convert with "fast" flag set if nothing else succeeded (if Unicode checks failed while conversion itself succeeded) - Made encodings_convert_to_utf8 (in fact, its duplicate) return not only converted text, but also its length ('cause if a binary file contains nulls it would be truncated instead) - Made document_open_file_full use SCI_ADDTEXT instead of SCI_SETTEXT when showing file contents. The former is able to handle NULLs while the latter isn't
Finally, I was able to view, edit and correctly save a binary file with NULLs. I've attached 2 screenshots for you to see what it looks like.
I should underline that this feature (and this patch) is HIGHLY experimental and I didn't perform any more or less satisfactory testing (just opened a file with NULLs, added "ABCDEF" and saved). So using any advanced features (search and replace etc) may lead to unpredictable behaviour. At least there should be a choice how to handle binary files (allow open or open a new empty file as before). I'll make some testing in a regular working environment and, if necessary, issue other patches (of course, if someone is interested in that feature).
Regards, Alexey
On Sat, 06 Feb 2010 04:26:37 +0300, Алексей wrote:
Hey,
I have been recently playing around handling binary files with Geany. At least Notepad++, which is also Scintilla-based, is able to do that (opens and saves binaries correctly). This feature looks very promising for me, as there are sometimes files with mixed content (e.g. Outlook forms, resource files or malformed e-mails).
First, I've found a problem (seems that was partially due to my charset handling patch http://lists.uvena.de/geany-devel/2009-October/001480.html): when opening a binary (or malformed) file Geany would hang eating 100% CPU. My 1st patch is to fix that (tested for some cases, all worked... but should be checked once more).
Could you provide a test case to reproduce the problem?
Then, I tried to make Geany open binaries correctly (the 2nd patch). What I did was:
- Made encodings_convert_to_utf8 attempt to convert with "fast" flag
set if nothing else succeeded (if Unicode checks failed while conversion itself succeeded)
- Made encodings_convert_to_utf8 (in fact, its duplicate) return not
only converted text, but also its length ('cause if a binary file contains nulls it would be truncated instead)
- Made document_open_file_full use SCI_ADDTEXT instead of SCI_SETTEXT
when showing file contents. The former is able to handle NULLs while the latter isn't
Finally, I was able to view, edit and correctly save a binary file with NULLs. I've attached 2 screenshots for you to see what it looks like.
I don't see why we should support this. Geany is a text editor. Text files don't contain NULL bytes. If you want edit binary files, hex editors are more useful, regardless of what Notepad++ does.
And as you said, make such files opening isn't enough at all, there are certainly many places in Geany which rely on the fact files are real text files and I don't think this is wrong at all. Geany is a text editor.
Regards, Enrico
Could you provide a test case to reproduce the problem?
Take some binary/malformed file (a small ZIP archive is a good example) and try opening it with the latest SVN version of Geany (that can ocassionaly happen). You'll get a hangup with 100% CPU usage. If it is OK, try setting a "Default charset for opening files" to some 1-byte charset (e.g. ISO-8859-1).
I don't see why we should support this. Geany is a text editor. Text files don't contain NULL bytes. If you want edit binary files, hex editors are more useful, regardless of what Notepad++ does.
They are, but sometimes I want just to see what's inside (given an unknown or broken file that I need to extract data from). Even poor old Windows notepad can handle binaries - why can't this pretty editor? More, some editors (Notepad++ or Midnight Commander's builtin) have a builtin Hex editor, which is sometimes useful. I'm using Geany as a primary text editor both on Windows and Linux, and I just wanted to make it more universal.
And as you said, make such files opening isn't enough at all, there are certainly many places in Geany which rely on the fact files are real text files and I don't think this is wrong at all. Geany is a text editor.
Sure. Even such simple things as changing file encoding/line endings will make a mess from your binary. But if we provide a very basic capability - that would be interesting. If we use this feature, there must be a warning like "You are opening a binary file in a text editor, this can lead to an unexpected result and we strongly recommend that you make a backup".
If a plugin crashes or hangs the whole program due to some unusual sequence of symbols - that's a problem of this plugin, I think. The other question is that the plugin will give unexpected results on unusual data - but that's what the end user should be warned and care about.
I do not intend to make you use my patches (except the first one, if you can reproduce my behaviour)- just a proposal I find interesting. At least, nobody would prohibit me to use the patch myself:)
-----Original Message----- From: Enrico Tr?ger enrico.troeger@uvena.de To: geany-devel@uvena.de Date: Sat, 6 Feb 2010 17:06:10 +0100 Subject: Re: [Geany-devel] Binary files - a hangup fix and some more
On Sat, 06 Feb 2010 04:26:37 +0300, Алексей wrote:
Hey,
I have been recently playing around handling binary files with Geany. At least Notepad++, which is also Scintilla-based, is able to do that (opens and saves binaries correctly). This feature looks very promising for me, as there are sometimes files with mixed content (e.g. Outlook forms, resource files or malformed e-mails).
First, I've found a problem (seems that was partially due to my charset handling patch http://lists.uvena.de/geany-devel/2009-October/001480.html): when opening a binary (or malformed) file Geany would hang eating 100% CPU. My 1st patch is to fix that (tested for some cases, all worked... but should be checked once more).
Could you provide a test case to reproduce the problem?
Then, I tried to make Geany open binaries correctly (the 2nd patch). What I did was:
- Made encodings_convert_to_utf8 attempt to convert with "fast" flag
set if nothing else succeeded (if Unicode checks failed while conversion itself succeeded)
- Made encodings_convert_to_utf8 (in fact, its duplicate) return not
only converted text, but also its length ('cause if a binary file contains nulls it would be truncated instead)
- Made document_open_file_full use SCI_ADDTEXT instead of SCI_SETTEXT
when showing file contents. The former is able to handle NULLs while the latter isn't
Finally, I was able to view, edit and correctly save a binary file with NULLs. I've attached 2 screenshots for you to see what it looks like.
I don't see why we should support this. Geany is a text editor. Text files don't contain NULL bytes. If you want edit binary files, hex editors are more useful, regardless of what Notepad++ does.
And as you said, make such files opening isn't enough at all, there are certainly many places in Geany which rely on the fact files are real text files and I don't think this is wrong at all. Geany is a text editor.
Regards, Enrico
-- Get my GPG key from http://www.uvena.de/pub.asc
ATTACHMENT: application/pgp-signature _______________________________________________ Geany-devel mailing list Geany-devel@uvena.de http://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Лучшие предложения интернет-магазинов на Товары@Mail.Ru http://r.mail.ru/cln4267/torg.mail.ru/
On 06.02.2010 17:37, Алексей Антипов wrote:
Could you provide a test case to reproduce the problem?
Take some binary/malformed file (a small ZIP archive is a good example) and try opening it with the latest SVN version of Geany (that can ocassionaly happen). You'll get a hangup with 100% CPU usage. If it is OK, try setting a "Default charset for opening files" to some 1-byte charset (e.g. ISO-8859-1).
I don't see why we should support this. Geany is a text editor. Text files don't contain NULL bytes. If you want edit binary files, hex editors are more useful, regardless of what Notepad++ does.
They are, but sometimes I want just to see what's inside (given an unknown or broken file that I need to extract data from). Even poor old Windows notepad can handle binaries - why can't this pretty editor? More, some editors (Notepad++ or Midnight Commander's builtin) have a builtin Hex editor, which is sometimes useful. I'm using Geany as a primary text editor both on Windows and Linux, and I just wanted to make it more universal.
And as you said, make such files opening isn't enough at all, there are certainly many places in Geany which rely on the fact files are real text files and I don't think this is wrong at all. Geany is a text editor.
Sure. Even such simple things as changing file encoding/line endings will make a mess from your binary. But if we provide a very basic capability - that would be interesting. If we use this feature, there must be a warning like "You are opening a binary file in a text editor, this can lead to an unexpected result and we strongly recommend that you make a backup".
If a plugin crashes or hangs the whole program due to some unusual sequence of symbols - that's a problem of this plugin, I think. The other question is that the plugin will give unexpected results on unusual data - but that's what the end user should be warned and care about.
I do not intend to make you use my patches (except the first one, if you can reproduce my behaviour)- just a proposal I find interesting. At least, nobody would prohibit me to use the patch myself:)
I would love this. I find myself needing additional text editors (like scite) because geany is unable to view binary files. I think the mixed content example in the first email shows why geany should support it.
Is that a costly feature or what speaks against it?
Best regards.
On Sat, 06 Feb 2010 19:37:53 +0300, Алексей wrote:
Could you provide a test case to reproduce the problem?
Take some binary/malformed file (a small ZIP archive is a good example) and try opening it with the latest SVN version of Geany (that can ocassionaly happen). You'll get a hangup with 100% CPU usage. If it is OK, try setting a "Default charset for opening files" to some 1-byte charset (e.g. ISO-8859-1).
Ok, got it. Committed, thanks.
I don't see why we should support this. Geany is a text editor. Text files don't contain NULL bytes. If you want edit binary files, hex editors are more useful, regardless of what Notepad++ does.
They are, but sometimes I want just to see what's inside (given an unknown or broken file that I need to extract data from). Even poor old Windows notepad can handle binaries - why can't this pretty editor? More, some editors (Notepad++ or Midnight Commander's builtin) have a builtin Hex editor, which is sometimes useful. I'm using Geany
A hex editor is something completely different from a text editor. I also use mc's builtin hex editor which is pretty cool and I especially use it in case I got any files which contain NULL bytes (for whatever reason).
And as you said, make such files opening isn't enough at all, there are certainly many places in Geany which rely on the fact files are real text files and I don't think this is wrong at all. Geany is a text editor.
Sure. Even such simple things as changing file encoding/line endings will make a mess from your binary. But if we provide a very basic capability - that would be interesting. If we use this feature, there must be a warning like "You are opening a binary file in a text editor, this can lead to an unexpected result and we strongly recommend that you make a backup".
If a plugin crashes or hangs the whole program due to some unusual sequence of symbols - that's a problem of this plugin, I think. The other question is that the plugin will give unexpected results on unusual data - but that's what the end user should be warned and care
Not much doubt about that. But users still get annoyed and we will get the bug reports about the crashes...
I do not intend to make you use my patches (except the first one, if you can reproduce my behaviour)- just a proposal I find interesting.
Is that a costly feature or what speaks against it?
I just find it less useful. Besides Alexey's patch, there are probably certain other changes necessary to get this working properly, or at least good enough to put the remaining risks into the users' responsibility. For me, the first statement still counts: Geany is a text editor. Yes, this is a bit strict but still true.
At least, nobody would prohibit me to use the patch myself:)
Haha, yeah. That's one of the benefits of the GPL.
Regards, Enrico
Alright, then I would like to announce that everyone who's interested in this option (like Thomas Martitz) can feel free to contact me and get the latest version of the patch. But please note that this is distributed "as is" and is not intended to provide any features more advanced than just "open - replace some bytes - save".
-----Original Message----- From: Enrico Tr?ger enrico.troeger@uvena.de To: geany-devel@uvena.de Date: Sun, 7 Feb 2010 19:54:07 +0100 Subject: Re: [Geany-devel] Binary files - a hangup fix and some more
On Sat, 06 Feb 2010 19:37:53 +0300, Алексей wrote:
Could you provide a test case to reproduce the problem?
Take some binary/malformed file (a small ZIP archive is a good example) and try opening it with the latest SVN version of Geany (that can ocassionaly happen). You'll get a hangup with 100% CPU usage. If it is OK, try setting a "Default charset for opening files" to some 1-byte charset (e.g. ISO-8859-1).
Ok, got it. Committed, thanks.
I don't see why we should support this. Geany is a text editor. Text files don't contain NULL bytes. If you want edit binary files, hex editors are more useful, regardless of what Notepad++ does.
They are, but sometimes I want just to see what's inside (given an unknown or broken file that I need to extract data from). Even poor old Windows notepad can handle binaries - why can't this pretty editor? More, some editors (Notepad++ or Midnight Commander's builtin) have a builtin Hex editor, which is sometimes useful. I'm using Geany
A hex editor is something completely different from a text editor. I also use mc's builtin hex editor which is pretty cool and I especially use it in case I got any files which contain NULL bytes (for whatever reason).
And as you said, make such files opening isn't enough at all, there are certainly many places in Geany which rely on the fact files are real text files and I don't think this is wrong at all. Geany is a text editor.
Sure. Even such simple things as changing file encoding/line endings will make a mess from your binary. But if we provide a very basic capability - that would be interesting. If we use this feature, there must be a warning like "You are opening a binary file in a text editor, this can lead to an unexpected result and we strongly recommend that you make a backup".
If a plugin crashes or hangs the whole program due to some unusual sequence of symbols - that's a problem of this plugin, I think. The other question is that the plugin will give unexpected results on unusual data - but that's what the end user should be warned and care
Not much doubt about that. But users still get annoyed and we will get the bug reports about the crashes...
I do not intend to make you use my patches (except the first one, if you can reproduce my behaviour)- just a proposal I find interesting.
Is that a costly feature or what speaks against it?
I just find it less useful. Besides Alexey's patch, there are probably certain other changes necessary to get this working properly, or at least good enough to put the remaining risks into the users' responsibility. For me, the first statement still counts: Geany is a text editor. Yes, this is a bit strict but still true.
At least, nobody would prohibit me to use the patch myself:)
Haha, yeah. That's one of the benefits of the GPL.
Regards, Enrico
-- Get my GPG key from http://www.uvena.de/pub.asc
ATTACHMENT: application/pgp-signature _______________________________________________ Geany-devel mailing list Geany-devel@uvena.de http://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Тесты новейших гаджетов на Hi-tech.Mail.Ru http://r.mail.ru/cln5823/hi-tech.mail.ru/
