[Github-comments] [geany/geany] Geany encoding determination broken? (#2910)

[elextr]

You should include error messages so we can see where the failures happen, but some are obvious:

1. None of the files are ASCII, ASCII is less than 127 (0x80) and all files contain 0xC0
2. There is no such thing as "no encoding", the selection in `Preferences->Files->Encodings->Use fixed encoding` might be seen as misleading in this case.  The Geany buffer _must_ be UTF-8 so loading without doing any encoding conversion still must check for UTF-8 and fails because in all cases your files are not UTF-8, so to try to be helpful Geany falls back to finding an encoding that works and converts to UTF-8.  
3. The first it happens to find for `iso88591.txt` happens to be ISO-8859-1.
4. `breaks.txt` contains a null character which is documented to not work, as the manual says. "Only text files are supported, i.e. opening files which contain NULL-bytes may fail. Geany will try to open the file anyway but it is likely that the file will be truncated because it can only be read up to the first occurrence of a NULL-byte. All characters after this position are lost and are not written when you save the file."  Geany is C code, C strings terminate in NULL, so all operations will stop at that point as noted in the manual.  If its possible truncation would happen Geany will not open the file to avoid possible data loss.
5. `utf16le.txt` also searches for an encoding that works and happens to find UTF-16LE.  I'm not sure what you mean by "the displayed output does not change for me", the code points the file represents in UTF-16LE may not exist in your font, so the code point value may be shown, here I get a CJK character for 0x61C0 and 0A00 for 0x0A00 since my font doesn't have whatever that is.
6. Detection of encodings for files a few bytes long is entirely unreliable, its just deterministic randomness which encoding will be found first (the order of an internal list that can change in any release).

> Is Geany really unable to properly open any of the above files?

The first and third open "properly" by the Geany definition of "properly", and the second is refused for the reason given.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/geany/geany/issues/2910#issuecomment-929695285
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.geany.org/pipermail/github-comments/attachments/20210928/664a27f9/attachment.htm>