[geany/geany] b23201: encodings: Fix detection of non-UTF-8 inline encodings - Commits

8 Dec 2023


      Branch:      refs/heads/encodings-fixes
Author:      Colomban Wendling ban@herbesfolles.org
Committer:   Colomban Wendling ban@herbesfolles.org
Date:        Fri, 08 Dec 2023 20:43:40 UTC
Commit:      b23201c01390e6ed6d631cf232e404691ffe91ec
             https://github.com/geany/geany/commit/b23201c01390e6ed6d631cf232e404691ffe91...
Log Message:
-----------
encodings: Fix detection of non-UTF-8 inline encodings
If there are non-UTF-8 data in the first 512 bytes we failed to detect
the inline encoding using the regex match, because our regexes required
a fully valid UTF-8 buffer.
This kind of defeats the purpose of the feature; so relax the match to
use plain bytes instead.
Modified Paths:
--------------
    src/encodings.c
    tests/test_encodings.c
Modified: src/encodings.c
2 lines changed, 1 insertions(+), 1 deletions(-)
===================================================================
@@ -323,7 +323,7 @@ void encodings_select_radio_item(const gchar *charset)
 static GRegex *regex_compile(const gchar *pattern)
 {
    GError *error = NULL;
-	GRegex *regex = g_regex_new(pattern, G_REGEX_CASELESS, 0, &error);
+	GRegex *regex = g_regex_new(pattern, G_REGEX_CASELESS | G_REGEX_RAW, 0, &error);
if (!regex)
    {
Modified: tests/test_encodings.c
3 lines changed, 1 insertions(+), 2 deletions(-)
===================================================================
@@ -191,8 +191,7 @@ static void test_encodings_convert_iso8859_to_utf8_auto(void)
     * - gnome-text-editor(1) is just broken, opens as gedit, but says I don't
     *   have permission to open that file :)  looks like a generic error. */
    /*TEST(TRUE, "\xef\xbb\xbf""not B\xd3M", "ï»¿not BÓM", NULL);*/
-	/* fails to detect the suggested encoding */
-	/*TEST(TRUE, "coding:iso-8859-2 \xd8""ed", "coding:iso-8859-2 Řed", NULL);*/
+	TEST(TRUE, "coding:iso-8859-2 \xd8""ed", "coding:iso-8859-2 Řed", NULL);
    /* with NULs */
    TEST(FALSE, "W\xec""th\0z\xe9""r\xf8""s", "Wìth\0zérøs", "ISO-8859-1");
    TEST(FALSE, "W\xec""th\0z\xe9""r\xf8""s", "Wìth\0zérøs", "ISO-8859-15");
--------------
This E-Mail was brought to you by github_commit_mail.py (Source: https://github.com/geany/infrastructure).

    