The file browser plugin does its own thing, which is to convert to lower case and then sort by Unicode code point value. That is ok for ASCII based locales, but less so for non-ascii where the Unicode collation is more appropriate eg sorting ñ between n and o in Spanish.
The collation ordering is performed by system library code, not Geany code. It is supposed to follow the Unicode standard I linked above, so I suspect its not modifiable.