I didn't review the code, just an early note: the Scintilla lexer should be first submitted to and accepted by the Scintilla project (https://www.scintilla.org/). Also the Ctags parser should be first submitted to and accepted by the Universal Ctags project (https://github.com/universal-ctags/ctags). See also https://github.com/universal-ctags/ctags/issues/1566#issuecomment-515902669 where someone also started a parser.

We want to stay inline with the upstream projects and so the code should be accepted there first.

They will need to be tested by julia+geany users first anyway.

And a reminder to those who test it to comment, that lets @getzze know its working (or what needs fixing :) and it lets Geany project devs know there is a demand.

@getzze pushed 2 commits.

59712ae8d4f06e39711847301044beae932ab83b lexer cleanup f1da05e2bf5380b1bb1d853647420365c889ccab ctag: parse short functions

@getzze pushed 1 commit.

bb7fc603c1c540caac0e007d9714f5d144483586 ctags: better parse functions. add type parsing

Upstream merge: - [ ] Scintilla Merge Request: https://sourceforge.net/p/scintilla/code/merge-requests/25/ - [ ] Universal-ctags PR: https://github.com/universal-ctags/ctags/pull/2654

@getzze pushed 5 commits.

00dba4526aa0b8ea16c5d26f38402e21ca7e9358 lexer: add unicode support identifiers 4a4cc382ddc25ee64e91a7313f6465c6ec567937 ctags: add backtick recognition c544769be5047a6833b02f825c4fe5aec29b9055 lexer: add backtick recognition c255b2c68728fb014c5e53bc3de7dadaa0001415 merge from ctags 7f69aebeec9b4089fb122867ab0e25db94e8f66d update lexer

Thanks for your efforts so far @getzze. I'd be happy to see this merged. I wanted to test, but have trouble building the branch `julia_lexxer` in a standard Kubuntu 20.04 environment (gcc 9.3.0): ``` CXX lexers/LexMake.lo lexers/LexJulia.cxx: In static member function ‘static Scintilla::ILexer* LexerJulia::LexerFactoryJulia()’: lexers/LexJulia.cxx:123:25: error: invalid new-expression of abstract class type ‘LexerJulia’ 123 | return new LexerJulia(); | ^ lexers/LexJulia.cxx:90:7: note: because the following virtual functions are pure within ‘LexerJulia’: 90 | class LexerJulia : public DefaultLexer { | ^~~~~~~~~~ In file included from lexers/LexJulia.cxx:25: ./include/ILexer.h:92:34: note: ‘virtual const char* Scintilla::ILexerWithIdentity::PropertyGet(const char*)’ 92 | virtual const char * SCI_METHOD PropertyGet(const char *key) = 0; | ^~~~~~~~~~~ CXX lexers/LexMarkdown.lo ```

Thanks for testing it, I commited a correction, you should be able to compile it now. Happy to hear your feedback.

@elextr @getzze I have not discovered any further issues. If more testers or feedback on the level of demand is required, I suggest announcing this on [https://discourse.julialang.org/%5D(https://discourse.julialang.org/).

@AndiMD Thanks for reporting these problems! I think it would be great to announce it on Julia discourse.

1. Highlighting interpolated strings seems inconsistent: Depending on using `$(var)` vs `$var` and single quote vs triple quote strings, I see different results. Example:
   `"($scanPosX,$scanPosY): (k±Δk)=$(kLaserApprox) $(begin x+5 end)"`
   `"""($scanPosX,$scanPosY): (k±Δk)=$(kLaserApprox)  $(begin x+5 end)"""`

Interpolation highlighting is only done for single-quote strings, but I am planning to allow it in triple-quote and commands also. For `$(var)` vs `$var`, I didn't find a way to highlight `$(` in the former so I ended up highlighting only what is inside the parenthesis. Because in the latter case there are no parenthesis, nothing is highlighted. I can try to find a way to highlight both, including the `$`.

2. Macros confuse the lexer (possibly we have to live with that, since macros can have arbitrary syntax)
   Example:

Base.@kwdef struct A a::Float64 = 1E-19 b::Float64 = 1E-22 end

Thanks for reporting, that are actually two bugs: - defined values for the attributes are not parsed at all - `Base.@kwdef` is not recognized as a macro (`@kwdef` is well recognized):

```julia @eval fun1(x) = $pi # recognizes `fun1` short function definition Base.@eval fun1(x) = $pi # does not recognize `fun1` short function definition ```

@getzze pushed 2 commits.

09e8137b6f8db325289fd94b5577247446d56be7 better handling of struct with @kwdef 04cf6b3c16fc3cb3c6c615b843c99abd6f92b6e2 interpolation of all strings and correct folding of list comprehension

Found another edge case: `(2E9)` / `(2e9)`/ `(2f9)`: the right bracket is highlighted as part of the number.

Thanks for the feedback. I pushed a patch for it, along with some other improvements: - merge ctags from upstream - do not highlight interpolation for raw strings - correct folding - add REPL autocompletion using snippets (needs to change wordchars)

What's the status on this? I love both Julia and Geany and this looks like a really good improvement.

@getzze pushed 4 commits.

6400fe99fc439a48001b614bc27d2109e772f102 correct scintilla warnings 613d38bad18b36f6bbb2267642b40b2dcfabeb0e merge conflict 8f9e59f14786d2ef5c135b768b18092c8edf2125 merge conflict 31bede9a57915916e1066d380925f0fef3677680 merge conflict

@getzze pushed 1 commit.

eb36bae9efa1ee20e3aef8a3b4f479d4b6e81040 make raw string literals list modificable

@getzze pushed 2 commits.

9b7d6e4dc3d620041c6c4e024bcc96fecf2ad4df merge upstream lexilla corrections 9b377e2e5869c0720667350f407e098dd9819b12 set SCLEX_JULIA=133 like lexilla upstream

This looks almost complete.... I hope we get this in the newer versions of Genie.... :smiley: Again @getzze Thanks!

There are still some open issues with Lexilla: https://github.com/ScintillaOrg/lexilla/pull/13

@getzze can you push some inconsequential change (or a big change if you have one :) to see if CI has been fixed?

@getzze commented on this pull request.

+# MIME type

+mime_type=text/x-julia + +# single comments, like # in this file +comment_single=# +# multiline comments +comment_open=#= +comment_close==# + +[indentation] +width=4 +# 0 is spaces, 1 is tabs, 2 is tab & spaces +type=0 + + +[build_settings]

Is it `[build_menu]` or `[build-menu]` ? (`filetypes.python` has '[build-menu]')

Yay CI works again!!

One thing you might like to consider (can be later) is a test for the ctags parser, see `tests/ctags` so CI would notice if a ctags infrastructure update like #2830 broke your parser.

Have approved the changes (pending build-menu :) and LGTM, but havn't tested it yet, might have time in a few days then should be mergable.

Closed #2584.

IIUC most of the tests are simply from ctags anyway, many of their names relate to ctags bug reports

IIUC the tests are just one (or more) source file(s) mentioned in `Makefile.am` and tags file(s) with the same name but extension `.tags` (generated by Geany, they are tagmanager format, which is binary, so not manually generated). I think mentioning the source files in `Makefile.am` sources list makes the existing test harness feed the source to Geany and generate and compare to the tags files.

I pushed some changed from Lexilla (like using a new keyword list for coloring builtin functions).

I added a patch file in the `scintilla` folder, so the lexer works with lexilla v4. Also added a ctags test file (I didn't manage to generate the binary file).

@getzze pushed 1 commit.

bc627857f1236d162cbf6419c0ba91ee29c7c7a3 repair CI

@getzze pushed 1 commit.

17e670ab45833ec528d9da8112f44465b7d340f9 update LexJulia patch

@getzze pushed 2 commits.

2330dae685550f752e347aea231d1b2344496803 add SCE_JULIA_TYPEOPERATOR and remove no-interpolaton option 897ae6ad15a9808ec039e5c147e44edcf98864e0 upstream merge

I just rebased to two commits. I left the ctags test file in a separate commit because I am not sure it is needed, julia parser is well tested in ctags upstream.

WFM, although testing limited, thanks @getzze

@elextr Any idea when's the next release?

Hello, in the following snippet, the 4th line after the `#` char constant parsed as comment: ```julia ok=true for r=2:R+1,c=2:C+1 if table[r,c]=='#' all(table[r+d[1],c+d[2]]!='#' for d=[(1,0),(-1,0),(0,1),(0,-1)])&&(ok=false;break) end end ``` it can be resolved by putting an extra space after the operator. (geany version 1.39)

Its actually lexing the first `'` as an operator, possibly a postfix transpose, then the `#` is indeed a comment.

Lexers come from the [Lexilla](https://github.com/ScintillaOrg/lexilla/issues) project. Since Geany folks don't know Julia, please post the issue there so the Julia lexer maintainer can talk to you directly.

Post a link to the issue here.

Thanks for catching this bug! I submitted a fix to lexilla: https://github.com/ScintillaOrg/lexilla/pull/98