There were mostly expected "problems" like new source files that had to be added, rename of `perl6.c` to `raku.c`, update of kind mappings, etc.
The only unexpected thing is the behavior of the javascript parser - this is from the javascript commit message:
There are lots of differences because of
https://github.com/universal-ctags/ctags/commit/6d85089456ed215ce6b6a673744a...
Also
https://github.com/universal-ctags/ctags/commit/b1870b826a384c35671937743720...
seems to confuse the parser in simple.js so it doesn't generate `my_global_var2`.
Finally, Geany reports ``` (geany:820768): Tagmanager-WARNING **: 20:38:28.755: ignoring null tag in /home/parallels/projects/geany/doc/reference/jquery.js(line: 2, language: JavaScript) ```
@b4n Does the PR in general and the javascript parser in particular look good to you? I didn't investigate the javascript parser differences much as I'm not a javascript user myself. At least the NULL warning should be fixed though. You can view, comment on, or merge this pull request online at:
https://github.com/geany/geany/pull/3859
-- Commit Summary --
* Update perl6.c to raku.c parser * Update all parsers and related files to ctags p6.1.20240421.0 * Add ldscript parser * Update parser kind mappings * Map title/subtitle of the rst parser * Map defines to tm_tag_variable_t of verilog so we get the same output as before * Map filter to tm_tag_function_t for powershell to get the same output as before * Update asm test * Disable roles for macro kinds in C/C++ * Update unit tests for javascript
-- File Changes --
M ctags/Makefile.am (14) M ctags/dsl/es.c (37) M ctags/dsl/es.h (6) M ctags/dsl/optscript.c (240) M ctags/dsl/optscript.h (2) M ctags/main/CommonPrelude.c (96) M ctags/main/args.c (6) M ctags/main/colprint.c (11) M ctags/main/ctags.h (50) M ctags/main/debug.c (5) M ctags/main/debug.h (14) M ctags/main/dependency.c (26) M ctags/main/dependency.h (1) M ctags/main/dependency_p.h (1) M ctags/main/e_msoft.h (1) M ctags/main/entry.c (551) M ctags/main/entry.h (159) M ctags/main/error_p.h (4) M ctags/main/field.c (155) M ctags/main/field.h (10) M ctags/main/field_p.h (12) M ctags/main/fmt.c (12) M ctags/main/general.h (2) M ctags/main/htable.c (200) M ctags/main/htable.h (16) A ctags/main/interval_tree_generic.h (197) M ctags/main/keyword.c (38) M ctags/main/keyword.h (20) M ctags/main/kind.c (7) M ctags/main/lregex.c (977) M ctags/main/lregex_p.h (7) M ctags/main/lxpath.c (17) M ctags/main/lxpath.h (2) M ctags/main/lxpath_p.h (1) M ctags/main/main.c (2) M ctags/main/mbcs.c (4) M ctags/main/mini-geany.c (48) M ctags/main/nestlevel.c (50) M ctags/main/nestlevel.h (17) M ctags/main/numarray.c (12) M ctags/main/numarray.h (3) M ctags/main/options.c (238) M ctags/main/options_p.h (5) M ctags/main/param.c (92) M ctags/main/param.h (4) M ctags/main/param_p.h (21) M ctags/main/parse.c (655) M ctags/main/parse.h (35) M ctags/main/parse_p.h (24) M ctags/main/parsers_p.h (40) M ctags/main/promise.c (56) M ctags/main/promise.h (2) M ctags/main/ptag.c (39) M ctags/main/ptag_p.h (5) M ctags/main/ptrarray.c (18) M ctags/main/ptrarray.h (4) M ctags/main/rbtree.c (713) M ctags/main/rbtree.h (193) A ctags/main/rbtree_augmented.h (247) M ctags/main/read.c (144) M ctags/main/read.h (21) M ctags/main/read_p.h (9) M ctags/main/repoinfo.h (2) M ctags/main/routines.c (59) M ctags/main/routines.h (5) M ctags/main/script.c (10) M ctags/main/script_p.h (6) M ctags/main/seccomp.c (2) M ctags/main/selectors.c (445) M ctags/main/selectors.h (20) M ctags/main/sort.c (44) M ctags/main/strlist.c (4) M ctags/main/subparser.h (1) M ctags/main/subparser_p.h (1) M ctags/main/trace.h (14) M ctags/main/trashbox.c (1) M ctags/main/types.h (4) M ctags/main/unwindi.c (6) A ctags/main/utf8_str.c (83) A ctags/main/utf8_str.h (22) M ctags/main/vstring.c (32) M ctags/main/vstring.h (61) M ctags/main/writer-ctags.c (79) M ctags/main/writer-json.c (34) M ctags/main/writer.c (4) M ctags/main/writer_p.h (8) M ctags/main/xtag.c (6) M ctags/main/xtag_p.h (4) M ctags/parsers/abaqus.c (6) M ctags/parsers/ada.c (74) M ctags/parsers/asciidoc.c (42) M ctags/parsers/asm.c (654) M ctags/parsers/autoit.c (34) M ctags/parsers/basic.c (326) M ctags/parsers/bibtex.c (93) A ctags/parsers/bibtex.h (25) M ctags/parsers/clojure.c (38) M ctags/parsers/cobol.c (12) M ctags/parsers/cpreprocessor.c (505) M ctags/parsers/cpreprocessor.h (26) M ctags/parsers/css.c (9) M ctags/parsers/cxx/cxx.c (16) M ctags/parsers/cxx/cxx_debug.c (17) M ctags/parsers/cxx/cxx_keyword.c (81) M ctags/parsers/cxx/cxx_keyword.h (15) M ctags/parsers/cxx/cxx_parser.c (133) M ctags/parsers/cxx/cxx_parser_block.c (100) M ctags/parsers/cxx/cxx_parser_function.c (76) M ctags/parsers/cxx/cxx_parser_internal.h (16) M ctags/parsers/cxx/cxx_parser_lambda.c (3) A ctags/parsers/cxx/cxx_parser_module.c (331) M ctags/parsers/cxx/cxx_parser_namespace.c (4) M ctags/parsers/cxx/cxx_parser_template.c (43) M ctags/parsers/cxx/cxx_parser_tokenizer.c (86) M ctags/parsers/cxx/cxx_parser_typedef.c (13) M ctags/parsers/cxx/cxx_parser_using.c (26) M ctags/parsers/cxx/cxx_parser_variable.c (123) M ctags/parsers/cxx/cxx_qtmoc.c (3) M ctags/parsers/cxx/cxx_scope.c (25) M ctags/parsers/cxx/cxx_scope.h (5) A ctags/parsers/cxx/cxx_side_chain.c (230) A ctags/parsers/cxx/cxx_side_chain.h (33) M ctags/parsers/cxx/cxx_tag.c (159) M ctags/parsers/cxx/cxx_tag.h (50) M ctags/parsers/cxx/cxx_token.c (18) M ctags/parsers/cxx/cxx_token.h (11) M ctags/parsers/cxx/cxx_token_chain.c (92) M ctags/parsers/cxx/cxx_token_chain.h (10) M ctags/parsers/diff.c (4) M ctags/parsers/erlang.c (10) M ctags/parsers/flex.c (18) M ctags/parsers/fortran.c (188) M ctags/parsers/gdscript.c (74) M ctags/parsers/geany_lcpp.h (23) M ctags/parsers/go.c (13) M ctags/parsers/haskell.c (2) M ctags/parsers/haxe.c (57) M ctags/parsers/html.c (181) M ctags/parsers/iniconf.c (24) M ctags/parsers/jscript.c (0) A ctags/parsers/jscript.h (0) M ctags/parsers/json.c (0) M ctags/parsers/julia.c (0) M ctags/parsers/lisp.c (0) M ctags/parsers/lua.c (0) M ctags/parsers/make.c (0) M ctags/parsers/markdown.c (0) M ctags/parsers/markdown.h (0) M ctags/parsers/nsis.c (0) M ctags/parsers/objc.c (0) M ctags/parsers/ocaml.c (0) M ctags/parsers/pascal.c (0) M ctags/parsers/perl.c (0) M ctags/parsers/perl.h (0) M ctags/parsers/php.c (0) M ctags/parsers/powershell.c (0) M ctags/parsers/python.c (0) M ctags/parsers/r.c (0) R ctags/parsers/raku.c (0) M ctags/parsers/rst.c (0) M ctags/parsers/ruby.c (0) A ctags/parsers/ruby.h (0) M ctags/parsers/rust.c (0) M ctags/parsers/sh.c (0) A ctags/parsers/sh.h (0) M ctags/parsers/sql.c (0) M ctags/parsers/tcl.c (0) M ctags/parsers/tcloo.c (0) M ctags/parsers/tex.c (0) M ctags/parsers/typescript.c (0) M ctags/parsers/verilog.c (0) M ctags/parsers/vhdl.c (0) M meson.build (0) M src/tagmanager/tm_parser.c (0) M src/tagmanager/tm_parser.h (0) M src/tagmanager/tm_parsers.h (0) M tests/ctags/1795612.js.tags (0) M tests/ctags/1850914.js.tags (0) M tests/ctags/1878155.js.tags (0) M tests/ctags/1880687.js.tags (0) M tests/ctags/3470609.js.tags (0) M tests/ctags/arraylist.js.tags (0) M tests/ctags/bracematch.js.tags (0) M tests/ctags/bug1950327.js.tags (0) M tests/ctags/bug2888482.js.tags (0) M tests/ctags/bug3571233.js.tags (0) M tests/ctags/complex-return.js.tags (0) M tests/ctags/js-class-related-unterminated.js.tags (0) M tests/ctags/js-const.js.tags (0) M tests/ctags/js-let.js.tags (0) M tests/ctags/js-string-continuation.js.tags (0) M tests/ctags/js-unknown-construct-nesting.js.tags (0) M tests/ctags/jsFunc_tutorial.js.tags (0) M tests/ctags/masm.asm.tags (0) M tests/ctags/parenthesis-rvalue.js.tags (0) M tests/ctags/secondary_fcn_name.js.tags (0) M tests/ctags/simple.js.tags (0) M tests/ctags/simple.rst (0) M tests/ctags/simple.rst.tags (0) M tests/ctags/ui5.controller.js.tags (0)
-- Patch Links --
https://github.com/geany/geany/pull/3859.patch https://github.com/geany/geany/pull/3859.diff
@techee pushed 8 commits.
7943b7d9ec47805caac6b071ca13da4a73c14326 Add ldscript parser f21e1de08482a73fdc41694e7dbc81c9735a29dc Update parser kind mappings e00a2684d34446ae7426ee8e00349f8d3595e2fc Map title/subtitle of the rst parser cdf1aa1fbc7ba8d93dc3373a0c8600a80a4c03be Map defines to tm_tag_variable_t of verilog so we get the same output as before 786819ba79574afcfd0cd92d0ac561a925441c1c Map filter to tm_tag_function_t for powershell to get the same output as before 65bad205df3ce7a5bc089d3a71beac27e03b54de Update asm test 27659bdef5000de4cdf79ca0c154f7c9ad1dcd01 Disable roles for macro kinds in C/C++ 978eb03e3bb33612013495054238fbc2a66683cf Update unit tests for javascript
@techee pushed 2 commits.
aa0c796b5c1e6a1b6a4050086309a5864819eca3 Switch to uctags regex-based matlab parser d515bce2167fd8623c83dc51e38562a4b822d019 Use repoinfo.h with the used tag version
I also added a commit updating the matlab parser to the upstream regex-based one. I'd like to avoid the situation that we have to maintain our (non-regex) version of the parser like in:
https://github.com/geany/geany/pull/3358 https://github.com/geany/geany/pull/3563
Also, the kinds generated by the upstream parser are different than those we parse using our parser so the ctags file are incompatible.
I'll look into this Soon™, but the JS changes might have an issue (might be what you see, might be something else), see [this](https://github.com/universal-ctags/ctags/commit/6d85089456ed215ce6b6a673744a...) which doesn't look right to me.
In any case, thanks for doing this :)
Regarding JS:
The only unexpected thing is the behavior of the javascript parser - this is from the javascript commit message:
There are lots of differences because of
[universal-ctags/ctags@6d85089](https://github.com/universal-ctags/ctags/commit/6d85089456ed215ce6b6a673744a...)
Yeah, all this look OK. Some can be viewed as implementor's choice, but it's fair so I'm fine with it.
Also
[universal-ctags/ctags@b1870b8](https://github.com/universal-ctags/ctags/commit/b1870b826a384c35671937743720...)
seems to confuse the parser in simple.js so it doesn't generate `my_global_var2`.
Despite what's in the test file, this is *not* valid JS (AFAICT, or as far as SpiderMonkey tells me), so it's understandable the parser doesn't get along very well with it. If you add a semicolon after it it fixes it, as it then consider it the end of the construct and "resets" itself.
Finally, Geany reports
(geany:820768): Tagmanager-WARNING **: 20:38:28.755: ignoring null tag in /home/parallels/projects/geany/doc/reference/jquery.js(line: 2, language: JavaScript)
Is this actually anything new? I just fixed a few upstream in https://github.com/universal-ctags/ctags/pull/3993, but I believe we already had this before.
Also, there's one instance in that jQuery code where it's somewhat expected, as there is indeed a zero-length property. For that remaining one, I'm not sure what the parser should do, as it can't really emit an empty name (format would probably not accept it, and anyway consumers are unlikely to like it), yet there *is* one in the input…
@b4n Does the PR in general and the javascript parser in particular look good to you? I didn't investigate the javascript parser differences much as I'm not a javascript user myself. At least the NULL warning should be fixed though.
Yeah, the JS changes look (almost) fine. I raised a couple issues upstream, but it's nothing serious nor that actually affect us much (see e.g. https://github.com/universal-ctags/ctags/issues/3995 and https://github.com/universal-ctags/ctags/issues/3997).
Two things: * we should probabmy use a newer snapshot (some parser changes, e.g. related to class vs. object vs. variable are gonna lead to additional test result changes, which are admittedly better in the newer version, and I'm not sure it's worth having an intermediary step here) * we should probably rather fix *simple.js* test to remove the invalid syntax that confuses the parser. I'll try and do that Soon™.
BTW:
Sync our ctags to the latest tag version (p6.1.20240421.0)
Tags `p*` are NOT releases, but probably pre-versions, or even nightly snapshots. Releases are `v*`. It's probably fine, but noteworthy :slightly_smiling_face:
@b4n Thanks for having a look at the javascript parser.
we should probabmy use a newer snapshot (some parser changes, e.g. related to class vs. object vs. variable are gonna lead to additional test result changes, which are admittedly better in the newer version, and I'm not sure it's worth having an intermediary step here)
Can do that. Should it be part of this PR or a separate one after this one is merged?
we should probably rather fix simple.js test to remove the invalid syntax that confuses the parser. I'll try and do that Soon™.
Could do it myself but better someone who at least remotely knows what the correct syntax is :-).
Tags p* are NOT releases, but probably pre-versions, or even nightly snapshots. Releases are v*.
It's probably fine, but noteworthy 🙂
OK, I didn't notice the release tags. But since these are just about once a year, I guess we want the latest thing and "risk" the consequences, right?
@b4n commented on this pull request.
+static TMParserMapGroup group_LDSCRIPT[] = {
+};
This gives me: ``` src/tagmanager/tm_parser.c:1165:44: warning: ISO C forbids empty initializer braces [-Wpedantic] 1165 | static TMParserMapGroup group_LDSCRIPT[] = { | ^ src/tagmanager/tm_parser.c:1165:25: error: zero or negative size array 'group_LDSCRIPT' 1165 | static TMParserMapGroup group_LDSCRIPT[] = { ``` Might be related to my pedantic flags, but that's a bit problematic.
What about having actual groups, even if unused? ```suggestion static TMParserMapGroup group_LDSCRIPT[] = { {N_("Section"), TM_ICON_STRUCT, tm_tag_struct_t}, {N_("Symbol"), TM_ICON_METHOD, tm_tag_method_t | tm_tag_function_t}, {N_("Version"), TM_ICON_OTHER, tm_tag_other_t}, {N_("Input Section"), TM_ICON_NAMESPACE, tm_tag_namespace_t}, }; ```
Or just a dummy one? ```suggestion static TMParserMapGroup group_LDSCRIPT[] = { {"unused", TM_ICON_NONE, tm_tag_undef_t}, }; ```
@b4n commented on this pull request.
On tests/ctags/matlab_test.m.tags:
that's a bit sad… I'll see if I can get those fixed upstream (it's easy enough to patch actually -- yet, I don't have a clue about matlab :grin: )
@b4n commented on this pull request.
@@ -557,6 +562,7 @@ static TMParserMapEntry map_FREEBASIC[] = {
{'t', tm_tag_struct_t}, // type {'v', tm_tag_variable_t}, // variable {'g', tm_tag_externvar_t}, // enum + {'n', tm_tag_undef_t}, // namespace
No clue about freebasic, but wouldn't it be worth including? kinds are already silly, so the fact `tm_tag_namespace_t` is already in use is not really a reason not to add it, is it?
Can do that. Should it be part of this PR or a separate one after this one is merged?
Whichever you prefer. I'd have said this one given it'll introduce yet another set of changes that are probably best diffed from current master, but I don't really mind either way in practice, so if it's easier for you just do another one afterwards.
Tags p* are NOT releases, but probably pre-versions, or even nightly snapshots. Releases are v*. It's probably fine, but noteworthy 🙂
OK, I didn't notice the release tags. But since these are just about once a year, I guess we want the latest thing and "risk" the consequences, right?
Yeah, I think we can "risk it" :slightly_smiling_face:
@techee pushed 2 commits.
0b52e79c93942d115d94a2df6015a1c704fb0ad6 Add dummy TMParserMapGroup for ldscript to avoid warnings de9ea30900a3c602e89b39666b22029abc9a839a Map freepascal namespaces and add a unit test for them
@techee commented on this pull request.
+static TMParserMapGroup group_LDSCRIPT[] = {
+};
I added the dummy one in the latest commit.
@techee commented on this pull request.
@@ -557,6 +562,7 @@ static TMParserMapEntry map_FREEBASIC[] = {
{'t', tm_tag_struct_t}, // type {'v', tm_tag_variable_t}, // variable {'g', tm_tag_externvar_t}, // enum + {'n', tm_tag_undef_t}, // namespace
Yes, these might be missing in the hierarchy. Added them in the latest commit including the unit test I copied from universal-ctags.
Can do that. Should it be part of this PR or a separate one after this one is merged?
Whichever you prefer. I'd have said this one given it'll introduce yet another set of changes that are probably best diffed from current master, but I don't really mind either way in practice, so if it's easier for you just do another one afterwards.
I just wanted to avoid rebasing this PR on top of another ctags version so I'd prefer a separate PR (which will be much smaller and easier to review).
@techee pushed 1 commit.
acb4678486110d90dd46eb6dea336f5907cafb14 Map freepascal namespaces and add a unit test for them
@b4n commented on this pull request.
+static TMParserMapGroup group_LDSCRIPT[] = {
+};
BTW, it's not merely a warning, but I had an error. However, it's a [GCC extension](https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html)
@b4n pushed 2 commits.
77614bd228aaba187faaef3631d127f231fce1f4 Map PowerShell classes and enums ae75c73b48db70a5d59e2dc02a566f1ebfa2ef46 Fix a JavaScript test
@techee I just added 2 more commits -- feel free to comment and reject them :) Anyway, looks good to me now.
@b4n approved this pull request.
I didn't review all parser's changes (sorry :sob:), but the Geany changes, tests and behavior look good :+1:
Merged #3859 into master.
@techee I just added 2 more commits -- feel free to comment and reject them :)
Yes, that's what I'm gonna do, just to enjoy the power :-).
Anyway, great, I've merged this PR. If you plan to do some more ctags changes, just ping me when you are done and the changes are merged upstream and I'll prepare another sync PR with them.
Once this is dealt with that'd be fine:
* https://github.com/universal-ctags/ctags/pull/3993 * https://github.com/universal-ctags/ctags/pull/3998 (new feature, so quite optional) * https://github.com/universal-ctags/ctags/pull/3999
@techee FWIW, the PRs I wanted merged now are, so it's whenever you like :)
Great, I think I'll just wait for the next ctags tag and will use that one.
github-comments@lists.geany.org