Right now parsing of modules is not enabled. The problem is that module name in ocaml is the name of the source file and with our unit tests the source file name is a temporary name so we get different tag name every time and the unit test fails. You can view, comment on, or merge this pull request online at:
https://github.com/geany/geany/pull/3163
-- Commit Summary --
* Add Ocaml ctags parser
-- File Changes --
M ctags/Makefile.am (1) A ctags/parsers/ocaml.c (2094) M meson.build (1) M src/filetypes.c (2) M src/tagmanager/tm_parser.c (21) M src/tagmanager/tm_parser.h (1) M src/tagmanager/tm_parsers.h (3) M tests/ctags/Makefile.am (1) A tests/ctags/simple.ml (34) A tests/ctags/simple.ml.tags (8) M tests/meson.build (1)
-- Patch Links --
https://github.com/geany/geany/pull/3163.patch https://github.com/geany/geany/pull/3163.diff
How does ctags tests for ocaml work?
I just disabled the module kind tag generation so these tags aren't generated right now. But this also means that for (sub)modules created this way: ``` module ModuleFoo = struct type foobar = ConstructorFoo | ConstructorBar of int * char list end ``` the tag `ModuleFoo` isn't created either and then we get `ModuleFoo.foobar` in the symbol tree because the parent node `ModuleFoo` doesn't exist. But I think it isn't such a big deal.
How does ctags tests for ocaml work?
Oh, or do you mean how it works in universal-ctags? There the unit tests work differently and preserve the name of the tested file so the output is always the same.
Oh, or do you mean how it works in universal-ctags?
Yes
There the unit tests work differently and preserve the name of the tested file so the output is always the same.
Ok, I thought there were other languages that required modules to be in files of the same name, but maybe ctags doesn't parse modules for those.
with our unit tests the source file name is a temporary name
Where does that happen? AFAICT `runner.sh` doesn't copy the source file, it just puts the tagfile in a temporary directory?
Or do you mean that by convention our test file names are chosen to reflect the test purpose (hence your choice of `simple.ml` for the first test file)? If so thats just a convention, and if it clashes with the language requirements it can be changed and your test file called `modulefoo.ml`.
Personally I would have disabled the tests not the parsing.
Where does that happen? AFAICT runner.sh doesn't copy the source file, it just puts the tagfile in a temporary directory?
Just had a look and it's actually Geany itself:
https://github.com/geany/geany/blob/df27d1b226bd7261be5482b06d3a69123ff0c514...
Note that the code is very c/c++ specific and the `includes` parameter of the function is actually source files passed on the Geany command-line. When these are really include files, it creates a new file containing `#include "file"` for all the includes which it then pre-processes with gcc.
When these aren't include files, it combines all source files into a single temporary file which it then parses. This appears to be really stupid because, first, creation of the temporary file is unnecessary, but second, worse, the concatenated file may not be syntactically valid for the given language. I'll look into this and try to fix it by parsing the files one by one and combining the tags.
Personally I would have disabled the tests not the parsing.
Then there couldn't be any tests for ocaml because the module tag is generated for every file which isn't ideal either. But as I said above, this should be fixable.
Note that the code is very c/c++ specific
To be fair the manual basically says that :stuck_out_tongue_winking_eye:
But if Geany is run with -P it doesn't use the cpp, but of course can only do one file at a time. Still that should do for ctags testing shoudn't it?
@techee pushed 3 commits.
80bc43384f6afe58a3aedd0476a990563478445d Rename variables in tag file writing functions to make things clearer c4c50051d109259f9f4fe82088f57141d722be92 Don't use temporary file when creating tag files without running preprocessor 122cc6559978e2252a861c72f083eddc7572c443 Enable module parsing for Ocaml
But if Geany is run with -P it doesn't use the cpp, but of course can only do one file at a time. Still that should do for ctags testing shoudn't it?
No, it can be all the files passed on the command-line (including glob patterns) - they would just be parsed one by one and the resulting tags combined. This is what I did in the just pushed commit. There's the small difference in the result of parsing anonymous tags described in the commit message but this isn't something we care about in global tags.
What part(s) does the no apply to?
1. -P doesn't use cpp? 2. one file at a time 3. thats enough for testing
If Geany still copies a single listed source into a temporary file thats indeed silly, so yes fixing that is good.
Ok, I guess the manual can be read that Geany makes tags for multiple files even with -P but isn't clear the files are catenated.
I see your point that for languages that are not C, catenating them may not make sense. Changing that, but still putting all the symbols in the one tag file makes sense. If the user wants catenation they can either do it themselves to a temporary, or not specify -P, let their language be C and use cpp to catenate. And the name of the variable you changed gives the game away that its intended for languages that are C only :smile:
What part(s) does the no apply to?
(2) is correct ;-) (i.e. it is still possible to get tag file for multiple source file even with `-P`). To make things clearer, for
``` geany -g file1.h file2.h file3.h ```
Geany creates `tmp.h` containing ``` #include "file1.h" #include "file2.h" #include "file3.h" ``` and makes the preprocessor perform the includes and Geany then parses the resulting file (this is valid only for C/C++, for other languages Geany always behaves as if `-P` were used). The generated header file is syntactically valid, C/C++ doesn't care about file name so there's no problem with this method.
With ``` geany -P -g file1.h file2.h file3.h ``` Geany used to concatenate the files and parse the resulting file. After this patch, Geany parses each of the files separately, combines the resulting tags, sorts them and removes duplicates. The should be the same except for the minor difference for anonymous tags described in the previous post which isn't important for global tags.
I really don't see any reason for performing the concatenation - first, there's a need for the temporary file which may be a problem for languages like Ocaml, but second, it may even lead to a syntactically invalid file for some languages. For instance Go programs start with package specification followed by imports and these are not allowed to appear later in the source. So when you concatenate two files like ``` package main
import "fmt"
func main() { fmt.Println("Hello, 世界") } ``` the result is invalid. It then depends on how the ctags parser is constructed and if it is tolerant enough to allow that - it may, it may not but we are risking problems.
@techee thanks for the clear explanation that confirms my understanding of how Geany now works and what your change does.
As I said I agree that the change to parse each file separately and combine the tags is a better solution.
I have been away from home helping friends who were flooded, but the intention is to be back home after Easter when I will get a chance to try lots of the outstanding PRs (noting I also have other stuff to catch up with as well).
Oh sure, there are even more important things in the world than the tag manager :-).
@techee pushed 1 commit.
5ec42e5054373a07bdb2507bbae53523b4bf89df fixup! Enable module parsing for Ocaml
There's both the Ocaml parser and the change not to concatenate parsed files in this PR. Should I split these into two separate PRs?
Yeah probably best to separate.
I created #3204 with the "non-concatenation" patch and will update this PR when/if it gets merged.
@techee pushed 1 commit.
cb650bdd2253226ec05670dc0596b406f5443e1b Add Ocaml ctags parser
Since #3204 is already merged, I rebased this PR on top of master.
@techee pushed 1 commit.
d25365aed87acd92065a02962142d848cd301bbf Add Ocaml ctags parser
@techee pushed 1 commit.
c509b9000633acd9c9af93f15a84a3f4e6dd5ad3 Add Ocaml ctags parser
@eht16 @b4n What do you think about including the ocaml ctags parser? It's not strictly necessary but since it's in ctags and "for free", I'd say why not.
I'll look at the details later, but sure it looks like a worthy improvement for little expanse :+1:
... or expense. ;-P
Oops :) But yay, /me learnt a word
I tested this last week and forgot to comment :).
It works fine and parses the random Ocaml code I found on the net properly. Though I don't know the language at all :D.
Merged #3163 into master.
github-comments@lists.geany.org