Custom filetypes are customisations using existing filetype capabilities so no coding is needed.
Couple of problems with your attempts,
1. you create a `filetypes.awk`, but as the manual says about [custom filetypes](https://www.geany.org/manual/current/index.html#custom-filetypes) the file should be `filetypes.Xxx.conf` where Xxx is a case sensitive filetype name. The `filetypes.xxx` ones are built-in and need code support within Geany. Perhaps look at the existing custom ones such as `filetypes.Scala.conf` and see how they relate to the underlying build-in filetype, eg Scala to C. 2. you added `*.awk` to perl, which as I said elsewhere tells Geany that awk files are perl, not what you wanted. The original addition defining a new `Awk` filetype to extension mapping is the right one, but without the `Awk` matching a filename as above being right it won't work, as you found. 3. The `[groups]` stuff tells Geany where to put the custom filetype in the stuffed to the gills filetypes menu, probably needed.