Geany 1.35, en_US.utf8 GTK 3.24.10, GLib 2.60.5
I changed the font under Edit -> Preferences -> Interface -> Fonts from "Monospace Regular" to "Inconsolata Regular".
From typing a bit after this change, it seems that all weirdness with the cursor is gone and works as expected.
This solves the issue, but still leaves me with a question: Why was the cursor so weird in the first place using the default font?