For me with GTK+ 3.22, if I press <kbd>Alt</kbd> it underlines the mnemonics. So pressing say <kbd>Alt</kbd>+<kbd>p</kbd>+<kbd>n</kbd> works fine, even though mnemonics aren't permanently visible.
Is this what you mean, or did I misunderstand?
If not, please mention version of Geany, and GTK+/GLib (Debug->Messages), as well as operating system and desktop environment.