Would you try the snapshot from here:
https://download.geany.org/snapshots/geany-1.38_osx-3.dmg
Basically I just rebuilt Geany with the latest version of gtk-osx with all the macOS 12 patches and it seems it fixed the problem at least on my Intel-based macbook.