Interesting. I see the same problem with the official releases but recently I made new binaries with the latest gtk-osx here and these don't seem to exhibit the problem:
https://download.geany.org/snapshots/geany-1.38_osx_arm64.dmg https://download.geany.org/snapshots/geany-1.38_osx.dmg
These binaries use newer version of GTK with some macOS 13 fixes so maybe this is the reason.