Just to clarify, the described behavior is not implemented. To me it first read like a bug report but it is not, the current behavior is rather intended. Though it could be changed if someone wants to.
It is completely unrelated to Windows, on Linux Geany behaves the same (except that the default directory is usually $HOME and not the installation directory).
Not sure where that bug number is from, Sourceforge never went that high, maybe its pre-Sourceforge SVN. One of our [elephants]
Uh, apparently I'm an elephant. The bug number in the code corresponds to issue https://sourceforge.net/p/geany/bugs/268/. Some years ago, Sourceforge converted the long bug numbers to shorter ones, you can still search for the old IDs on the Sourceforge site and it finds the correct issues.
But still unrelated to the OP.