Offering only remove or close would prevent any LTR code to be edited in Geany just because some text in the file is RTL.
Remember its entirely valid for at least comments and strings to have RTL content in many/most programming languages.
And just testing on opening is insufficient as you demonstrated by pasting such text.
Possibly a better way would be for someone to provide a plugin to scan for such things and indicate where they occur (maybe by highlighting) so the user can ensure they only occur inside legal places.