Yeah, that's the usual behaviour, it would be annoying if created the file, for example if you made or a typo or decided you didn't want to save the file, you'd have to go into terminal/file manager and manually delete it.
I think this issue could probably be closed as you can do what the title says already.