@LarsGit223 basically when you play with the selection as a matter of having the "current element", using the cursor instead is the basically same and IMO makes more sense as the selection can possibly allow multiple rows and such. But anyway, it doesn't really matter, I just like it better, it shouldn't change anything -- and I tested, and it doesn't magically fix the issue.
However, I did one experiment: if I remove the GtkTreeModelFilter, the bug disappears. Obviously we can't filter anymore, but the selection/cursor doesn't jump around anymore. I'll try and investigate whether it's a mistake on our end (like confusing filtered and unfiltered stores, although I kind of doubt it as it happens when the filter doesn't do anything), or if it's indeed a bug in there. Unless someone else beats me to it ;)