This might solve much of your use-case, if you scroll away and don't move the cursor you can scroll back to the current line with `ctrl+shift+L`.
Otherwise you are saying you want to have a history of *some* cursor positions (since presumably you don't want every arrow key movement while you were editing, that would be useless). So you need to come up with a definition of when a cursor move is savable in history in a form that can be programmed easily and reliably.