The mouse wheel would look broken if scrolling stopped when control is accidentally pressed.
I'd argue that it's just as true with any incorrect usage that doesn't do what you wanted. For a more unreasonable example, if I hit <kbd>Del</kbd> when I meant to write "hello", I wouldn't blame the software but myself.
I get that it's annoying the zoom changes by mistake, but is it truly a problem for you to re-scroll without control pressed?
@elextr hum… sounds hacky, but might happen to work, I don't know.