Keybindings are a linear search through the list, whilst adding each additional one is immaterial, if each one isn't actively challenged then they keep adding until keyboard operation becomes slow.
Just a note, _every_ key goes through the search, and its actually keys that are not keybindings that take the longest since they search the whole list exhaustively.