The `gettimeofday()` calls might come from checking if the file changed on disk, IIRC this happens when switching tabs with some interval. The slowiness could result from files on a network filesystem.
However, this contradicts the observation that it behaves differently with Wayland and Xorg. At least I cannot see any relation between file timestamp checking and a display server. @sej7278 can you ensure that no network filesystem (SMB, Fuse, GVFS, sshfs, ...) is involved? If so, the `gettimeofday()` calls are probably not the related.