Interesting. What OS are you running? I'm on Arch linux, and it installs to $PREFIX/lib and runs fine.
I guess ld-linux does not pick up $PREFIX/lib/x86_64-linux-gnu for non-standard $PREFIXes by default. Is there any configuration around this?
Thanks for the test!