sudo rm /usr/lib/rstudio/bin/libQt*
and all was well. (NB, the specific link to these files will vary depending on the flavor of Linux you use.)
**EDIT: If you are installing this on pure Debian (I just confirmed this with a new Jessie install) you will need to install appropriate libraries too:
sudo aptitude install libqtwebkit4