vmmouse for Linux VMWare guest

I decided to give Ubuntu a try to see what all the fuss is about. But I wasn't ready to install it on my laptop (I'm in the middle of a project in which I rely quite a bit on the laptop), so I decided to try Ubuntu in VMWare server on my desktop. Ubuntu installed OK, but the mouse didn't work well: I had to click in the VMWare window to make the mouse work in Ubuntu, and then I had to press Ctrl-Alt for VMWare to release the mouse (to use it anywhere outside the VMWare window). It was doing this even after I installed VMWare-tools.

That's a real drag, all the more so since that doesn't happen when running Windows XP in VMWare (the mouse 'just works': you can just roll the cursor in and out of the VMWare window and it works as it should).

The trick is to install the 'vmmouse' driver (which comes with VMWare-tools) in X.org in the Linux guest. (This solution comes mostly from a post by 'zaroff' on the Ubuntu Forums.)

After installing Linux in VMWare server, click VM->Install VMWare Tools... on the VMWare menu. This makes Linux think you've just mounted a CD with a couple of files on it (the 'CD' will probably show up on the desktop). Unpack the .tar.gz file, cd into the vmware-tools-distrib directory, and run the vmware-install.pl installer.

When I did this, I found that when the installer re-wrote /etc/X11/xorg.conf, it didn't put in a DefaultDepth directive in the "Screen" section, and I had to add a DefaultDepth 24 line to that section.

Next you need to install the vmmouse driver. A good start is to run the following inside the vmware-tools-distrib directory:

find . -type f -name 'vmmouse*'

You need to copy the correct vmmouse driver (depending on what version of X.org your Linux guest is running) into the X.org input modules directory. For an Ubuntu 6.10 Linux guest, I needed to copy the XOrg/7.0/vmmouse_drv.so to /usr/lib/xorg/modules/input. (A friend was having the same trouble running a CentOS 4.4 guest: he needed to copy XOrg/6.8.x/vmmouse_drv.o to /usr/X11R6/lib/modules/input/.)

Next you need to make 2 changes to /etc/X11/xorg.conf:

1. add Load "vmmouse" to the "Module" section
2. change Driver "mouse" to Driver "vmmouse" in the "InputDevice" section

Then restart X (or reboot).