Why are Additions required for using virtual machines through Remote Desktop?

I was recently asked why it is required to have Virtual Machine Additions installed when trying to control virtual machines by connecting to the host operating system with Remote Desktop (as discussed here). 

To set the terminology correctly – we have two modes of mouse operation: integrated and relative mode.  Integrated mode is the mode that we use when Virtual Machine Additions are installed, it is also the mode used by Remote Desktop.  In this mode the absolute coordinates of the mouse (x,y) are translated to appropriate coordinates for the remote / virtual computer and are sent directly.  This is the most efficient and usable mode of mouse operation – however it requires software to be running in the remote / virtual environment that can understand this sort of data input.

Using integrated mode for a virtual machine through Remote Desktop works just fine.

Relative mode is the mode that we use when there are no Virtual Machine Additions installed.  In this mode the mouse is ‘captured’ – which we do by:

1. Hiding the mouse cursor
2. Repeatedly setting the mouse cursor position to the center of our window 

Every time that we reset the mouse cursor position (this is done ~60 times a second) we record the amount of space traveled by the mouse (the delta) and send this information to the emulated PS/2 device (this is the sort of information that PS/2 mice expect).  So if you moved your mouse from (100,100) to (130,110) – we would record and send something like:

(x,y)
+3, +0
+3, +1
+4, +2
+3, +1
+2, +0
+3, +1
+3, +2
+3, +0
+3, +2
+3, +1

Now the first problem with using relative mode with Remote Desktop is that it does not support the call to set the mouse position.  If the remote / virtual computer makes this call – Remote Desktop returns success, but it does not actually change the mouse position.  This results in the delta values cumulating – so that the above numbers would end up looking like this:

(x,y)
+3, +0
+6, +1
+10, +3
+13, +4
+15, +4
+18, +5
+21, +7
+24, +7
+27, +9
+30, +10

Which means that when you tried to move from (100,100) to (130,110) you actually end up at (267, 150).  This is the behavior you see today (with an additional aggravation that I will talk about now).  Now – you might be thinking that we should just get Remote Desktop to support setting the mouse cursor position – but this won’t solve the problem either – because of the problem around latency and acceleration.  As I mentioned earlier – when we are in relative mode we sample the mouse delta very regularly.  In fact, we do it much more regularly than Remote Desktop does (especially on a slow connection).  So if Remote Desktop did support setting the mouse cursor position – the original data set would actually end up looking like:

(x,y)
+0, +0
+0, +0
+10, +3
+0, +0
+0, +0
+8, +2
+0, +0
+0, +0
+0, +0
+12, +5

Looking at this data – you might be thinking “Yeah – the mouse will be a bit jerky – but it will still work, right?”.  Well the problem here is that most modern operating systems support a concept called “mouse acceleration” where they move the mouse further if you move it faster (you can try this on your computer by moving your mouse very slowly to the right, and then by jerking it back to the left by the same amount of ‘physical desk space’, you should see that the mouse cursor on the screen goes a lot further to the left than to the right).  This means that the above data set will actually result in you overshooting your expected destination (BTW – acceleration also affects the middle data set).

So – long story short – there is no good solution for this.

Cheers,
Ben

Comments

• Anonymous
  June 06, 2007
  Why aren't you using ClipCursor to keep the mouse within the bounds of the screen and just using the actual co-ordinates of the pointer relative to the last sample position?

• Anonymous
  June 07, 2007
  Ben, What is the function of the VPC2004 DOS mouse VMADD?

• Anonymous
  June 08, 2007
  ...Or, you could TS directly to the guest and enjoy the pristine TS experience! This is how I usually do it. Of course, it's not usable during boot time / OS install / etc.

• Anonymous
  June 17, 2007
  Mike Dimmick - This is not supported over TS either - and would not solve the problem. Wesley - It is just a standard DOS mouse driver. Cheers, Ben