Normal network activity to and from the VM seems function normally so the underlying OS is obviously fine to continue working. You can even connect via RDP to the VM and use it that way if you wish. However lacking a console session to the VM is not exactly very useful. A restart of the VM will sort it out temporarily, but again this is not very helpful as the issue will undoubtedly occur again. Here are my tips to counter this unpleasant isssue.

The Solution

The first (and also simplest) of solutions is to uninstall the VMware SVGA driver and revert to Microsoft’s Standard VGA driver. However I am sure nobody really wants to do this as it results in sluggish console performance and a low console resolution for your vCenter view of the VM.

If you want to sort this out properly though (as I’m sure you do), try the following:

1. First off you’ll need your VM to be running the latest VMware Tools and be on VM Hardware version 7. This means you’ll probably be running a vSphere environment with ESX or ESXi 4 hosts. I wouldn’t rule out VMware Workstation 7.0 or Fusion 3.0 either as they also support VM Hardware 7 Virtual Machines, although I have not used any of these guest operating systems in either of these two additional products.

2. Start by shutting the VM down. Edit the settings of the VM and ensure its Video RAM setting is set to 8MB or higher. If you don’t perform this step you may have issues with the console of the VM being blank when you start the VM up. This happened to me the first time I installed this driver as I had not seen VMware’s requirements to use this driver! I was using a setting of 4MB of Video RAM.

3. After applying the Video RAM setting, start the VM up again, login to Windows and open up the Device Manager.

4. Expand the “Display Adapters” tree item and right-click your existing Video driver (this should be called “VMware SVGA II”) – select “Update Driver Software”.

5. Browse for the driver and navigate to the ‘C:\Program Files\Common Files\VMware\drivers\wddm_video’ folder. Complete the wizard and you should end up with a driver labelled as “VMware SVGA 3D (Microsoft Corporation – WDDM)” being installed.

If the wizard doesn’t pick this driver up try browsing for the driver again, but this time opt to “Let me pick from a list of device drivers on my computer”. Navigate to the ‘C:\Program Files\Common Files\VMware\drivers\wddm_video’ folder again and choose the  “vm3d.inf” file. Finish the driver installation.

6. Restart your VM when prompted.

7. Once you are back in Windows check your Device Manager to ensure you now have the correct driver installed. It should now look like this:

8. Another point worth noting is that VMware state a minimum of 8MB of Video RAM is required when using this driver, however they recommend 32MB for best performance. In my case 8MB seems to have been sufficient for basic use.

Conclusion

It seems that this is known bug with VMware vSphere 4.0, and vSphere 4.0 Update 1 when using the VMware SVGA II driver. Hopefully it will be sorted out in Update 2, but for now if you are looking to resolve a troublesome console experience with your VMs, try these steps to get the WDDM driver installed.

According to VMware KB article 1016770 the WDDM driver is installed by default with Windows 7. However with Windows Server 2008, 2008 R2 and Vista, it is not installed by default but still supported.