Windows Server 2008 / R2 Console freezes when viewed from vCenter client

Posted by Sean Duffy in IT Professional, SysAdmin, VM on 02-06-2010

Tags: , , ,

Since I started deploying more and more Windows Server 2008 R2 virtual machines I have noticed an issue that the standard VMware Tools Display Driver has with Windows Server 2008, 2008 R2 and seemingly Vista too. It seems to occur at random times; you’ll definitely know when it strikes, as your VM’s console will completely freeze up on you.

Normal network activity to and from the VM seems function normally so the underlying OS is obviously fine to continue working. You can even connect via RDP to the VM and use it that way if you wish. However lacking a console session to the VM is not exactly very useful. A restart of the VM will sort it out temporarily, but again this is not very helpful as the issue will undoubtedly occur again. Here are my tips to counter this unpleasant isssue.

The Solution

The first (and also simplest) of solutions is to uninstall the VMware SVGA driver and revert to Microsoft’s Standard VGA driver. However I am sure nobody really wants to do this as it results in sluggish console performance and a low console resolution for your vCenter view of the VM.

If you want to sort this out properly though (as I’m sure you do), try the following:

1. First off you’ll need your VM to be running the latest VMware Tools and be on VM Hardware version 7. This means you’ll probably be running a vSphere environment with ESX or ESXi 4 hosts. I wouldn’t rule out VMware Workstation 7.0 or Fusion 3.0 either as they also support VM Hardware 7 Virtual Machines, although I have not used any of these guest operating systems in either of these two additional products.

2. Start by shutting the VM down. Edit the settings of the VM and ensure its Video RAM setting is set to 8MB or higher. If you don’t perform this step you may have issues with the console of the VM being blank when you start the VM up. This happened to me the first time I installed this driver as I had not seen VMware’s requirements to use this driver! I was using a setting of 4MB of Video RAM.

Editing the settings of the VM

3. After applying the Video RAM setting, start the VM up again, login to Windows and open up the Device Manager.

4. Expand the “Display Adapters” tree item and right-click your existing Video driver (this should be called “VMware SVGA II”) – select “Update Driver Software”.

Expanding the VM Display Settings

5. Browse for the driver and navigate to the ‘C:\Program Files\Common Files\VMware\drivers\wddm_video’ folder. Complete the wizard and you should end up with a driver labelled as “VMware SVGA 3D (Microsoft Corporation – WDDM)” being installed.

If the wizard doesn’t pick this driver up try browsing for the driver again, but this time opt to “Let me pick from a list of device drivers on my computer”. Navigate to the ‘C:\Program Files\Common Files\VMware\drivers\wddm_video’ folder again and choose theĀ  “vm3d.inf” file. Finish the driver installation.

6. Restart your VM when prompted.

7. Once you are back in Windows check your Device Manager to ensure you now have the correct driver installed. It should now look like this:

Checking your Device Manager

8. Another point worth noting is that VMware state a minimum of 8MB of Video RAM is required when using this driver, however they recommend 32MB for best performance. In my case 8MB seems to have been sufficient for basic use.

Conclusion

It seems that this is known bug with VMware vSphere 4.0, and vSphere 4.0 Update 1 when using the VMware SVGA II driver. Hopefully it will be sorted out in Update 2, but for now if you are looking to resolve a troublesome console experience with your VMs, try these steps to get the WDDM driver installed.

According to VMware KB article 1016770 the WDDM driver is installed by default with Windows 7. However with Windows Server 2008, 2008 R2 and Vista, it is not installed by default but still supported.

Comments (14)

[...] This post was mentioned on Twitter by Jeff Sullivan, Sean Duffy. Sean Duffy said: Having trouble with Windows Server 2008 R2 VM console freezes on vSphere? New blog post up at SysAdmin Talk – http://bit.ly/9pg0Pz [...]

A little update –

I’ve tried the Microsoft driver – fixes the freezes but mouse performance is terrible

I’ve tried the WDDM driver (setting video memory to 32MB) – fixes the freezes but mouse performance still is not up to what I would expect

The default VMware driver has great mouse performance but for unknown reasons the console will freeze up occasionally.

Also – I’ve recently applied the vSphere U2 and there is NO change to this issue.

Just realized that the tools on the one server weren’t updated with U2! I will test and report back.

Well – updated the tools to U2. Took less than 1 hour for the video console to lock up again. The server’s still alive – RDP works, network response ok. Just no console.

Anyone at VMware working on this?

Hi Mike,

Thanks for your comments and reporting on your findings. We are now also running U2 and the same issue appears to be prevalent. I also found that setting the video memory to 8MB or 32MB doesn’t seem to help with the console mouse performance. Luckily for us this is not too much of a problem, as our 2008 R2 machines are mainly all RD Servers. It still is a pain trying to perform administration from the console though. As soon as my current backlog of work has cleared up a bit I will open a case with VMware partner support and see what they say about this issue. It was a real pain going through all our templates and updating to the WDDM driver to start with. At least it helps avoid the freezing console issue for now though. Please do post back if you find the situation changes any time soon!

Sean

I’m using Windows Server 2008 R2, I installed VMware Tools, but I don’t modify Video RAM Settings, updated drivers and it works, thanks !

did this for a 2008r2 server i was having issues with. mouse flies now as if its local.

Turns out that the defualt install did not even setup the VMware driver, it was still using standard VGA.

Might be an idea to change the mouse driver from PS/2 pointing device to the VMware pointing device – makes the world of difference for me…

Hello,

Please don’t use the WDDM driver as mentioned in this blog! It may end up in Black screens, which is terrible to revert back!!

Ismail

Hello Ismail,

I think your “Black screens” issue is being caused by your VM having insufficient video memory – you more than likely had 4MB or less on the VM that this occured on. That was the reason I put the comment in point number 8 about ensuring you have the minimum recommended amount of video RAM configured on your VM when using the WDDM driver.

“8. Another point worth noting is that VMware state a minimum of 8MB of Video RAM is required when using this driver, however they recommend 32MB for best performance. In my case 8MB seems to have been sufficient for basic use.”

If you have this set up then you shouldn’t get the issue you describe.

Sean

I don’t appear to have a C:\Program Files\Common Files\VMware\drivers\wddm_video folder

Only Virtual Printer and vmci

Any thoughts?

Hi Michael,

What is your VM’s guest OS? Is the VM running at VM Hardware version 7 and have VMware Tools installed?

Has anyone raised this issue with vmware support at all? I emailed our account rep and sys engineer and the sys engineer really advised me he didn’t have time to investigate or pass around and advised best thing to do would be to log it with vmware support…

Has anyone had an official word from vmware support?

Thanks a lot for this Article, it’s working for me.

Write a comment