Windows 10 VM 99 percent disk utilization performance issue

In troubleshooting a recent performance issue in my home lab environment on a couple of Windows 10VMs that I had running, I had a peculiar issue that arose.  In looking at task manager when troubleshooting severe slowdowns in the OS, I noticed the disk i/o usage as portrayed by task manager was bouncing between 99 and 100% I/O utilization.  The peculiar thing is that the i/o shown to be used only amounted to a few megabytes a second – nothing that should bring the VM to a crawl.  Yet, Windows was at a standstill and performance was terrrible.  Also, disturbing was the fact these VMs were backed by an ultra fast Samsung Evo 960 NVMe drive.  I wanted to write up a quick post for you guys on some troubleshooting I went through with Windows 10 VM 99 percent disk utilization performance issue and a couple of steps in the end that seem to have resolved the issue for me.

Below is a screen clip of an instance of 99% disk utilization.  Notice, this is sorted by the largest to smallest offenders.  As you can see the odd thing is that usage as far as we are being shown is minimal, yet 99%.

Windows 10 VM 99 percent disk utilization performance issue

As mentioned briefly above, I had a Windows 10 Enterprise VM (version 1607) loaded on an ESXi 6.5 hostwith a Samsung EVO 960 NVMe drive as a datastore.  The VM was mildly configured with 100GB of space and 6 GB of RAM.  I had bumped up the memory just to see if I could affect the issue, however, it only resulted in temporary perceived improvement.  The issue quickly returned.  Most of the Googling around on this issue led me to a handful of proposed fixes that were said to work for various ones.  A summary of the reasons include:

• Windows Defender
• Superfetch service
• Windows Search service
• Google Chrome and interaction with Skype
• Microsoft KB – SSDs and Message Signaled Interrupt

The last issue peaked my curiosity in that it surely fit my scenario.  I was running on an SSD datastore and the driver loaded fit the driver detailed in the KB article.  However, this and all the other articles surrounded physical Windows 10 installations and no mention of virtual machines. So this for me was an interesting issue and trying the various proposed solutions to see if any applied in this scenario.

Things Tried

I went through the proposed list of troubleshooting.  I started with the services that many said were causing their Windows 10 disk I/O utilization performance issues.  The results of my testing with my VM:

• Windows Defender – After disabling via group policy, no effect on the issue.
• Superfetch service – After disabling and rebooting, no effect on the issue.
• Windows Search service – Again, no effect.
• Google Chrome and interaction with Skype – This didn’t apply to me.
• Microsoft KB – SSDs and Message Signaled Interrupt – Interestingly, this “seemed” to help.  After a reboot, I didn’t see the 99% usage for a long while.  However, it did return.

Microsoft KB

The Microsoft KB referenced above, did seem to affect my issue or I perceived it to at least.  The steps involve according to MSFT:

1. Under IDE ATA/ATAPI Controllers right-click on the AHCI controller node and select Properties. This node is usually called “Standard SATA AHCI Controller.”
2. Navigate to the driver tab and click Driver Details.
3. If you see “StorAHCI.sys” in the list, you are running the inbox driver.

The driver was indeed present with my installation.

Navigating in the registry to the affected device, you should have a Device Parameters >> Interrupt Management >> MessageSignaledInterruptProperties and then MSISupported REG_DWORD.

Change the value from 1 to 0 to turn this off.  Then rebooted.

 Windows 10 1703 and VMware Optimization Tool

After the above troubleshooting, the VMs still were not performing as well as I had hoped.  Even after disabling the registry key, I still saw the disk I/O utilization peaking up to 99% although maybe perceived, it didn’t seem to be doing this as often or for as long, but still didn’t seem normal.

My next thought here was to upgrade to Windows 10 1703 Creator’s update from Microsoft.  I simply ran the interactive upgrade on the VM by mounting the ISO.

After the upgrade process was complete, I verified that the VM was now running version 1703, which it was.

One other additional step I took since this was a VM was to run the VMware OS Optimization Tool.  This tool is normally run with VMware Horizon installations, however, I wanted to run the tool to benefit from the numerous performance tweaks to provide additional performance benefits.

Results?

Interestingly, I can say now that after running the Windows 10 1703 Creator’s update on the problematic VM, it has no longer presented with the 99-100% disk I/O utilization.  However, it is still a bit early on after running the update.  I will keep you guys posted on any findings with 1703 and this issue if it is still ongoing.  However, so far so good.

Thoughts

Troubleshooting this Windows 10 VM 99 percent disk utilization performance issue has been an interesting bit of trial and error.  What I can’t say definitively is if this was a problem for me with 1607 or if 1703 simply corrected an issue with the VM installation itself.  As mentioned above, I will update this post with any findings.