Backup Virtual Machines with Shared Multi-Writer Disks

There are several different use cases for using a multi-writer disk on a virtual machine. I have used these for a number of different reasons. You can take a look at my post here of how to provision a “poor man’s” cluster between two virtual machines and share information quickly between them by using the multi-writer disk. What about backing up virtual machines that are configured with a multi-writer disk? How can you successfully protect these types of workloads? Let’s take a look at how to backup virtual machines with shared multi-writer disks.

What is VMware vSphere Multi-writer flag?

To give just a bit of background on the multi-writer flag and what it technically achieves, let’s get a bit of background on the technology. VMware vSphere uses the VMFS file systems which is a “cluster-aware” file system. This means it understands multiple hosts talking to the same storage subsystem. When it comes to a single VMDK virtual disk, VMFS has the multi-writer flag disabled by default.

Typically, bad things can start to happen if two VMs try to write to the same VMDK in an uncoordinated way. However, the multi-writer flag is a VMFS option that turns on the functionality for more than one VM to write to a single VMDK. This can only be safely used if you have a “cluster-aware” application that understands how to coordinate disk I/O activity from more than one VM to the same VMDK without causing corruption.

Windows Server Failover Clustering is an example of a technology that understands how to coordinate I/O between more than on VM to the same storage in a way that does not cause data corruption. In a KB, VMware notes specific examples of applications that can make use of the multi-writer flag. These include the following:

• VMware fault tolerance
• Third-party cluster-aware applications (Example is Oracle RAC)

The following are features noted in the same KB that are supported for multi-writer disks. As you can see, snapshots, as taken by a virtual backup solution that utilizes vStorage APIs are note supported.

Actions or Features	Supported	Unsupported	Notes
Power on, off, restart virtual machine	√
Suspend VM		×
Hot add virtual disks	√		Only to existing adapters
Hot remove devices	√
Hot extend virtual disk		×
Connect and disconnect devices	√
Snapshots		×	Virtual backup solutions leverage snapshots through the vStorage APIs; for example, VMware Data Recovery, vSphere Data Protection. These are also not supported.
Snapshots of VMs with independent-persistent disks	√		Supported in vSphere 5.1 update2 and later versions
Cloning		×
Storage vMotion		×	Neither shared nor non-shared disks can be migrated using Storage vMotion due to the virtual machine stun required to initiate the storage migration.
Changed Block Tracking (CBT)		×
vSphere Flash Read Cache (vFRC)		×	Stale writes can lead to data loss and/or corruption
vMotion	√		Supported for ORAC only and limited to 8 ESX/ESXi hosts

Backup multi-writer disks

Recently, in the lab, I was looking at a backup job running on a host I had setup that was performing backups of all virtual machines and noticed it was not grabbing on of the VMs on the host. After further investigation, I noticed this was a virtual machine that I had setup with a multi-writer disk to do some other testing, sharing data between this VM and another VM, which was also not getting backed up.

Is there a way to use your backup solution to backup virtual machines that have multi-writer disks? Yes, there is. It is all in the disk mode you choose for the shared multi-writer disk.

There are three disk modes that can be chosen for a hard disk on a virtual machine. These are:

• Dependent
• Independent – Persistent
• Independent – Nonpersistent

Basically, the purpose of the disk modes is to determine how the disk will behave with a snapshot operation.

The default mode is dependent. When a snapshot is created on a virtual machine, dependent disks are included in the disks for which a snapshot is created.

Independent disks mean the disks are “independent” of a snapshot. There are two types of independent disks. Persistent means the data is persistent and written to the disk permanently. Nonpersistent means a redo log is created for the disk, however, if the snapshot is deleted or the VM is powered off, the changes are discarded.

If you don’t change the default setting when you created a multi-writer disk, the disk mode will be dependent. In this configuration, you will not be able to create backup of the VM using today’s backup solutions that make use of vStorage APIs.

Multi-writer disk with dependent mode disk will not allow you to backup your VM

By changing the multi-writer disk to Independent persistent or nonpersistent, you will be able to create snapshots of the VM. This also means that you can create backups of the VM using your backup solution.

The multi-writer disk will just be excluded by the backup solution so you will still get all the other critical disks to your VM just as your OS, and other data drives that are configured in the dependent disk mode.

Independent disk mode VMDKs will allow creating snapshots on the VM and also allows backing up your data

Thoughts

Understanding disk modes and how these play into a multi-writer configured VMs is important to understanding how to best protect your virtual machines that may need to have a multi-writer disk configured. Knowing how to backup virtual machines with shared multi-writer disks helps to ensure you do not experience total data loss by not grabbing backups at all of your VMs in production-critical environments.