What is VMware vSAN HCI Mesh?

If you have used VMware vSAN, you would probably agree it is one of the coolest storage technologies in the enterprise. It brings a tremendous amount of flexibility, scalability, performance, manageability, and many other benefits to your enterprise data center. With the release of VMware vSAN 7 Update 1, VMware has introduced a really great new feature called vSAN HCI Mesh which promises to deliver even more flexibility and functionality to your vSAN installations and open up even more interesting use cases. In this post, we will answer the question, what is VMware vSAN HCI Mesh to help understand this exciting new capability found in vSAN 7 Update 1.

The need behind VMware vSAN HCI Mesh

Before we can understand the value that a particular solution brings to the table in terms of new features or capabilities, we need to effectively understand what problem we are trying to solve. Before we get into looking at what VMware vSAN mesh is exactly, lets take a look at a few challenges that have been encountered by a number of organizations.

Even in HCI clusters, resource disparity can certainly take place. Certain clusters may have a high demand for compute resources. This may leave a large amount of “stranded” storage capacity. In other cases, there may be low CPU demand, however, a particular cluster may consume the available storage capacity in the cluster due to logging or some other application specific need of a workload.

Even with storage vMotion, with the workload, both compute and storage go together. What if there was a way to basically unlock stranded storage in a particular vSAN cluster and make this storage available to another vSAN cluster? With VMware vSAN Mesh found in vSAN 7 Update 1, this is now possible.

What is VMware vSAN HCI Mesh?

With the new VMware vSAN HCI Mesh capability, VMware has provided a new diaggregated approach to compute and storage resources in VMware vSAN. With VMware vSAN HCI Mesh, vSAN clusters can essentially be brought together to provide a disaggregated approach to compute and storage resources that allows multiple clusters to participate in a cross-cluster architecture.

This allows easily opening up access to stranded capacity found across your vSAN infrastructure. The beauty of the new capability of vSAN 7 Update 1 is that it does not require special hardware or some other complex configuration or added complexity. It simply allows mounting vSAN datastores from other clusters so that the storage appears native to the local cluster.

Now, you effectively have the possibility for a local vSAN cluster with excess compute to mount storage from a remote vSAN cluster that has stranded storage that can be used.

VMware has kept with the spirit of vSAN’s simplicity and elegance in management. With the new VMware vSAN HCI Mesh, vSAN clusters are allowed to remotely mount datastores from other vSAN clusters within the vCenter inventory. So, there is no additional management interface or other special UI that has to be accessed to make this new feature work. It is just simply part of the new vSAN 7 Update 1 features.

How does the new VMware vSAN HCI Mesh work? You may be thinking, isn’t this already possible with the recently new vSAN file services? Well, it is true that vSAN is now becoming increasingly more powerful in the area of native file services. With VMware vSAN 7 Update 1, VMware has now added native SMB access with the ability to tie into Active Directory permissions. This is on top of the other native file services features such as NFS and iSCSI access to existing vSAN datastores.

However, VMware vSAN HCI Mesh provides native vSAN datastore access to the remote vSAN cluster. VMware vSAN HCI Mesh works by communicating using the native vSAN protocol over existing vSAN VMkernel ports.

VMware vSAN HCI Mesh architectural overview

This leads to many advantages over mounting storage using other protocols:

1. SPBM management is preserved end to end.
2. Lower compute and IO overhead is preserved by using the native vSAN RDT protocal end to end. 
3. The vSAN performance service can allow for end to end monitoring of IO.
4. There is no need to manage islands of storage within LUNs or NFS exports, and no need for datastore clustering or VAAI to try to work around issues that would come from adding another layer of abstraction. 
5. Storage is still managed and maintained as a cluster resource.

What are the requirements and limitations?

• There will be special network considerations for the solution to work since this storage is connected via the vSAN VMkernel ports
• It can support up to 64 hosts across clusters
• A client cluster can mount up to five datastores
• You must be running vSAN enterprise or enterprise plus licensing
• It is at this time not possible to split up the VMDKs of a VM between a local cluster and a remote cluster

The following are not supported with vSAN HCI Mesh:

• Data-in-transit encryption
• Stretched Clusters
• 2-Node clusters
• Remote provisioning workflows for File Services, iSCSI, or CNS based block volume workloads (they can exist locally, but not be served remotely)
• Air-gapped vSAN networks, or clusters using multiple vSAN vmkernel ports are not supported with HCI Mesh.

VMware vSAN HCI Mesh Network Design

To make the native protocol use possible, there are many network design considerations with vSAN HCI Mesh including the following:

• The native RDT over TCP/IP that exists in traditional vSAN clusters are used so connections are made directly between the clusters in HCI Mesh
• HA and vSAN communication share the same vmkernel ports. In the event of a communication issue between the two clusters sharing compute and storage, an APD will happen in 60 seconds after an isolation event and an attempt to restart the VM will take place (e.g. 180 seconds)
• VMware recommends 25Gbps end-to-end networking using storage class gear
• VMware recommends sub millisecond latency for meshed clusters
• vDS switches is needed to allow for proper bandwidth sharing using NIOC
• L2 and L2 are supported. You will need to configure routing for vSAN vmkernel port traffic as needed

Wrapping Up

VMware vSAN HCI Mesh will certainly open up the capability needed for satisfying many different use cases. The disaggregation capabilities that vSAN HCI Mesh brings found in vSAN 7 Update 1 will allow customers to more effectively make use of stranded storage across vSAN environments. This will make it easy to bring together available resources including both compute and storage across many different clusters.