2014/02/17

Veeam improves backing up VMs running on NFS

Working with Veeam Backup & Replication allows you to protect VMs in a very efficient way. Reading the data is done completely agentless and so all data is fetched via the hypervisor. For VMware, there are 3 ways of reading the data (Transport modes):
  • Direct SAN: Reading the data at the block level or in other words reading the data directly from the VMFS volumes. This is the best method as you don't consume any CPU or Memory of the production environment.
  • Hot-Add or Virtual appliance mode: Adding the VMDK of the VM you are trying to backup to a proxy VM. In this case your proxy must be a VM residing in the same cluster as the original VM. Notice that the proxy will consume CPU and Memory of the production cluster as it is a VM in the end. Most of the time this is not an issue because the backup process runs during the night.
  • NBD or Network mode: Reading the data via the vmkernel port. This is the least efficient method. However it is popular because 
    • It's super easy to setup. If the proxy can talk to the ESXi management kernel, it is setup.
    • It works good on 10GB networks
Roughly the order of picking the most efficient proxy with Veeam will be SAN > Hot-Add > NBD  for obvious reasons.

When you are looking at NFS there are only 2 options left, Hot-add and NBD. However, due to NFS v3 locking mechanisms, customers are only left with NBD as the only viable solution. What's the problem? Well if the proxy and the VM you are trying to backup are not on the same host, you might experience stuns when the VMDK is released after the backup. This is not a Veeam problem. In fact VMware has a nice kb article, you can find here:http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2010953

VMware suggest 2 solutions:
  • To work around this issue, ensure that the virtual machine resides in the same host where the backup appliance is installed.
  • Alternatively, you can use LAN/NBD transport that uses the NFC (Network File Copy) in your backup solution or disable SCSI hot-add through the backup software.
So most of the customers forced their proxies to use NBD so that they didn't have the stuns in favor of less efficient backups.

They could also make one job, tie it to one or two VMs and couple it to one proxy. Then using DRS affinities rules, you could keep the VMs and the proxy together on one host. Needless to say this approach is horrible from a management perspective

With the release of patch 3 (v7), however, Veeam has worked around this VMware issue. If you have NFS (Netapp, Nutanix, etc.) , please go to the patch page and download it now! :D

After you have installed it, you can use the following registry key:
EnableSameHostHotaddMode (DWORD)  : Intelligent load balancing can now be configured to give preference to backup proxy located on the same host

So you will need to enable the key in the registry. All our keys are stored under:
HKLM\SOFTWARE\VeeaM\Veeam Backup and Replication

Create the DWORD key there and set it to 1.

Once you have done this, you should create a proxy VM on each host in your cluster. Personally I have always felt that Windows Core Edition is perfect for this job. It installs quickly, doesn't consume any unnecessary resources and the Veeam proxy doesn't have any GUI anyway. Also, disable DRS for this VM so it stays on its corresponding host. Alternatively, install it on a local drive. Most server have a local disk that your are not using because you want your VM's to be mobile. In this case, this is what you are trying to avoid.

I also wondered, what will happen if the proxy is already busy with another task. However I got the feedback that Backup & Replication will wait forever for a proxy to become available, that is residing on the same host as long as there is a proxy online that can do Hot-add. Only if there is no Hot-add proxy or they all are offline, Veeam will switchover to NBD.