yyaazz's tech thoughts

In a typical NetApp & vSphere environment there are two well-known options to create a clone of a Virtual Machine. The first one is the vSphere default and the second is the NetApp rapid clone. The vSphere default clone is a simple copy operation from filesystem point of view, so if you browse the datastore you’ll see the same files in the source and target VM’s directory, except for the log files. The NetApp rapid cloning utility is doing the things in an other way, we’ll see how. But first of all, let’s talk about the configuration.

• NetApp FAS2040, 500GB NFS datastore (standard 1Gbit onboard port)
• vSphere 4.1 with NetApp Virtual Storage Console 2.0.1
• Windows 2008R2 VM, contains a ~7GB VMDK

Let’s see first how to use Rapid Cloning Utility. The wizard starts from the GUI (right-click on the source VM):

The following 3 parameters have to be selected within the wizard

• NetApp controller
• target, where the clone should be placed
• provisioning format (thick, thin, same)

The next screen is the most important part of the cloning configuration:

We have some options to fine-tune the cloning. Set the name base, numbers, increasing… The interesting part just starts when the finish button pushed on the summary screen. Let’s see what’s happening on the controller:

ntap>

Fri Mar 11 06:01:38 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : Provisioning and Cloning Capability (create) v3.1.1: vcUserId: Administrator VMware Version:4.1.0 Build: 258902 API: 4.1

Fri Mar 11 06:01:39 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : gold=null,aggrOrVol=null,proto=null,DsNames=null,vmFile=Configuration File

Fri Mar 11 06:01:39 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : gold=null,aggrOrVol=null,proto=null,DsNames=null,vmFile=[flexvol] clone1/clone1.vmdk

Fri Mar 11 06:01:39 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : vms=1,vmsPerDatastore=0,vcVersion=4.1.0,rcuGroup=ntap_rcu1299823290303

Fri Mar 11 06:01:39 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : hostInfo,version=VERSION_4_0

Fri Mar 11 06:01:42 GMT [wafl.snap.delete:info]: Snapshot copy dense_clone.0.ca2ba564-4d35-11e0-be34-00a0982880ba on volume flexvol NetApp was deleted by the Data ONTAP function dense_clone_delete_snapshot. The unique ID for this Snapshot copy is (2, 48).

Fri Mar 11 06:01:42 GMT [dense.clone.aborted:info]: Clone operation on file ‘/vol/flexvol/ntap_rcu1299823290303/0000hd0000-flat.vmdk’ was aborted successfully. The clone operation ID was 20.

OK-OK, blah-blah, but look this:

Ndmpcopy: Starting copy [ 2 ] …

Ndmpcopy: ntap: Notify: Connection established

Ndmpcopy: ntap: Notify: Connection established

Ndmpcopy: ntap: Connect: Authentication successful

Ndmpcopy: ntap: Connect: Authentication successful

Ndmpcopy: ntap: Log: DUMP: creating “/vol/flexvol/../snapshot_for_backup.4″ snapshot.

Ndmpcopy: ntap: Log: DUMP: Using Partial Volume Dump of selected subtrees

Ndmpcopy: ntap: Log: DUMP: Date of this level 0 dump: Fri Mar 11 06:01:44 2011.

Ndmpcopy: ntap: Log: DUMP: Date of last level 0 dump: the epoch.

Ndmpcopy: ntap: Log: DUMP: Dumping /vol/flexvol/clone1/clone1-flat.vmdk to NDMP connection

Ndmpcopy: ntap: Log: DUMP: mapping (Pass I)[regular files]

Ndmpcopy: ntap: Log: DUMP: Reading file names from NDMP.

Ndmpcopy: ntap: Log: DUMP: mapping (Pass II)[directories]

Ndmpcopy: ntap: Log: DUMP: estimated 7010605 KB.

Ndmpcopy: ntap: Log: DUMP: dumping (Pass III) [directories]

Ndmpcopy: ntap: Log: DUMP: dumping (Pass IV) [regular files]

Ndmpcopy: ntap: Log: RESTORE: Fri Mar 11 06:01:46 2011: Begin level 0 restore

Ndmpcopy: ntap: Log: RESTORE: Fri Mar 11 06:01:46 2011: Reading directories from the backup

Ndmpcopy: ntap: Log: RESTORE: Fri Mar 11 06:01:46 2011: Creating files and directories.

Ndmpcopy: ntap: Log: RESTORE: Fri Mar 11 06:01:46 2011: Writing data to files.

Ndmpcopy: ntap: Log: DUMP: dumping (Pass V) [ACLs]

Ndmpcopy: ntap: Log: RESTORE: Fri Mar 11 06:03:06 2011: Restoring NT ACLs.

Ndmpcopy: ntap: Log: RESTORE: RESTORE IS DONE

Ndmpcopy: ntap: Log: RESTORE: The destination path is /vol/flexvol/ntap_rcu1299823290303/.tmpNdmpDir/

Ndmpcopy: ntap: Log: DUMP: 6996887 KB

Ndmpcopy: ntap: Log: DUMP: DUMP IS DONE

Ndmpcopy: ntap: Log: DUMP: Deleting “/vol/flexvol/../snapshot_for_backup.4″ snapshot.

Ndmpcopy: ntap: Notify: restore successful

Ndmpcopy: ntap: Notify: dump successful

Ndmpcopy: Transfer successful [ 1 minutes 28 seconds ]

Ndmpcopy: Done

Fri Mar 11 06:03:40 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : Finished work, cleaning up temporary directories.

Fri Mar 11 06:03:46 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : Finished executing.

So, the rapid cloning is basically an NDMP dump/restore operation. It looks the following in vSphere:

Let’s see some interesting info.

Time

On a normal 1Gbit connection with NFS datastore (no VAAI here) the vSphere clone took 7 minutes while the RCU only 2 minutes. On the other hand, rapid cloning has the ability to create more then one clone in one step. For example 5 clone creation will look like this:

….

Ndmpcopy: ntap: Notify: restore successful

Ndmpcopy: ntap: Notify: dump successful

Ndmpcopy: Transfer successful [ 1 minutes 30 seconds ]

Ndmpcopy: Done

Fri Mar 11 06:28:42 GMT [dense.clone.finish:info]: Clone operation on file ‘/vol/flexvol/ntap_rcu1299824793374/0001hd0000-flat.vmdk’ completed successfully. The clone operation ID was 23.

Fri Mar 11 06:28:46 GMT [dense.clone.finish:info]: Clone operation on file ‘/vol/flexvol/ntap_rcu1299824793374/0002hd0000-flat.vmdk’ completed successfully. The clone operation ID was 24.

Fri Mar 11 06:28:50 GMT [dense.clone.finish:info]: Clone operation on file ‘/vol/flexvol/ntap_rcu1299824793374/0003hd0000-flat.vmdk’ completed successfully. The clone operation ID was 25.

Fri Mar 11 06:28:54 GMT [dense.clone.finish:info]: Clone operation on file ‘/vol/flexvol/ntap_rcu1299824793374/0004hd0000-flat.vmdk’ completed successfully. The clone operation ID was 26.

Fri Mar 11 06:29:54 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : Finished work, cleaning up temporary directories.

Fri Mar 11 06:29:59 GMT [app.log.info:info]: vcenter: Provisioning and Cloning Capability 3.1.1: (0)  : Finished executing.

Skipped the dump/restore part, as it’s the same as the previous example, but after the successful restore another 4 vmdk has been created. How long did it take? Around 2.5 minutes. Not that much, huh? For ten clones, around 3-3.5 minutes, and so on.

Space

At the beginning, with one VM:

ntap> df -hs

Filesystem                used      saved       %saved

/vol/flexvol/           6801MB        0MB           0%

dedupe, just for fun…

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/           5079MB     1711MB          25%

create a vSphere clone:

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             11GB     1711MB          13%

create a rapid clone:

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             18GB     1711MB           8%

one more…

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             24GB     1711MB           6%

Both clones took around 7GB above. Not surprising, fun starts here. Let’s create 5 clones:

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             31GB       28GB          47%

wow. 7GB again. Let’s create 10:

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             38GB       28GB          42%

7GB again. hmm, one more time

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             44GB       87GB          66%

OK, turn off dedupe operation, clone 10 again:

ntap> sis off /vol/flexvol

SIS for “/vol/flexvol” is disabled.

ntap> sis status /vol/flexvol

Path                           State      Status     Progress

/vol/flexvol                   Disabled   Idle       Idle for 00:54:57

ntap> df -hs /vol/flexvol/

Filesystem                used      saved       %saved

/vol/flexvol/             51GB      146GB          74%

7GB again and the %saved is growing. Summary? RCU creates deduped, space efficient clones very fast. That’s all.