VMworld 2017 and Hurricanes and Future Projects

Wow has it been a busy few weeks. First wore myself out at VMworld. This was an experience. No other way to put it. I've never been bombarded with so many people, vendors, concepts and sessions in one place at the same time. This doesn't even touch on the topic of all the other user organized events (most notably for me would be the vExpert related events).

I want to thank the folks over at Veeam for sponsoring my conference pass. Without their help it would not have been possible for me to attend this amazing conference and meet all the great people I did.

Hurricane Irma hit Florida (my home state) shortly after returning from VMworld, so that kept me busy with preparations and riding out the mix of power outages and consistent internet outages, even being in North Florida where it was a Tropical Storm by time it hit us. This has given me time to begin thinking about my personal projects that I've been kicking around and debating about.

Speaking of projects, I think I'…

It has begun! VMworld 2017!

Ok, so its really like day 2.3 of VMworld. The first two days were over in a flash and I find myself here on Tuesday trying to process all that I've done so far. The whirlwind of sessions, meeting people and viewing booths kept me distracted from how sore I was undoubtedly becoming. Meeting up with Ariel Sanchez and Edgar Sanchez, being introduced to a whole gaggle of other people much smarter than I.... taking sessions that are WAY out of my current league *cough*VSAN*cough*. So far it has been an amazing experience that I hope to repeat next year!I want to shout out to Veeam and Infinio for making my trip possible. Without their sponsorship I wouldn't have been able to attend. It was awesome to meet some of the vCommunity celebrities like  Edward Haletky, Jorge Torres, Mostafa Khalil, Corey Romero and so many more that I could fill a post with just names.I had a "Meet the Experts - Virtual Volumes with Pete Flecha", the "vDocumentation" session with Ariel…

Where are you going? I'm going to VMworld!

Hello! I wanted to take a minute to thank Greg Stuart, @vDestination, Rick Vanover, @RickVanover, Kirsten Stoner (couldn't find a twitter profile), as well as the other nice people over at Veeam, @veeam, as well as Infinio, @Infinio, for making it possible to go to VMworld for the first time! I'm looking forward to meeting everyone I can at VMworld, and to be honest with you, it wouldn't be possible without the support of these great people and vendors!

I fully plan to check out both vendors at the Vendor Showcase and hope I get to learn about both of their products. After VMworld I plan to take my experience and share it with others, as well as possibly finding a way to contribute to the VMware Community, as well as the next VMworld.

Again, thanks Greg for organizing the "Win a trip to VMworld 2017" contest! Hope to see you at VMworld!

Now to begin packing! It's going to be a busy trip! :)


NetBackup 8.0 Certificates for port 8443

So we ran into an issue with our PCI Compliance scan with our NetBackup 8.0 environments. This happened on both the appliance and the standard Unix system install. The issue was related to port 8443 using a certificate in which the subject common name (CN) didn't match the name of the box (usually because it was FQDN).

The appliances started off with the subject being "CN = nb-appliance, OU = NetBackup, O = nb-appliance". This I assume is a factory default setting. The standard install actually had it as "CN = servera, OU = NetBackup, O = servera" where "servera" is a name of the system (I obviously replaced our system name with this for confidentiality reasons).

The process to correct this problem was surprisingly easy, although not publicly documented well. This ONLY replaces the port 8443 certificates. It is as follows (Please only do this while under the supervision of a Veritas Support Engineer):

Log into the appliance via SSH with a privileged ac…

VSAN Backup and Recovery Q&A

I'm just beginning to learn VSAN basics and of course my backup background immediately kicks in. I intend for this to be a living post with additional questions and answers being added in. *To clarify, this information is based on using the VDDK - HotAdd NDB portion of VADP

1) Q: Does VSAN support direct storage backup using VADP bypassing the VM Host running the data (such as how storage can be directly connected to with traditional storage)?:
A: Currently, No, since VSAN does not operate like a traditional storage array using common protocols (iSCSI, NFS or FCoE) direct connection to the storage is not possible - this answer was provided by Jase McCarty via vBrownBag Session @jasemccarty

2)Q: In a multinode VSAN environment, does the VMDKs transfer/copy to the host running the guest VM?
A: No, the VMDK's do not tranfer/follow the guest from one host to another, BUT some data would be cached on the host running the Guest. This probably would not be a full set of info, but primar…

CommVault VADP Backups - Single file restores gotcha Part Deux

I wanted to take a second to clarify a couple of parts that could be misleading or easily misunderstood.

First, the "all or nothing" part of the restore mentioned refers ONLY to VMDK restores, not to single file restores. I included the comment in the post as this was part of the discussion as an option that I was provided by CommVault Support today.
Second, the suggestion to use either the Linux Restore appliance or the VMDK restore method was recommended when restoring a total size >100GB or a very large number of files (think 1,000+ or so).
Third, the total size of free space required is an amount equal to or greater than the TOTAL size of data to be restored, and needs to be available on both the staging disk (where the agent is installed by default) as well as the final destination location. An example discussed was if I wanted to restore a 10GB file to the D:\ drive, then by default, you would need 10GB+ available on both the D:\ (final destination) as well as 10GB+…

CommVault VADP Backups - Single file restores gotcha

Ran in to this little issue today when performing single file restores from a CommVault 11 VMware VADP based backup:

When running a single file restore from a VADP backup to an Agent based client recovery you have to be careful. The restore will queue up all data to be recovered to a temporary folder on the destination/proxy client, which by default is in the CommVault Client install path (defaults on C:\Program Files\CommVault).

This means that if the TOTAL size of the restore is 200GB, then you MUST have 200+GB of FREE SPACE on the volume. It doesn't matter if it's 200 files 1GB in size, or 1 200GB file, you must have enough free space to buffer the ENTIRE restore.

The work around I had to utilize today was to run multiple smaller restores that didn't exceed the free space on the drive that CommVault was installed.

I know, you can load the Linux Appliance and use that as the proxy, or you could create a VMDK and mount it that way... There are other options, but this assu…

Backups, Snapshots and Arrays, oh my!

There seems to be some confusion about what qualifies as backups and what doesn't. I'd like to take a minute to clear this up.

Backup: According the, a backup would be defined as "a copy of computer data"... emphasis on the word "copy"

First, let me cover what snapshots are in almost every system that uses them. A snapshot goes through certain steps:
1) Snapshot is requested
2) System (be it virtualization system, OS or array henceforth referred to collectively as "system") quiesces the data to commit any buffered data
3) Data is locked by the system (now referred to as source)
4) Change log is created (referred to a "new data")
5) As data changes are requested, pointers are created that reference the source with the new data updated. As more data is updated, the change log can grow up to the size of the original source data
6a) New snapshot (multiple snapshot config): change log is frozen like the original source data and n…

NetBackup Virtualization Resource Limits

Alright everyone, I wanted to share another little gem that isn't really well known in NetBackup:
Resource Limits

This is available for both VMware and Hyper-V, and allows you to limit backups based on various options to prevent you from thrashing your virtualization farms. Sorry Xen, you're still not supported yet. You get to it from Host Properties -> Master Server -> Properties -> Resource Limit

First, let's look at the options available for the 800-lbs gorilla in the room, VMware:

vCenter, snapshot, Cluster, ESXserver, VMXDatastore, Datastore, DatastoreFolder, DatastoreType, VMXDatastoreNFSHost, DatastoreNFSHost and DatastoreCluster. Sounds promising right?

Let's go down the list:
vCenter - Maximum jobs that can run per vCenter - good if you have a small vCenter Server
snapshot - Maximum snapshot creates/deletes that can run at the same time - will not limit backups, only snapshot operations
Cluster - Limits jobs running on a vSphere Cluster
ESXserver - Limits …

NetBackup VMware Backups - to SWAP or not to SWAP

In our configuration of NetBackup, we came across the option to "Exclude swapping and paging files" when performing VMware Backups. This sounds good right? Less to backup (making backup windows smaller) as it'd all be recreated on reboot anyways?

Caveat emptor!

First, excluding the swap doesn't exclude the files, merely the data within.

Second, depending on your version, if you have multiple swap files configured in Windows, it may only exclude the first and not the remaining, per article:

Third, and what I find to be the most critical, when backing up Linux VMs with this option enabled (which I think it is by default), upon restore the /swap partition isn't created and/or the swap is disabled. This will require you to manually re-enable or recreate the swap partition. This is stated in the following knowledge base article:

Unless you are backing…