Posts

Showing posts from 2017

Pre-VMUG GNV Update

So I'll admit, I've been slackng. No updates. Been busy with various projects not B&R or virtualization related. Had a failure in my homelab with my synology 1815+ locking up followed by the blinking power light. Synology is repairing it but told me once they receive it then it's a 7-10 business day leadtime. hope its faster. I did receive a few upgrades for my lab once the nas is back. Future plans (at least pre-vmug metting) will be to upgrade my backup systems in my lab and hopefully come up with a decent outline for a backup design virtualization session. Waiting on some community edition products to come out for testing and design considerations. We shall see. And now, onto the meeting! Charles @whitehattechs

VMworld 2017 and Hurricanes and Future Projects

Wow has it been a busy few weeks. First wore myself out at VMworld. This was an experience. No other way to put it. I've never been bombarded with so many people, vendors, concepts and sessions in one place at the same time. This doesn't even touch on the topic of all the other user organized events (most notably for me would be the vExpert related events). I want to thank the folks over at Veeam for sponsoring my conference pass. Without their help it would not have been possible for me to attend this amazing conference and meet all the great people I did. Hurricane Irma hit Florida (my home state) shortly after returning from VMworld, so that kept me busy with preparations and riding out the mix of power outages and consistent internet outages, even being in North Florida where it was a Tropical Storm by time it hit us. This has given me time to begin thinking about my personal projects that I've been kicking around and debating about. Speaking of projects, I think

It has begun! VMworld 2017!

Ok, so its really like day 2.3 of VMworld. The first two days were over in a flash and I find myself here on Tuesday trying to process all that I've done so far. The whirlwind of sessions, meeting people and viewing booths kept me distracted from how sore I was undoubtedly becoming. Meeting up with Ariel Sanchez and Edgar Sanchez, being introduced to a whole gaggle of other people much smarter than I.... taking sessions that are WAY out of my current league *cough*VSAN*cough*. So far it has been an amazing experience that I hope to repeat next year! I want to shout out to Veeam and Infinio for making my trip possible. Without their sponsorship I wouldn't have been able to attend. It was awesome to meet some of the vCommunity celebrities like  Edward Haletky, Jorge Torres, Mostafa Khalil, Corey Romero and so many more that I could fill a post with just names. I had a "Meet the Experts - Virtual Volumes with Pete Flecha", the "vDocumentation" session with Ar

Where are you going? I'm going to VMworld!

Hello! I wanted to take a minute to thank Greg Stuart,  @ vDestination , Rick Vanover,  @ RickVanover , Kirsten Stoner (couldn't find a twitter profile), as well as the other nice people over at Veeam,  @ veeam , as well as Infinio,  @ Infinio , for making it possible to go to VMworld for the first time! I'm looking forward to meeting everyone I can at VMworld, and to be honest with you, it wouldn't be possible without the support of these great people and vendors! I fully plan to check out both vendors at the Vendor Showcase and hope I get to learn about both of their products. After VMworld I plan to take my experience and share it with others, as well as possibly finding a way to contribute to the VMware Community, as well as the next VMworld. Again, thanks Greg for organizing the "Win a trip to VMworld 2017" contest! Hope to see you at VMworld! Now to begin packing! It's going to be a busy trip! :) Charles @whitehattechs

NetBackup 8.0 Certificates for port 8443

So we ran into an issue with our PCI Compliance scan with our NetBackup 8.0 environments. This happened on both the appliance and the standard Unix system install. The issue was related to port 8443 using a certificate in which the subject common name (CN) didn't match the name of the box (usually because it was FQDN). The appliances started off with the subject being "CN = nb-appliance, OU = NetBackup, O = nb-appliance". This I assume is a factory default setting. The standard install actually had it as "CN = servera, OU = NetBackup, O = servera" where "servera" is a name of the system (I obviously replaced our system name with this for confidentiality reasons). The process to correct this problem was surprisingly easy, although not publicly documented well. This ONLY replaces the port 8443 certificates. It is as follows (Please only do this while under the supervision of a Veritas Support Engineer): Log into the appliance via SSH with a privi

VSAN Backup and Recovery Q&A

I'm just beginning to learn VSAN basics and of course my backup background immediately kicks in. I intend for this to be a living post with additional questions and answers being added in. *To clarify, this information is based on using the VDDK - HotAdd NDB portion of VADP 1) Q: Does VSAN support direct storage backup using VADP bypassing the VM Host running the data (such as how storage can be directly connected to with traditional storage)?: A: Currently, No , since VSAN does not operate like a traditional storage array using common protocols (iSCSI, NFS or FCoE) direct connection to the storage is not possible - this answer was provided by Jase McCarty via vBrownBag Session @jasemccarty 2)   Q: In a multinode VSAN environment, does the VMDKs transfer/copy to the host running the guest VM? A: No, the VMDK's do not tranfer/follow the guest from one host to another, BUT some data would be cached on the host running the Guest. This probably would not be a full set of in

CommVault VADP Backups - Single file restores gotcha Part Deux

I wanted to take a second to clarify a couple of parts that could be misleading or easily misunderstood. First, the "all or nothing" part of the restore mentioned refers ONLY to VMDK restores, not to single file restores. I included the comment in the post as this was part of the discussion as an option that I was provided by CommVault Support today. Second, the suggestion to use either the Linux Restore appliance or the VMDK restore method was recommended when restoring a total size >100GB or a very large number of files (think 1,000+ or so). Third, the total size of free space required is an amount equal to or greater than the TOTAL size of data to be restored, and needs to be available on both the staging disk (where the agent is installed by default) as well as the final destination location. An example discussed was if I wanted to restore a 10GB file to the D:\ drive, then by default, you would need 10GB+ available on both the D:\ (final destination) as we

CommVault VADP Backups - Single file restores gotcha

Ran in to this little issue today when performing single file restores from a CommVault 11 VMware VADP based backup: When running a single file restore from a VADP backup to an Agent based client recovery you have to be careful. The restore will queue up all data to be recovered to a temporary folder on the destination/proxy client, which by default is in the CommVault Client install path (defaults on C:\Program Files\CommVault). This means that if the TOTAL size of the restore is 200GB, then you MUST have 200+GB of FREE SPACE on the volume. It doesn't matter if it's 200 files 1GB in size, or 1 200GB file, you must have enough free space to buffer the ENTIRE restore. The work around I had to utilize today was to run multiple smaller restores that didn't exceed the free space on the drive that CommVault was installed. I know, you can load the Linux Appliance and use that as the proxy, or you could create a VMDK and mount it that way... There are other options, but th

Backups, Snapshots and Arrays, oh my!

There seems to be some confusion about what qualifies as backups and what doesn't. I'd like to take a minute to clear this up. Backup : According the Merriam-Webster.com, a backup would be defined as "a copy of computer data"... emphasis on the word " copy " First, let me cover what snapshots are in almost every system that uses them. A snapshot goes through certain steps: 1 ) Snapshot is requested 2 ) System (be it virtualization system, OS or array henceforth referred to collectively as "system") quiesces the data to commit any buffered data 3 ) Data is locked by the system (now referred to as source) 4 ) Change log is created (referred to a "new data") 5 ) As data changes are requested, pointers are created that reference the source with the new data updated. As more data is updated, the change log can grow up to the size of the original source data 6a ) New snapshot (multiple snapshot config): change log is frozen like the o

NetBackup Virtualization Resource Limits

Image
Alright everyone, I wanted to share another little gem that isn't really well known in NetBackup: Resource Limits This is available for both VMware and Hyper-V, and allows you to limit backups based on various options to prevent you from thrashing your virtualization farms. Sorry Xen, you're still not supported yet. You get to it from Host Properties -> Master Server -> Properties -> Resource Limit First, let's look at the options available for the 800-lbs gorilla in the room, VMware: vCenter, snapshot, Cluster, ESXserver, VMXDatastore, Datastore, DatastoreFolder, DatastoreType, VMXDatastoreNFSHost, DatastoreNFSHost and DatastoreCluster. Sounds promising right? Let's go down the list : vCenter - Maximum jobs that can run per vCenter - good if you have a small vCenter Server snapshot - Maximum snapshot creates/deletes that can run at the same time - will not limit backups, only snapshot operations Cluster  - Limits jobs running on a vSphere Cl

NetBackup VMware Backups - to SWAP or not to SWAP

In our configuration of NetBackup, we came across the option to "Exclude swapping and paging files" when performing VMware Backups. This sounds good right? Less to backup (making backup windows smaller) as it'd all be recreated on reboot anyways? Caveat emptor! First , excluding the swap doesn't exclude the files, merely the data within. Second , depending on your version, if you have multiple swap files configured in Windows, it may only exclude the first and not the remaining, per article: https://www.veritas.com/support/en_US/article.000025484 Third , and what I find to be the most critical, when backing up Linux VMs with this option enabled (which I think it is by default), upon restore the /swap partition isn't created and/or the swap is disabled. This will require you to manually re-enable or recreate the swap partition. This is stated in the following knowledge base article: https://www.veritas.com/support/en_US/article.000023097 Conclusion : U

3rd Party VADP Backup selection criteria

As a Backup Administrator, I try to look for ways to more efficiently manage my time and resources, as well as automating what I can. In our environment we have 3 big name backup solutions that we support due to various reasons, and many of our clients use dedicated backup agents on the boxes. VMware based backups that were done used manual VM selection criteria due to various reasons. To me, this seemed a bit wasteful. When my VMware Admin approached me informing me of a plan to migrate some hosts and upgrade VMware I figured this was a good time to approach the idea of automating the VM selection method. We sat down and looked at our 3 backup software solutions to find what we could use. Names, Tags, Datastores, Folders. As we looked we realized that while all our solutions supported VADP to some extent, some support some methods, some would have to be upgraded to support methods and some wouldn't support other methods. As a spoiler, Tags, while nice, was the least supported

Adventures in setting up NetBackup 8.0 VMware Web Client Plugin files

Image
So I was working on deploying my NetBackup 8.0 into my home lab when it hit me: "Hey! Why don't you go ahead and load the Web Client plugin for NetBackup into vCenter! Then you can see all your VMware backups right from there AND be able to run restores!" *Author's comments: If you want to skip the details and get to the answer skip down to the "problem and solution" section towards the bottom "Good idea!" I thought to myself. Little did I realize this would lead me down the path of around 6hrs of digging through web pages and scratching my head. My setup is as follows: VMware VCSA 6.5, 2 PowerEdge 1950's, Synology 1815+ NAS, Windows 2016 DC+DNS Server, Windows 2016 Certificate Authority Server (setup for other reasons), Windows 2012 R2 NetBackup Server. Followed Veritas' guide to setup the web plugin which says you need a web server (thought my Synology would have worked). Run the jar file as instructed in the guide: ht

OVA Deployment failures with vSphere 6.5

Image
OVA Deployment issues with vSphere 6.5 Ran into this lovely little problem when I was attempting to deploy an OVA file to my lab using the WebConsole. This happened both with Chrome and MS Edge: Now, I know what you're thinking: "Did you follow the link? Did you install the certificates?" The answer is "YES!". Installed the certificates from the vCenter host and even one of the ESXi hosts. Even closed and re-opened my browser windows as well as rebooting the vCenter host. No Joy! After searching for various combinations of errors I finally stumbled upon the vSphere 6.5 Release notes: http://pubs.vmware.com/Release_Notes/en/vsphere/65/vsphere-esxi-vcenter-server-65-release-notes.html Digging through this I came across these two messages: ==================================================================== Deploying an OVF or OVA template from a local file with delta disks in the vSphere Web Client might fail When you deploy an OVF t