March 25th, 2015 by JFrappier

Jonathan Frappier Virtxpert

Some time around the release of vSphere 5.5 (Update 2 maybe?) VMware officially(?) didn’t not support vCenter on a Windows Failover Cluster. I say didn’t not support because there still seems to be very limited documentation and KB’s on how to do this. The VMware vCenter Server Availability Guide documents available options such as using HA for vCenter availability, but also how to install vCenter on a Windows Failover Cluster, and configure the services appropriately since the application itself is not other cluster aware, for example like installing SQL on a failover cluster.

If you have done a failover cluster on Windows before, the process is a bit different so don’t just dive in as I did. So what does my environment look like:

  • SSO has been already deployed and working
  • A management vCenter is running; you will need this or some other means to clone the first virtual machine after installation

So wait why are you clustering vCenter if there is already a vCenter you ask? Many reasons, but primarily our availability of our management vCenter is less of a concern. The clustered vCenter is being deployed to support vRealize Automation so end users will rely on this vCenter to be able to request catalog items. Availability was more of a concern for this purpose than strictly management.

  • Start with only a single Window 2012 R2 64-bit virtual machine (not 2) as you will later clone this virtual machine to act as the 2nd node
  • I placed the original, and clone on two separate physical hosts
  • Each virtual machine has a single 60GB (C) drive for the OS
  • 2 additional volumes will be added which, in my case, are XtremIO volumes presented as a physical RDM. This should also work using in-guest iSCSI for example
  • 1 of the 2 additional volumes is a 60GB (D) drive which vCenter will be installed on and the other a quorum disk for the failover cluster
  • Each virtual machine has two NICs – one for production/client access the other for cluster communication
  • The Windows Failover Cluster will have an IP address, as well as the vCenter Service role which you will create; in total this is 6 IP address
  • An AD account was created for the vCenter services, added to the local administrators group and given permission on the SQL server as required

A few notes before I review the process;

  • If you are using RDMs, make sure you read this KB to mark the RDMs as perennially claimed otherwise storage rescans and boot times will be drastically affected (hosts were taking roughly an hour to boot)
  • The directions have you install the vCenter Web Client, Inventory Service, and vCenter services to the D drive. There is a known bug that causes the web client to not function properly when installed to a non-default location (though it seems more that it doesn’t work when not installed to the C drive). You’ll need this KB article which walks you through creating a symbolic link, after implementing this the web client operated as expected. Also, once installation is complete and working on the primary node, you’ll need to failover to the secondary node to create the sym link (well at least I did, would it let you create a sym link to a drive that didn’t exist? hmmm)

So, with that out of the way there is a few things to define before you bring up your first virtual machine – specifically the names and IP addresses of both virtual machines, the Window cluster, and the vCenter cluster. For example:

jQuery(document).ready(function() {jQuery(‘#table_95684235’).dataTable( {“bPaginate”: false,”bLengthChange”: false,”bFilter”: false,”bSort”: false,”bInfo”: false,”bStateSave”: true,”bAutoWidth”: true,”sPaginationType”: “full_numbers”,”oLanguage”: {
“sLengthMenu”: “Display _MENU_ records per page”,
“sZeroRecords”: “Nothing found – sorry”,
“sInfo”: “Showing _START_ to _END_ of _TOTAL_ records”,
“sInfoEmpty”: “Showing 0 to 0 of 0 records”,
“sSearch”: “Search: “,
“sInfoFiltered”: “(filtered from _MAX_ total records)”
}} );});

Purpose Name IP
vCenter Cluster VC2
Windows Cluster VC2Win
Primary vCenter Node VC2-1
Secondary vCenter Node VC2-2


This is important, and I misinterpreted this step the first time I did this: When you create the first virtual machine – give it the name and IP address of what will ultimately be the vCenter cluster – using the example above you will name the computer VC2, with an IP address of and join it to your domain. After the initial install this will be changed.

Create the virtual machine, with 2 NICs and the RDMs. Mount one of the RDMs as D and one as whatever letter makes you happy, for my OCD that would be Q for quorum. Create your system DSN as you normally would, log in as your vCenter service account and perform a custom installation (not simple), installing each of the components to the D drive. During the installation process note that the name being added to SSO is the name that will ultimately be the vCenter cluster.

Before removing the RDMs, make sure to note their original file name, volume ID, and SCSI controller; they need to be added back in the same order.

These steps are pretty straight forward in the guide, change all of the vCenter services to manual, shutdown the virtual machine, remove the RDMs, and make a clone of the virtual machine. One item not clear was when to re-add the RDMs, I chose to play it safe and kept them out of the virtual machine for now. Once the clone is complete, power on the cloned virtual machine and rename it to the secondary vCenter node hostname and IP address. Power on the original virtual machine, unjoin it from the domain, rename and IP it with the hostname for the primary vCenter node, and rejoin the domain. Now you can power off the virtual machines, re-add the RDMs to the primary node, then the secondary as you typically would, making sure the SCSI controller is set to physical sharing.

Power on the virtual machines and install the Failover Cluster feature on each. Once complete, create a new cluster on the primary node – during the creation you will be asked for a cluster name and IP address – use the Windows Cluster name (VC2Win) from the example above – this is NOT the vCenter cluster name and IP address which you used on the initial virtual machine during installation. Unlike with the SQL post I wrote, you can add all available cluster storage as both additional drives are used for the cluster (D – App, Q – Quorum). Now that the cluster has been created, you should have an AD object called VC2Win. Using option #2 from this MSDN blog post, create your vCenter cluster AD object. Failing to do this will cause the cluster to fail when you attempt to start it.

The rest of the steps for creating the vCenter cluster role are well documented with one caveat, so rather than copy paste them here finish reading the VMware vCenter Server Availability Guide. That caveat, because your vCenter services were set to manual, and thus not started after the reboots, when you create the initial vCenter role service it will come us as failed – which made me go  ZOMG not again! This message is actually just the status of the clustered service, which is stopped, thus failed from a Windows Failover cluster perspective – it is okay to proceed with creating the remaining services and setting the dependencies.

At this point, you should be able to start the cluster and have all services come up.

vCenter services on Windows Failover Cluster

vCenter services on Windows Failover Cluster

Once it is up, access the web client and set permissions as required. For example, as you can see in this screenshot, here is both vCenters in the web client after since my account was given the appropriate permissions to both.

vCenter on a Windows Failover Cluster

vCenter on a Windows Failover Cluster

The last item I have to tackle is automating the backup, copy, and restore of the ADAM database. There are a lot of words in the doc which basically says – xcopy the backup to the correct location. The document talks about stopping/starting services before placing the file. But if the services aren’t running on VC2-2, I should just be able to drop it in. Now when the services start there is an up to date file which will get loaded.

So, quick a dirty like…

del d:backup*.* /Q
%windir%system32dsdbutil.exe “ac i VMwareVCMSDS” ifm “create full D:backup” q q
xcopy /osy d:backupadamntds.dit “VC2-2C$ProgramDataVMwareVmware VirtualCenterVMwareVCMSDS”

VMware vCenter on Windows 2012 Failover Cluster

Posted in Tech Tagged with: , , , , , , , , , , , , , , , , , , ,

February 23rd, 2015 by JFrappier

Jonathan Frappier Virtxpert

I wanted to share some of the example Ansible playbooks used during last Wednesday’s US #vBrownBag. During the show I went over examples of how you can use Ansible to create, clone, and update virtual machines in vCenter without the need for other provisioning tools. Based on my testing (and I’m still learning as well), the items noted in the comments are the bare minimum needed to run the playbook, even though the official documentation may currently state otherwise. If you are already using Ansible for configuration management, this is a handy option to have as you can perform the provisioning tasks without leaving Ansible.

All playbooks have been uploaded to my GitHub Ansible-Test-Playbooks repository (

#vBrownBag Using Ansible with vCenter Examples

Posted in Tech Tagged with: , , , , , , , , , , , , , , , , , ,

February 10th, 2015 by JFrappier

Jonathan Frappier Virtxpert

Scenario: You try to install the VMware vCenter Server Appliance (VCSA) or Platform Services Controller but receive an error during the installation. After correcting the problem during installation you attempt to re-install the appliance but receive the following error message:

Virtual Machine Already Exists

Virtual Machine Already Exists

As of the release candidate of vSphere 6.0, the vCenter Server Appliance installation wizard does not clean up deployed virtual machines after failed deployments. Virtual Machines deployed are still present on the selected ESXi hosts inventory. Log into the ESXi host, power off, and delete the virtual machine from the failed deployment.


If you attempt to redploy the virtual machine with a different name (appliance and host name) using the same IP address you receive the following error message:

Encountered an internal error. see /var/log/firstboot/vmafd-firstboot.py_6399_stderr.log

Because the virtual machine was deployed and powered on, there is a duplicate IP address on the network.

Error during deployment of vCenter Server Appliance or Platform Services Controller following error

Posted in Tech Tagged with: , , , , , , , , , , , , , , , , , , , , ,

February 9th, 2015 by JFrappier

Jonathan Frappier Virtxpert

Generally, installing virtual appliances has been pretty straight forward – import an OVA and enter the necessary details in the deployment wizard, or access the virtual appliances management interface (such as those typically on port 5480 from VMware). However, as of the Release Candidate for VMware vSphere 6.0, the vCenter Server Appliance (VCSA) installation takes a much different approach than what you’ve been used to.

A few vCenter Server Appliance prerequisites

First, it should be noted that you can only install the vCenter Server Appliance (VCSA) from Windows. I was first turned onto the VCSA because I was at an all OSX/Linux shop so it made sense to use something we were accustomed to using already. For now, you’ll need a Windows box to at least get the appliance deployed;  then you can punt (please note also this is based on Release Candidate (RC) code and could change in the final release).

You CAN deploy the VCSA 6.0 to both ESXi 5.5 or 6.0 host. If you currently have a 5.5 environment you can deploy the VCSA without upgrading your hosts, but if you did not take  the plunge into 5.5 you’ll have to bring at least one host online running 5.5. or 6.0.

Finally, before getting started, you MUST create DNS records before running the installer. I was struggling with the new installer because I’ve just been used to doing my DNS records after I deployed the VCSA, but before running the setup through the management interface. However with a little help from Emad Younis (@Emad_Younis) I was able to point me in the right direction. With 6.0 all of the configuration is done from the initial setup wizard. When it’s finished installing, vCenter is ready to run.

The installation wizard will NOT give you an error if this does not exist, instead it will fail during the installation!

As you can see here I have my forward and reverse DNS records ready to go on .9


Installing the vCenter Server Appliance

As with the older versions of the VCSA, it all starts with a download; however in this case you will be downloading an ISO image. Once the ISO image is downloaded either mount the ISO on your Windows box or extract the ISOs into a folder (as seen here).


Now that you have access to the files, drill down into the vcsa folder, there you will find the VMware-ClientIntegrationPlugin-6.0.0. Install this application on your Windows box (double click, Next, Accept/Next, Next, Install, Finish). Once the plugin finishes installing, back up one folder level and open the index file. As you can see here I am on Windows Server 2012, thus at least IE10 however opening the index in IE10 gives me a warning that I need to upgrade to at least IE10 or 11, so yea I’m going with Chrome. As with any plugin, you must enable it in Chrome. Click on the puzzle piece with the red x, then click Always allow and refresh the page and click the Allow button.


You should now see the vCenter icon along with a large Install button, click on it. You will get a UI very similar to what you would get deploying a virtual appliance.

vmware-vcsa-6 -installer

1.  After carefully reading the license agreement, printing it for your records, and having it signed by an attorney, click the I accept… check box and click Next.

2.  Now you can chose to deploy to your target server. Specify your ESXi host (5.5 or above!), username and password – now click Next.

If you are using self signed/untrusted certificates click Yes when prompted.

3.  The next step is to name your appliance. In my case, like I have created in DNS, my appliance name will be vxprt-vc02.vxprt.local. Click Next

4.  On the deployment type you can chose to install an embedded Platform Services Controller (which includes Single Sign-On in vSphere 6.0), just the the PSC, or just vCenter. You can have multiple Platform Services Controllers, and they can be different types. For example you could do a stand-alone PSC and have an embedded one with the VCSA. When the installer says “embedded” it really just means the components will be installed on the same virtual appliance as vCenter. I’ll be doing embedded here. Click Next

5.  Chose whether you have an existing SSO domain or you will be creating a new one. I will do this install as a greenfield type deployment, so select Configure Single Sign-On. Now enter the administrator password, and domain. To stay consistent with what I know about SSO, I’ll enter vsphere.local here. Click Next.

VMware vCenter Server Appliance (VCSA) step 5 - configure SSO

VMware vCenter Server Appliance (VCSA) step 5 – configure SSO

6.  Select the appliance size that supports your environment, including the new “tiny” deployment for up to 20 hosts. Click Next

7.  Select the datastore you will to install to, and whether to THIN PROVISION the vmdk (no VMware, I’m not calling it “Thin Disk Mode” – THIN PROVISION!). Click Next

8.  If you’re an Oracle shop, you have a choice on step 8, otherwise just click Next.

9.  Chose a network (this will be based on the host you deployed to), and how to assign IP information including the host name – This MUST match DNS. I’ll select static as that is what I would want to do for this type of server. Finally enter the NTP server and click next (I’ve also enabled SSH so I can connect directly to the virtual machine.

VMware vCenter Server Appliance (VCSA) installation - Network Settings

VMware vCenter Server Appliance (VCSA) installation – Network Settings

10.  Review the settings you’ve enter, make sure your IP information and host name are all correct and click Finish. The installation of vCenter and the VCSA will start. You’ll even see it installing packages, that’s right this is a ground up build, not just a bunch of packages pre-installed on a virtual machine!

VMware vCenter Server Appliance (VCSA) installation process

VMware vCenter Server Appliance (VCSA) installation process

Once the installation is complete, you can connect to https://fqdn/vsphere-client (no more 9443! One less question on the VCP6 I guess :) ).

vmware-vcsa-6-installation-completeLog in as the [email protected] you configured during the installation.


So far on the release candidate I’ve had trouble deploying to a port group on a VDS (it gives errors almost immediately) even though it appears as a valid port group on the network settings page. It would be nice if VMware added more validation on the various steps to ensure there will be no errors during the installation. If you do run into an error, you need to re-run the installation wizard.

Installing the VMware vCenter Server Appliance 6.0 VCSA

Posted in Tech Tagged with: , , , , , , , , , , , , , , , , , , , , , , ,

February 9th, 2015 by JFrappier

Jonathan Frappier Virtxpert

During the installation of of the VMware vCenter Server Appliance (VCSA) 6.0 or the Platform Services Controller (PSC) Appliance 6.0, you receive the following message:

Firstboot script execution Error.


The supplied System Name [name] is not valid

VMware vCenter Server Appliance (VCSA) and Platform Services Controller (PSC) error during installation – supplied system name is not valid

VMware vCenter Server Appliance (VCSA) and Platform Services Controller (PSC) error during installation - supplied system name is not valid

VMware vCenter Server Appliance (VCSA) and Platform Services Controller (PSC) error during installation – supplied system name is not valid

Additionally, logs found at %USERPROFILE%AppDataRoamingVMwarevSpherevcsasessionssession_####logs do not provide additional details, only

2015-02-06 22:41:09.330738 Progress Controller: [VCSA ERROR] – First Boot error

This problem is likely due to incorrect DNS configuration, either in the DNS server IP address provided during the VCSA or PSC installation or there is no matching DNS record.

Verify that both forward and reverse DNS lookup zones exist and re-run the installation, validating that DNS is working. Below is an example of running nslookup FQDN. The first when the record doesn’t exist, the 2nd after it has been added. Ensure you resolve the expected IP address from NSLOOKUP and re-run the installer.



Error: Supplied System Name is not valid during vCenter Server Appliance 6 installation

Posted in Tech Tagged with: , , , , , , , , , , , , , , , , , ,