EnterpriseAdmins.org

I was recently asked if it would be possible for two virtual machines to be automatically assigned the same MAC addresses. Knowing that vCenter handles these assignments and ensures uniqueness, I figured it wouldn’t be possible…but it got me to thinking. Since I have multiple vCenters, what would prevent each vCenter from reusing MAC addresses? My search lead to this KB article: http://kb.vmware.com/kb/1024025 which in turn resulted in this blog post: http://enterpriseadmins.org/blog/scripting/multiple-vcenters-and-mac-address-conflicts/.

Question answered — this would not automatically happen.

A few days later, this same guy shows up with two guest names and one MAC address asking me who actually has that MAC? I check — then double check — only to confirm that both VMs have that same MAC address automatically assigned! One interesting thing to note was that each VM was in fact in a different vCenter.

Thats when I turned to some solid code I remembered seeing from mattboren on the VMware PowerCLI communities page (http://communities.vmware.com/thread/339358). A few slight modifications and I had the super fast code shown below to kick out a list of any duplicate MAC addresses:

Side note, the Client.serviceUrl property was something I found pretty quick…I wouldn’t call that a ‘best practice’ on figuring out which vCenter an object is part of. Also, the code that is removing the https and 443/sdk is a very bad approach and isn’t going to work in all scenarios. I just wanted something quick and dirty to get the info I needed.

The above code showed that the problem was much larger than just 2 virtual machines — I had nearly 50 virtual machines sharing 25 MAC addresses!

After looking into several of the virtual machines, I started seeing a pattern — a pattern I created a few months back. I had moved a bunch of virtual machines from my production vCenter a disaster recovery site vCenter (separated for SRM) on the same network. To limit the amount of downtime, the move was completed by presenting one temporary LUN to hosts in each vCenter, using storage VMotion to move the data, then during a change window the VMs were removed from inventory in production and added to the disaster recovery inventory. Virtual machines were then storage VMotioned to the proper LUNs and the temporary LUN removed. During this move, virtual machines kept their original MAC addresses, but as they had been removed from production inventory, the production vCenter was able to reissue those unused addresses.

There were also a couple examples where a similar approach was used to ‘fail back’ select virtual machines after a disaster recovery test, leaving the production vCenter having MAC addresses generated by the disaster recovery site vCenter.

I have a little more information on this issue — including details on how vCenter generates MAC addressess — that I will share in another post. Stay tuned!

It seems every release of the VMware vSphere vCenter server service has more dependency on SSL.  I always replace the default self signed SSL certificates with ones created from an internal/trusted certificate authority.  There is a good four part guide to creating the properly formatted certificates available here: http://www.virtualvcp.com/vsphere-4-technical-guides/136-replace-ssl-certificates-prepare-openssl-and-microsoft-cs

As of last count, you need to place the custom SSL certificate in 4 places to make sure you don’t see any pesky SSL warnings. Here is a listing of those paths and instructions required to make the certificate work:

vCenter Server Service (VPXD)
SSL location: C:\ProgramData\VMware\VMware VirtualCenter\SSL
Run the command: “D:\Program Files\VMware\Infrastructure\VirtualCenter Server\vpxd.exe –p” to reset the vpxd password.
*Note: This will cause all of the hosts to become disconnected and require each to be reconnected to vCenter

vSphere Web Client Server
SSL location: D:\Program Files\VMware\Infrastructure\vSphere Web Client\DMServer\config\ssl
Restart the “vSphere Web Client” Service.

VMware vCenter Update Manager
SSL location: D:\Program Files\VMware\Infrastructure\Update Manager\SSL
Find/Replace the existing server name in the D:\Program Files (x86)\VMware\Infrastructure\Update Manager\extension.xml file and replace it with your servers alias/SSL certificates common name.
Run the command: “D:\Program Files (x86)\VMware\Infrastructure\Update Manager\vciInstallUtils.exe –vc myvcenter.mydomain.com –port 80 -U myusername -P mypassword -S extension.xml -C . -L . -O extupdate”
Where myvcenter.mydomain.com is the common name/friendly name of your SSL certificate and 80 is the http port of your vCenter.

vCenter Inventory Service:
SSL location: D:\Program Files\VMware\Infrastructure\Inventory Service\ssl
Stop thevCenter Inventory Service (Note: This will also stop the dependent service VMware vSphere Profile-Driven Storage Service)
Run the command: D:\Program Files\VMware\Infrastructure\Inventory Service\scripts\register.bat myvcenter.mydomain.com 443
Where myvcenter.mydomain.com is the common name/friendly name of your SSL certificate and 443 is the https port of your vCenter.
Start the vCenter Inventory Service
Start the VMware vSphere Profile-Driven Storage Service

VMware KB article 1024025 explains an issue with MAC address conflicts due to multiple vCenter configurations with the same instance ID. Here is a technical description straight from the KB article:

Each vCenter Server system has a vCenter Server instance ID. This ID is a number between 0 and 63 that is randomly generated at installation time, but can be reconfigured after installation. 

vCenter Server uses the vCenter instance ID to generate MAC addresses and UUIDs for virtual machines. If two vCenter Server systems have the same vCenter instance ID, they might generate identical MAC addresses for virtual machines. This can cause conflicts if the virtual machines are on the same network, leading to packet loss and other problems.

The solution is to verify that all of your vCenter configurations use different instance IDs for MAC address generation. This wouldn’t be difficult if you only had two or three vCenters, but as you scale out it becomes more of a challenge. Here is some PowerCLI code that will help you check for duplicate instance IDs.

By default, the script will let you know if you have any duplicates. However, if you want to know what all instance IDs are in use, they are stored in the $myReport variable. You can type “$myReport” (without the quotes) in the console window after this script completes.

I mentioned in a previous post (VMware View inside vCloud Director) that we had to make a last minute change to the lab content on the fly. This change was to Exercise 11 that covers the Get-ESXCli cmdlet. I should mention that this is a change to ESXi — if you are using PowerCLI 5.0 to access a 4.1 host, you will need to use the 4.1 methods.

The PowerCLI 4.x lab can be found here: Demo Days 4: PowerCLI Lab
The PowerCLI 5.x lab can be found here: GCVMUG PowerCLI 5.0 Lab

After connecting to the ESXi hosts directly with connect-viserver, we get the ESXCli and assign it to a variable:

$esxcli = Get-ESXCli

The changes apply to how you access virtual machine information.

# ESXi 4.1:
$esxcli.vms.vm.list()

# ESXi 5.0:
$esxcli.vm.process.list()

As you can see the root element for the VM namespace has changed as well as the child namespace. This has been updated in the lab and should be corrected for the next show.

Earlier this month I had the privilege of helping out with a PowerCLI hands on lab at the Greater Cincinnati VMUG regional event. The content of this GCVMUG PowerCLI 5.0 lab is now available online: GCVMUG PowerCLI 5.0 Lab

A special thanks goes to the GCVMUG, Jake Robinson, Ryan Birk, Bluelock and 10zig for making this lab possible.

I was looking for a way to enable “Check and upgrade Tools during power cycling” for more than one VM at a time, but without enabling for all VM’s in the cluster.
In this case our naming convention allowed for the quick and simple use of wildcards to make selections.

From our test lab, I wanted to target only VM’s with “cel” in the name.

First, verify the targeted VM’s are all set to manual.

#check policy status
Get-VM *cel* | sort name | Select Name,@{N="UpgradePolicy";E={$_.Extensiondata.Config.Tools.toolsUpgradePolicy}}

Name                                    UpgradePolicy
----                                    -------------
NAVCEL01                             	manual
WEBCEL01                             	manual
WEBCEL02                             	manual
DEVCEL01                             	manual
DEVCEL02                             	manual
DEVCEL03                             	manual

Next, update the policy (effectively checking the box)

Now to verify the targeted VM’s are all set to upgradeAtPowerCycle

#check policy status
Get-VM *cel* | sort name | Select Name,@{N="UpgradePolicy";E={$_.Extensiondata.Config.Tools.toolsUpgradePolicy}}

Name                                    UpgradePolicy
----                                    -------------
NAVCEL01                             	upgradeAtPowerCycle
WEBCEL01                             	upgradeAtPowerCycle
WEBCEL02                             	upgradeAtPowerCycle
DEVCEL01                             	upgradeAtPowerCycle
DEVCEL02                             	upgradeAtPowerCycle
DEVCEL03                             	upgradeAtPowerCycle

In the link below, you’ll find code to update all VM’s in a cluster.

Sources/Credits: Community member LucD at communities.vmware.com/message/1601811

I recently wrote about a PowerCLI lab I helped out with at the Greater Cincinnati VMUG (GCVMUG) whole day event (you can read more here). We had originally planned to have each lab station remote desktop into a vCenter server to run each exercise. In the previous article I mentioned a slight twist in our lab delivery — the zero clients provided only supported VMware View. I’ve previously been a part of a VMware View pilot, so I knew we had a couple design choices on how to accomplish this task:

Scenario #1: Automated Pool
Needing 25 temporary and identical desktops for a training lab is a perfect use case for VMware View automated non-persistent pools using linked clones. The non-persistence of the desktops would allow the machines to be refreshed each time someone completed the lab — keeping the desktops clean. Using linked clones would reduce disk space requirements and speed up provisioning operations (since we wouldn’t need to clone the full virtual machine).

Scenario #2: Manual Pool
As the name suggests, manual pools require the provisioning operations to be completed manually. Since I needed 25 desktops, this scenario would require that I manually create (or clone) each desktop.

A manual pool may seem like an odd suggestion, since it is a lot more work than an automated pool. However, this is the route we selected. The key factor was resource availability — the environment we had available was the vCloud environment at Bluelock’s Data Center. In order to create an automated pool a target host or cluster would have been required. Since vCloud Director provided the computing resources, we would have needed to create ESXi virtual machines (nested ESXi) to run the virtual desktops. To improve performance we decided to manually deploy the virtual desktops.

From my testing, it appears that you cannot install the View Agent prior to cloning. When I tried, the cloned VMs would not register in the View Manager interface. I had attempted to use the “Guest OS Customization > Customization Script” feature in the vCloud deployment to install the agent. This did not work either, as the View Agent attempted to reboot the virtual desktop before customization finished. I’m sure I was doing something wrong — the agent should be possible to install in the base image. However, I didn’t have a lot of time, so I went with a simple approach that I knew worked. I ended up creating a batch file on the desktop of my template. After the customization was complete, I manually logged in and ran the batch file. Here are the contents of that file:

C:\Downloads\VMware-viewagent-4.6.0-366101.exe /v"/qn VDM_VC_MANAGED_AGENT=0 VDM_SERVER=viewcon01.viewlab.local VDM_SERVER_NAME=viewcon01.viewlab.local VDM_SERVER_USERNAME=VIEWLAB\vdmadmin VDM_SERVER_PASSWORD=vdmadmin"

Since the desktops were manually deployed, we decided to to place the lab content into a shared folder. We used the domain controller’s NETLOGON share since all clients would have native access. I will post another post in the future about a change that was required during the lab.

Last week I had the opportunity to present a PowerCLI self paced lab along with Jake Robinson and Ryan Birk. We presented this same lab a few months ago, but this event at the Greater Cincinnati VMware Users Group (GCVMUG) event added a slight twist. Our previous lab provided consisted of a vApp containing a vCenter and two ESXi hosts per student. Each student would then login to their vCenter using RDP. The twist in the GCVMUG event was due to the zero clients provided supported — they supported VMware View PCoIP connections only and didn’t provide an RDP option.

To work around this twist, we built a VMware View 4.6 environment inside of a vCloud Director environment at Bluelock. We went with View 4.6 because the zero clients were specifically certified with that version — the client would have likely worked with View 5.0, but we wanted to guarantee the lab would work, so we stuck with certified versions. This was an interesting twist and I learned a lot from it. Over the next few blog posts I plan to share some of the lessons learned.

The first additional requirement of this new twist was building Active Directory. VMware View leverages Active Directory authentication and authorization, so it was required to get this lab off the ground. I built a Windows 2008 R2 server and promoted it to a domain controller for the VIEWLAB.LOCAL domain. I then manually created an organizational unit for the View Lab users. I wanted to create unique user accounts for each lab station. Since we were going to have twenty stations I didn’t want to do this manually. The following code was used to create Active Directory users in the viewlab.local domain:

Each account is created in a disabled state. I could have scripted changed the userAccountControl attribute to enable the account, but I simply used the Active Directory Users and Computers interface to multi-select the users in this organizational unit and then bulk enabled them.

I have a couple other ideas for posts on this project…I hope to have them posted in the next few weeks.

Todays update includes two new sections to the general reporting section of the report. These sections are not configuration issues/problems, just additional information about your environment.

# Version 5.47- bwuch: Added memory per cluster check provided by Ed
# Version 5.46- bwuch: Added vRAM check using @LucD function

5.46 – Added @LucD’s function Get-vRAMInfo from http://www.lucd.info/2011/07/13/query-vram/. This function checks vRAM allocations against vSphere 5 licensing allocations. This check includes version all hosts — even those not running version 5.0. It is important to note the new licensing does not take effect until/unless you upgrade. This check is included for planning purposes.
5.47 – Added code provided by a reader named Ed in the comments for the vCheck 5.40 post. It shows the amount of allocated RAM per MB of physical RAM in a cluster. His comments are available here.

You can download the updated version here: vcheck5.47.ps1

When composing a View Composer desktop there are a handful of tasks that need completed. I created a script to help me keep this quick and consistent:

Please leave a comment if there are any other ideas or additions you can think of!