Verify VAAI settings with PowerCLI

I recently had a need to validate that all hosts in a specific vCenter had appropriate vStorage API for Array Integration (VAAI) settings. I didn’t want to change the values, I was just looking for the existing values. This short code will do just that:


$myreport = @()
Get-VMHost | %{
	$DataMoverHardwareAcceleratedMove = $_ | Get-VMHostAdvancedConfiguration -Name DataMover.HardwareAcceleratedMove
	$VMFS3HardwareAcceleratedLocking = $_ | Get-VMHostAdvancedConfiguration -Name VMFS3.HardwareAcceleratedLocking
	$DataMoverHardwareAcceleratedInit = $_ | Get-VMHostAdvancedConfiguration -Name DataMover.HardwareAcceleratedInit
	$myreport += new-object psobject -property @{
		Host = $_.Name
		DataMoverHardwareAcceleratedMove = [string]$DataMoverHardwareAcceleratedMove.Values
		VMFS3HardwareAcceleratedLocking = [string]$VMFS3HardwareAcceleratedLocking.Values
		DataMoverHardwareAcceleratedInit = [string]$DataMoverHardwareAcceleratedInit.Values
	}
}
$myreport

The resulting output will list the setting for each option value for each host. Note: 0 = disabled and 1 = enabled.

If you need to make changes to these values on a large number of hosts, check out this article: http://nickapedia.com/2011/03/01/powercli-and-vaai-quick-and-dirty-script/.

Posted in Scripting, Virtualization | Leave a comment

Creating and Reviewing Windows crash dumps in VMware virtual machines

The other day I was assisting a co-worker who needed a memory dump of a Windows virtual machine. He was working with Microsoft support and the support engineer mentioned that the process for gathering this data was different since we were using a virtual machine. I Googled vmware windows memory dump and found VMware KB 1001624, which links to a Microsoft KB article that documents how to make CTRL+ScrollLock+ScrollLock cause a blue screen (which creates a memory dump). This process works the same on both physical and virtual servers.

While that is a fun setting, and has a possible April Fools use case, it doesn’t really help troubleshoot issues. Using this option, the memory dump is caused by the user and requires an outage is required (a reboot to enable the registry key and a blue screen to create the memory dump).

The tool the Microsoft engineer was talking about is vmss2core.exe and it ships with VMware Workstation. I haven’t had a chance to use it in the past, but it is very simple. Here are the steps I used to test this process:

  1. Create a snapshot of a virtual machine with the ‘snapshot the virtual machines memory’ option selected.
  2. Browse to the virtual machine’s folder on the VMFS or NFS volume and download the ‘-snapshot1.vmsn’ file to a temporary directory.
  3. Copy the ‘vmss2core.exe’ file from a VMware Workstation installation to this temporary directory.
  4. Run the following command (where ‘w2k3test01-snapshot1.vmsn’ is the name of your snapshot file):
vmss2core.exe w2k3test01-snapshot1.vmsn -W

This process creates a memory.dmp file in the current directory. This file can be analyzed to determine what was running at the time of the snapshot creation. Note: the vmsn file is slightly larger than the amount of vRAM allocated to the virtual machine and the memory dump is equal to the amount of RAM installed. For a VM with 8GB of RAM you’ll need at least 16GB of disk space to complete this process.

How do I open these memory.dmp files to verify they are working? I’m glad you asked. I found a Network World post from 2005 that outlines the process: http://www.networkworld.com/news/2005/041105-windows-crash.html. You only need to download and install two applications (note: a few dependencies exist for these apps, such as the .NET Framework).

Download and Install Debugging Tools for Windows
Download Windows Symbol Packages

These downloads are free and give you the tools to open/review a crash dump. I don’t have the detailed understanding on how to work with these files, but I wanted to make sure the memory.dmp export worked correctly before sending an 8GB file to Microsoft for analysis.

Posted in Virtualization | 2 Comments

Disable SSL for VMware Converter 5 using powershell

Starting with VMware Converter Standalone 5.0, the default action is to secure the P2V data with SSL. While this is nice if you are converting machines across an untrusted network or need the additional level of security, it can really slow down the P2V process. There is a blog post at Jon Kohler’s Blog and a thread on the VMware Communities about this subject.

I’ve been working on a handful of P2Vs, and I always need to find my notes about the NFC SSL setting to remember exactly what I need to change. While I was waiting on one of these P2Vs to complete, I opened up powershell looking for a way to modify the XML file. I now have a simple script on my management server that:

1.) Finds the right file (based off operating system)
2.) Makes a backup copy of the XML
3.) Adds the useSSL element if it does not exist
4.) Modifies the value of config.nfs.useSSL
5.) Restarts the vmware-converter-agent service

If you are interested in the script, you can download it here: VMwareConverterDisableSSL.ps1

I did notice two possible concerns that aren’t important in my environment but wanted to share them just in case. The backup XML file will be overwritten if this script is ran more than one time. This could be fixed by appending a date stamp or random number to the file name (see line 38). Second, the powershell modified version of the XML file does not preserve white space in the comments, so the resulting output is less lines than the source file. The change works as expected and the service does start, so I wasn’t worried about this difference.

Posted in Scripting, Virtualization | Leave a comment

Why does the ‘Update Tools’ button not work from within VMware Tools?

I recently upgraded an ESXi 4.1 Update 1 cluster to ESXi 5.0 Update 1 — but have not upgraded the tools yet. One of the Windows admins I work with mentioned that he noticed a development guest needed a tools upgrade, and since the server needed a reboot anyway, he decided to click the ‘Upgrade Tools’ button from within the guest OS — but nothing appeared to happen. After a few minutes he tried again, and when nothing happened he rebooted the virtual machine manually. When the same issue appeared on another virtual machine a few days later he asked me if something was going on.

After some testing of my own, I was able to recreate and confirm this issue.

If you are using tools from ESXi 4.1 on virtual machine guest running on an ESXi 5.0 Update 1 host, selecting ‘Update Tools’ from within the guest operating system will not work. This is because the running tools version does not understand the setting ‘isolation.tools.guestInitiatedUpgrade.disable’ and does not display an appropriate error message.

Here are the details:

  • Windows guest (confirmed with XP and 2003)
  • VMware Tools 8.3.7-381511 (version number 8295)
  • ESXi 5.0 Update 1 (build 623860)
  • Symptom: Clicking ‘Upgrade Tools’ from within the guest OS does not update VMware Tools. No error message or warning is displayed.
  • Selecting ‘VM > Guest > Install/Upgrade Tools…’ from within vSphere Client works as expected.
  •  
    I tried upgrading the tools to the ESXi 5.0 non-Update 1 version [VMware Tools 8.6.0-515842 (version number 8384)]. The tools were still outdated, but on a more recent version. After those tools were installed, I then attempted to upgrade the tools from within the guest and received the error message: “Update Tools failed. Edit the virtual machine’s vmx file, add the line below and try again. Please read KB article 1714 on tips for editing a vmx file. isolation.tools.guestInitiatedUpgrade.disable=FALSE.” Here is a screenshot of that error message

    Somewhere along the line, the default option was changed to prevent tools upgrades initiated from within guest operating systems. The error handling code for this error must not exist in older tools versions. If you are using tools from ESXi 4.1 on virtual machine guest running on an ESXi 5.0 Update 1 host, selecting ‘Update Tools’ from within the guest operating system will not work. This is because the running tools version does not understand the setting ‘isolation.tools.guestInitiatedUpgrade.disable’ and does not display an appropriate error message.

    Instead of updating the vmx file (as suggested by the error message — and this KB article), I would recommend breaking out some PowerCLI. Here is code that will change the guest initiated upgrade behavior back to the previous 4.x style for a virtual machine named 2k3-test01:

    
    $gIU = New-Object VMware.Vim.optionvalue
    $gIU.Key="isolation.tools.guestInitiatedUpgrade.disable"
    $gIU.Value="FALSE"
    
    #Create a Machine Config Spec using the three option values specified above
    $vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
    $vmConfigSpec.extraconfig += $gIU
    
    #Get a VM View collection of all the VMs that need to have these options
    $vms = get-view -viewtype virtualmachine |where {$_.name -eq "2k3-test01"} # remove 'where' statement to update all virtual machines
    $vms | %{ $_.ReconfigVM($vmConfigSpec) }
    
    Posted in Scripting, Virtualization | 6 Comments

    Use PowerCLI to mark disk as as SSD

    I’ve recently been working with a few new ESXi hosts. These systems have a pair of solid state disks that I planned to use for host cache. However, ESXi 5.0 did not detect these disks as an SSD type. I found a couple articles describing how to trick ESXi into thinking a data store was SSD:

    Yellow-Bricks: Swap to host cache aka swap to SSD
    VirtuallyGhetto: How to Trick ESXi 5 in seeing an SSD Datastore

    After confirming with my third party vendors that this was a supported configuration, I tested and confirmed that using the enable_ssd option would resolve my issues. Since my host build script is written in PowerCLI, I wanted to re-write the vMA commands using Get-EsxCli.

    
    $localDisk = Get-ScsiLun | where {$_.ExtensionData.DisplayName -match "Local LSI Disk"}
    $canName = $localDisk.CanonicalName
    $esxcli = Get-EsxCli
    $satp = ($esxcli.storage.nmp.device.list() | where {$_.Device -eq $canName }).StorageArrayType
    $esxcli.storage.nmp.satp.rule.add($null,$null,$null,$canname,$null,$null,$null,"enable_ssd",$null,$null,$satp,$null,$null,$null)
    $esxcli.storage.core.claiming.reclaim($canName)
    

    The local VMFS data store needs to be created first. I encountered some errors when running the commands against an unformatted/RAW device.

    Posted in Scripting, Virtualization | 2 Comments