Content Library item validation with Power CLI

Have you ever had a new site come online and want to validate that the items in a subscriber content library are in sync? Or experience some storage issues/corruption and want to make sure your sync’d library is still intact? This post will use a PowerCLI script to quickly validate items in a content library.

$contentLibraryName = 'h114-sub'
$testHost = 'lab-esxi-02.example.org'
$testDatastore = 'local-hdd'

foreach ($clitem in ( Get-ContentLibraryItem -ContentLibrary $contentLibraryName -ItemType ovf | Sort-Object Name ) ) {
  "Starting deploy of item $($clitem.name)..."
  $thisNewVM = $clitem | New-VM -Name "Test_$($clitem.Name)" -VMHost $testHost -Datastore $testDatastore
  if ($thisNewVM) {
    "  New VM created. Will attempt power on."
    [void]($thisNewVM | Get-NetworkAdapter | Set-NetworkAdapter -StartConnected:$false -confirm:$false)
    $thisVMstarted = $thisNewVM | Start-VM | Wait-Tools
    if ($thisVMstarted) {
      "  Startup success, doing cleanup."
      [void]($thisVMstarted | Stop-VM -Confirm:$false)
      [void]($thisVMstarted | Remove-VM -DeletePermanently -Confirm:$false)
    } else {
      "  There was an issue with a test VM for content library item $($clitem.Name).  Cleanup not completed."
    }
  } else {
    write-warning "  Failure occured creating new VM based on $($clitem.Name)!"
  }
}

This script has three variables that must be defined — the name of the subscriber content library, a test host in the site/environment you wish to test, and a datastore from that test host to use for provisioning.

The script will retrieve a list of OVF templates, deploy a new VM from each using the creative name Test + the name of the content library item. Once the item is deployed, the VM will be powered on and wait for tools to start. If Tools start successfully, we can likely assume that the VM is in working order, so the script proceeds to power off/delete the test VM, otherwise it will throw a warning to the screen and continue.

During testing I encountered/created a few issues which I’ll describe below.

The first was caused by an ’empty’ VM which didn’t have an OS installed. It failed with an error that VMware Tools was not installed. The script didn’t delete this VM, but after reviewing an confirming this was expected behavior for the VM, I deleted it anyway.

Wait-Tools : 8/21/2023 6:53:45 AM       Wait-Tools              The specified VM 'Test_emptyVM-notools' does not have VMware Tools
installed.
At line:7 char:46
+     $thisVMstarted = $thisNewVM | Start-VM | Wait-Tools
+                                              ~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Test_emptyVM-notools:VirtualMachineImpl) [Wait-Tools], ViError
    + FullyQualifiedErrorId : Client20_VmGuestServiceImpl_WaitTools_ToolsNotInstalledOrVersionIncorrect,VMware.VimAuto
   mation.ViCore.Cmdlets.Commands.WaitTools

To contrive the next error, I ssh’d to the test host, found the content library item on the filesystem, and intentionally corrupted the VMDK by running echo blah > emptyVM-notools-1_248c256e-cab1-4512-b7a0-48322ad4a7bf-flat.vmdk. This resulted in the following error (read below the next error for the resolution to this):

New-VM : 8/21/2023 6:15:29 AM   New-VM          Transfer failed: Error during transfer of
ds:///vmfs/volumes/639245e5-399bba0d-e1e8-000c2927a169//Test_emptyVM-notools/Test_emptyVM-notools.vmdk: The virtual
disk is either corrupted or not a supported format..
At line:3 char:26
+ ... = $clitem | New-VM -Name "Test_$($clitem.Name)" -VMHost $testHost -Da ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-VM], ViError
    + FullyQualifiedErrorId : ViCore_VMServiceImpl_DeployFromLibraryItem_ViNetException,VMware.VimAutomation.ViCore.Cm
   dlets.Commands.NewVM

After capturing this error, I deleted the files associated with the content library item. I had assumed that these missing files would trigger the content library to re-sync the content. Instead, I received the following error:

New-VM : 8/21/2023 6:22:32 AM   New-VM          Transfer failed: Error during transfer of ds:///vmfs/volumes/639245e5-399bba0d-e1e8-000c2927a169//contentlib-21c270a1-11
7e-4982-a014-ac122fdb74eb/088a872b-1098-441b-807a-251f76402991/emptyVM-notools-1_248c256e-cab1-4512-b7a0-48322ad4a7bf.vmdk?serverId=bad2b868-9a1e-4d68-ba7e-26
0f397cc0ca: Failed to resolve source datastore file URI(ds:///vmfs/volumes/639245e5-399bba0d-e1e8-000c2927a169//contentlib-21c270a1-117e-4982-a014-ac122fdb74e
b/088a872b-1098-441b-807a-251f76402991/emptyVM-notools-1_248c256e-cab1-4512-b7a0-48322ad4a7bf.vmdk?serverId=bad2b868-9a1e-4d68-ba7e-260f397cc0ca).
At line:3 char:26
+ ... = $clitem | New-VM -Name "Test_$($clitem.Name)" -VMHost $testHost -Da ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-VM], ViError
    + FullyQualifiedErrorId : ViCore_VMServiceImpl_DeployFromLibraryItem_ViNetException,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewVM

To address the previous two items, I found the ‘bad’ content library item in the web interface, right clicked and selected delete item content. I had expected this operation to fail, as I assumed that content could only be deleted from the publisher repo and not the subscriber repo. However, this command was successful, removing the ‘size’ property of the item from the content library as if it had never sync’d. Rerunning the script to deploy this content library item caused the item to replicate and the deployment was successful.

Hopefully this script can help you validate content library items if needed. If you have any suggestions on how to improve this script, please leave a comment.