{"id":2289,"date":"2025-06-07T14:42:44","date_gmt":"2025-06-07T18:42:44","guid":{"rendered":"https:\/\/enterpriseadmins.org\/blog\/?p=2289"},"modified":"2025-06-07T14:42:44","modified_gmt":"2025-06-07T18:42:44","slug":"cleaning-up-orphaned-tag-associations-in-vcenter","status":"publish","type":"post","link":"https:\/\/enterpriseadmins.org\/blog\/scripting\/cleaning-up-orphaned-tag-associations-in-vcenter\/","title":{"rendered":"Cleaning Up Orphaned Tag Associations in vCenter"},"content":{"rendered":"\n

I was recently made aware of a KB article titled “Tag associations are not removed from vCenter Server database when associated objects are removed or deleted” (https:\/\/knowledge.broadcom.com\/external\/article?articleNumber=344960<\/a>). The article includes a script that removes orphaned tag assignments left behind in the vCenter Server database after object deletion. <\/p>\n\n\n\n

Investigating the Issue<\/h2>\n\n\n\n

After reviewing this article, I checked the vpxd.log<\/code> file on one of my lab vCenter Server instances and noticed frequent entries like the following:<\/p>\n\n\n\n

2025-06-07T17:33:50.765Z error vpxd[06442] [Originator@6876 sub=Authorize opID=4be68587-f898-41b0-bbd4-2764f0941eaa Authz-7c] MoRef: vim.Datastore:datastore-4936 not found. Error: N5Vmomi5Fault21ManagedObjectNotFound9ExceptionE(Fault cause: vmodl.fault.ManagedObjectNotFound<\/code><\/pre>\n\n\n\n
To quantify this, I ran:<\/p>\n\n\n\n
cat \/var\/log\/vmware\/vpxd\/vpxd.log | grep -i vmodl.fault.ManagedObjectNotFound | wc -l\n13738\n\ncat \/var\/log\/vmware\/vpxd\/vpxd.log | wc -l\n210258<\/code><\/pre>\n\n\n\n
This showed that roughly 6.5% of the log entries were related to this specific fault, which strongly suggested lingering tag associations.<\/p>\n\n\n\n
Reproducing the Issue<\/h2>\n\n\n\n
To test further, I moved to a clean vCenter environment with no history of tag usage. I created and tagged 10 virtual machines:<\/p>\n\n\n\n
$newCat = New-TagCategory -Name 'h378-category' -Cardinality:Multiple -EntityType:VirtualMachine\n\n0..9 |%{ New-Tag -Name \"h378-tag$_\" -Category $newCat }\n\nnew-vm -VMHost test-vesx-71* -Name \"h378-vm0\" -Datastore vc3-test03-sdrs -Template template-tinycore-160-cli-cc\n1..9 | %{ new-vm -VMHost test-vesx-71* -Name \"h378-vm$_\" -Datastore vc3-test03-sdrs -Template template-tinycore-160-cli-cc }\n\nNew-TagAssignment -Tag (Get-Tag \"h378*\") -Entity (Get-VM \"h378*\")\n\nGet-VM \"h378*\" | Remove-VM -DeletePermanently:$true -Confirm:$false<\/code><\/pre>\n\n\n\n
After deletion, there were no log entries related to orphaned tags. I queried the database using a modified version of the cleanup script in read-only mode and confirmed that no orphaned tag rows existed. This led me to revisit the KB and note that:<\/p>\n\n\n\n
\n
In vSphere 7 and 8, tag associations are automatically removed for Virtual Machines and Hosts when the associated object is deleted.<\/p>\n<\/blockquote>\n\n\n\n
Confirming with Cluster Objects<\/h2>\n\n\n\n
I then repeated the test using cluster objects, which are not automatically cleaned up:<\/p>\n\n\n\n
$newCat = New-TagCategory -Name 'h378-category-Cluster' -Cardinality:Multiple -EntityType:ClusterComputeResource\n0..9 |%{ New-Tag -Name \"h378-cluster-tag$_\" -Category $newCat }\n\n0..9 |%{ New-Cluster -Name \"h378-cluster-$_\" -Location (Get-Datacenter h378-test) }\n\nNew-TagAssignment -Tag (Get-Tag \"h378-cluster*\") -Entity (Get-Cluster \"h378*\")\n\nget-cluster \"h378*\" | remove-cluster -Confirm:$false<\/code><\/pre>\n\n\n\n
Shortly after deletion, the vpxd.log<\/code> showed ManagedObjectNotFound<\/code> errors. I verified the orphaned rows using the following SQL query:<\/p>\n\n\n\n
${VMWARE_POSTGRES_BIN}\/psql -U postgres VCDB -h \/var\/run\/vpostgres <<EOF\nselect * from cis_kv_keyvalue where kv_provider like 'tagging:%'\nand\nkv_key like 'tag_association urn:vmomi:ClusterComputeResource:%'\nand\nregexp_replace(kv_key, 'tag_association urn:vmomi:ClusterComputeResource:domain-c([0-9]+).*', '\\1')::bigint\nnot in (select id from vpx_entity where type_id=3);\nEOF<\/code><\/pre>\n\n\n\n
This confirmed 100 orphaned tag associations<\/strong>, which I then cleaned up using the provided tags_delete_job_all.sh<\/code> script.<\/p>\n\n\n\n
Returning to the initial vCenter Server with ~6% of vpxd.log<\/code> entries coming from this issue, I proceeded to create a snapshot and run the same script.  It only removed about 30 orphaned associations.  However, now I’m not seeing the new vmodl.fault.ManagedObjectNotFound<\/code> entries showing up every few seconds.<\/p>\n\n\n\n
Cleanup Results<\/h2>\n\n\n\n
Back on the original vCenter Server where the log showed high volumes of these errors, I took a snapshot and ran the cleanup script. It only removed around 30 entries, but new ManagedObjectNotFound<\/code> messages have stopped appearing<\/strong>.<\/p>\n\n\n\n
This reduction is easy to monitor in Aria Operations for Logs, especially across multiple vCenter environments.<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
Conclusion<\/h2>\n\n\n\n
In my environments, VM and Host deletions are the most common, and these objects now clean up their tag associations automatically in recent vSphere versions. However, orphaned associations from cluster or other object types<\/strong> may remain, especially in environments upgraded over time.<\/p>\n\n\n\n
By reviewing your vpxd.log and using the methods shown here, you can identify and remediate these issues efficiently.<\/p>\n","protected":false},"excerpt":{"rendered":"
I was recently made aware of a KB article titled “Tag associations are not removed from vCenter Server database when associated objects are removed or deleted” (https:\/\/knowledge.broadcom.com\/external\/article?articleNumber=344960). The article includes a script that removes orphaned tag assignments left behind in … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[3,4],"tags":[],"class_list":["post-2289","post","type-post","status-publish","format-standard","hentry","category-scripting","category-virtualization"],"_links":{"self":[{"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/posts\/2289","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/comments?post=2289"}],"version-history":[{"count":3,"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/posts\/2289\/revisions"}],"predecessor-version":[{"id":2294,"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/posts\/2289\/revisions\/2294"}],"wp:attachment":[{"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/media?parent=2289"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/categories?post=2289"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/enterpriseadmins.org\/blog\/wp-json\/wp\/v2\/tags?post=2289"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}