Enterprise Admins.org

ADAM Instance VMwareVCMSDS index may be corrupt

Posted on August 15, 2010 by brian.wuchner

VMware vSphere vCenter Server installs an Active Directory Application Mode (ADAM) [aka Active Directory Lightweight Directory Service (ADLDS)] instance called VMwareVCMSDS. I have had several problems with this service — one issue with the contents of the directory (see vCenter Migration) and a couple with the actual ADAM/ADLDS instance. In a recent upgrade I moved a lab environment from vSphere 4.0 to 4.1 and started getting the following event log message:

VMwareVCMSDS (2428) ADAMDSA: Database 'D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\adamntds.dit': The secondary index 'INDEX_00000003' of table 'datatable' may be corrupt. If there is no later event showing the index being rebuilt, then please defragment the database to rebuild the index.

The solution to this problem was rather simple — but required working directly with the ADAM instance. This took a little trial and error but here is a description of the solution in case you run into the same error:

Stop the VMwareVCMSDS service:

net stop ADAM_VMwareVCMSDS

Enter the directory service database utility application:

dsdbutil

The following commands get you to the file configuration settings:

Activate Instance VMwareVCMSDS
Files
info

Here is my test systems file configuration:

Drive Information:

C:\ NTFS (Fixed Drive  ) free(5.4 Gb) total(23.9 Gb)
D:\ NTFS (Fixed Drive  ) free(2.6 Gb) total(3.9 Gb)

DS Path Information:

Database   : D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\adamntds.dit - 4.1 Mb
Backup dir : D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\dsadata.bak
Working dir: D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS
Log dir    : D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS - 30.0 Mb total
edbres00002.jrs - 10.0 Mb
edbres00001.jrs - 10.0 Mb
edb.log - 10.0 Mb

To compact/defrag the database type the following command in the dsdbutil screen:

compact to d:\temp\defragged

Here are the results of the compact operation:

Initiating DEFRAGMENTATION mode...
Source Database: D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\adamntds.dit
Target Database: d:\temp\defragged\adamntds.dit

Defragmentation  Status (% complete)

0    10   20   30   40   50   60   70   80   90  100
|----|----|----|----|----|----|----|----|----|----|
...................................................

It is recommended that you immediately perform a full backup
of this database. If you restore a backup made before the
defragmentation, the database will be rolled back to the state
it was in at the time of that backup.

Compaction is successful. You need to:
copy "d:\temp\defragged\adamntds.dit" "D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\adamntds.dit"
and delete the old log files:
del D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\*.log

I created a zip file contianing all the files in: “D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS” before proceeding with the given instructions.

copy "d:\temp\defragged\adamntds.dit" "D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\adamntds.dit"

Answering yes to the overwrite files confirmation.

del "D:\Program Files\VMware\Infrastructure\VirtualCenter Server\VMwareVCMSDS\*.log"

After completing the defragmentation I started the VMwareVCMSDS service again but had problems authenticating back to vCenter. Instead of restarting the vCenter service I went ahead and rebooted the server — which seems to have cleared up all my errors.

Posted in Virtualization | Leave a comment

Active Directory: Find recent NO_CLIENT_SITE subnets

Posted on August 11, 2010 by brian.wuchner

From time to time I try to verify that all of my subnet boundaries are properly defined in AD Sites and Services. At some locations this is a little more challenging — especially when a site has many domain controllers. I recently threw together this PowerShell script to look at a site with over a dozen DCs:

Foreach ($server in get-content servers.txt) {Tail -10 \\$server\admin$\debug\netlogon.log | %{"$server,$_"}}

I had a file (servers.txt) that contained the name of each domain controller on a separate line. I ran the script using an Enterprise Admin account as it has permissions to the admin$ on each DC.

If you wanted to get really creative, you could add some select logic to get a list of all domain controllers. The get-date cmdlet could even compare the date format listed in the netlogon.log (MM-dd) and only give you the current days events. Throw in a scheduled task and now you could really be on top of this 🙂

Posted in Scripting | Leave a comment

Create a bunch of snapshots with PowerCLI

Posted on August 10, 2010 by brian.wuchner

Last week I was working with snapshots in my lab. I came up with this one liner to create a bunch of snapshots on the same virtual machine:

for ($i=1; $i -le 5; $i++) {get-vm "snaptest" |new-snapshot -name "snapshot00$i" -Description (get-date -format "'Created: 'yyyy-MM-dd HH:mm") -memory:$true}

As you can see I’m using the virtual machine snaptest and the snapshots will be named snapshot001-snapshot005 and have a description containing the date/time the snapshot was created.

Posted in Scripting, Virtualization | Leave a comment

Install VMware Tools in openfiler

Posted on August 8, 2010 by brian.wuchner

I have a couple of lab type vSphere setups that use Openfiler virtual machines for shared storage. For several months these have been running with out of date VMware tools. Several weeks ago I was able to upgrade these tools and wanted to share the required steps.

The first required task was to install a C Compiler — specifically gcc. While at it I updated all other components and shutdown the virtual machine:

conary updateall
conary update gcc
shutdown -h now

After the machine was powered down I added a virtual CD-ROM drive. This drive will be used to mount the VM tools ISO. After adding a CD-ROM drive the virtual machine can be powered back on. Select install/upgrade tools from the menu.

mkdir /mnt/cdrom
mount /dev/cdrom /mnt/cdrom
cp VMwareTools* /mnt
tar zxvf /tmp/VMwareTools*
cd vmware-tools-distrib
./vmware-install.pl

I accepted the defaults on the vmware-install.pl configuration script.

Special thanks to this communities article for a head start: http://communities.vmware.com/docs/DOC-9332;jsessionid=A7F5565895191AE1D161E4749AF86B12

Posted in Virtualization | 3 Comments

Check ESXi to see if sshd is listening

Posted on June 16, 2010 by brian.wuchner

In a recent service request with VMware, Tech Support enabled ssh on one of my ESXi hosts. Unfortunately, the outage that prompted my support request impacted multiple hosts in different clusters, so we were looking at many things in the entire environment. I didn’t write down the name of the host where we enabled SSH, or even the cluster it was in. The following script helped me identify which one I was looking for – I’m sure there are better ways to check this, but I was looking for something quick which prompted this combination of existing scripts:

[cc lang=”powershell”]
$VMH = Get-VMHost | Sort Name

#If the port is closed/filtered PowerShell will return a warning. This will suppress:
$ErrorActionPreference=”SilentlyContinue”

#The port I want to check for SSHD
$portnum =22

#The loop to check all hosts in a vCenter
$VMH | %{
#port check sample from http://halr9000.com/article/418
$socket = new-object Net.Sockets.TcpClient
$socket.Connect($_.name, $portnum)
if ($socket.Connected) {
$status = “Open”
$socket.Close()
}
else {
$status = “Closed / Filtered”
}
$socket = $null
write-output “$_`t$portnum`t$status”
}
[/cc]

This pointed out which of my hosts (it was only 1) that had SSH running.