Virtualizing a Windows Active Directory Domain Infrastructure

I’ve been thinking about virtualization of production domain controllers.  I have several virtualized DCs in a lab environment and even a very small production/standalone forest (less than 200 users) that only uses a single DC.  I’ve read a bunch of articles and outlined my findings below.  Please feel free to comment if you have any additional experience with this subject — I’m especially interested in large/multi-domain forest implementations.

  • Recommends x64 architecture
  • Use 1 vCPU – improves CPU scheduling flexibility
  • Use 4gb RAM – allows caching of most of the database – increase as needed for larger environments
  • Use replmon.exe to check, validate and initiate the KCC
  • Perform pre/post checks for best practices with “Best Practices Analyzer for Active Directory Domain Services” http://technet.microsoft.com/en-us/library/dd391875(WS.10).aspx
  • Control clock drift
    • VMs can easily (and fairly rapidly) drift
    • Use NTP and not VMware Tools for time sync
    • Change the PDC Emulator:
      • HKLM\System\CurrentControlSet\Services\W32Time\Parameters\Type = change from NT5DS to NTP
      • HKLM\System\CurrentControlSet\Services\W32Time\Parameters\NtpServer = change from time.windows.com,0x1 to a stratum 1 source like tock.usno.navy.mil,0x1
      • HKLM\System\CurrentControlSet\Services\W32Time\Config\AnnounceFlags = change the REG_DWORD value from 10 to 5
      • restart the w32time service (net stop w32time && net start w32time)
      • force a time sync (w32tm /resync /rediscover)
    • One clever option in VMware documentation was to set a group policy under ‘Domain Controllers’ that sets Computer configuration\Windows Settings\Administrative Templates\Windows Time Service and then use a WMI filter for PDC only:
      • Add namespace root\CIMv2
      • Add query (Select * from Win32_ComputerSystem where DomainRole = 5)
        • Roles are 0 = standalone, 1 = Member workstation, 2 = Standalone Server, 3 = Member Server, 4 = Backup domain controller, 5 = Primary domain controller
      • Link your created WMI filter to your configured group policy object (gpo)
        • Will only apply to Domain controller holding the PDC emulator FSMO role
        • Will automatically move to wherever the PDC emulator role is
  • Continue to backup the server using standard tools
  • Do not snapshot domain controllers
    • Per http://support.microsoft.com/kb/888794: Active Directory does not support other methods to roll back the contents of Active Directory. In particular, Active Directory does not support any method that restores a snapshot of the operating system or the volume the operating system resides on. This kind of method causes an update sequence number (USN) rollback. When a USN rollback occurs, the replication partners of the incorrectly restored domain controller may have inconsistent objects in their Active Directory databases. In this situation, you cannot make these objects consistent.
  • Use clean builds and dcpromo process – do not use P2V

Documentation exists from VMware on the topic http://www.vmware.com/files/pdf/Virtualizing_Windows_Active_Directory.pdf.  Several of these steps are also reinforced by Microsoft in the document titled “Considerations when hosting Active Directory domain controller in virtual hosting environments” available here: http://support.microsoft.com/kb/888794

Microsoft and VMware have a relationship through the Server Virtualization Validation Program (SVVP) which basically states that either vendor will work with the other to address issues.  As part of the troubleshooting efforts either vendor may request that the issue be created on physical hardware.

“Support policy for Microsoft software running in non-Microsoft hardware virtualization software” available here: http://support.microsoft.com/kb/897615/

“Customer Support Options for Microsoft Products Running within VMware Virtual Machines” available here: http://www.vmware.com/support/policies/ms_support_statement.html

Posted in Virtualization | Leave a comment

Script to Configure SSL on ESX/ESXi

I’ve had to create a bunch of SSL certificates for ESX/ESXi hosts. Its always a pain because there are a bunch of steps and I can’t remember what my company always uses for the Organization name/Organizational unit name. I created some documentation on the process and have to look at it every time I create a certificate. It works okay, but who wants to read documentation when you can lean on a simple batch file…like this one? You just need to specify the name of the batch file and the host name of your ESX/ESXi host from a command prompt:

[cc lang=”dos” width=”550″]
@echo off
REM ======================================================================
REM == Script to generate SSL Certificate for ESX/ESXi host ==
REM == Usage: esx_ssl.bat esxhostname.domain.name ==
REM == Note: On Windows 7 this script must be ran from an ==
REM == elevated/administrator command prompt. ==
REM ======================================================================

set pathToOpenSSLbin=C:\OpenSSL\bin
set pathTovCLIbin=C:\Program Files\VMware\VMware vSphere CLI\bin
echo The specified ESX/ESXi host: %1

REM Generate the SSL
“%pathToOpenSSLbin%\openssl.exe” genrsa 1024 > “%pathToOpenSSLbin%\%1.key”
“%pathToOpenSSLbin%\openssl.exe” req -new -key “%pathToOpenSSLbin%\%1.key” -subj “/CN=%1/OU=Department Name/O=Company Name/L=CityName/ST=State/C=US/emailAddress=user@domain.name” > “%pathToOpenSSLbin%\rui.csr”

REM Open the Certificate Signing Request in wordpad
“%ProgramFiles%\Windows NT\Accessories\wordpad.exe” “%pathToOpenSSLbin%\rui.csr”

REM The CSR contents must be manually copied to the internal cert server
echo
echo ======================================================================
echo The following steps must be manually completed
echo 1.) Select the contents of rui.csr and copy them to the clipboard
echo 2.) Open the certificate server site (https://internal-ca.domain.name/certsrv)
echo 3.) Select “Request a certificate”
echo 4.) Select “Advanced certificate Request”
echo 5.) Paste the contents of rui.csr into the saved request box
echo 6.) Select “Web Server” in the certificate template drop down
echo 7.) Save the Base 64 version of the certificate in the following path:
echo %pathToOpenSSLbin%\certnew.csr
echo ======================================================================
pause

“%pathToOpenSSLbin%\openssl.exe” x509 –in certnew.cer –out %1.cer

set /p hostpass=Please enter the root password for %1 :

“%pathTovCLIbin%\vifs.pl” –server %1 –put “%pathToOpenSSLbin%\%1.key” /host/ssl_key –username root –password %hostpass%
“%pathTovCLIbin%\vifs.pl” –server %1 –put “%pathToOpenSSLbin%\%1.cer” /host/ssl_cert –username root –password %hostpass%

REM Create a backup copy in case you need this SSL certificate again:
copy “%pathToOpenSSLbin%\%1.key” “\\vcenter\SSL_Certificates$\%1.key” /y
copy “%pathToOpenSSLbin%\%1.cer” “\\vcenter\SSL_Certificates$\%1.cer” /y

echo Please reboot the host %1 and verify the SSL certificate.
pause
[/cc]

Posted in Scripting, Virtualization | 1 Comment

vCenter Mobile Access (vCMA) tools upgrade

I recently went through and upgraded the VMware Tools on all of my virtual machines.  The only one I couldn’t get to work was my vCenter Mobile Access machine.  After some searching, I finally found this article on the VMware Communities site.  It took awhile to find, so I wanted to post the details here (so at least I can find them in the future).  Here is the source article with two takes on how to complete this task: http://communities.vmware.com/thread/216029?tstart=15

And here is a copy of the instructions for future reference:

  1. Shutdown the VCMA guest and add a “client” CD ROM to the VCMA appliance and power-on
  2. Snapshot the vCMA machine so you can undo if there are any problems!
  3. Create a directory for mount point
  4. mkdir /mnt/cdrom
  5. Select “VM->Guest->Install/Upgrade VMware Tools” from the console menu
  6. Copy the VMwareTools to the /tmp directory
  7. cp /mnt/cdrom/VMwareTools-4*.gz /tmp
  8. On the vCMA server console install the The World-Wide Web library for Perl.
  9. yum install perl-libwww-perl
  10. Extract the contents of the VMwareTools-4.0.0-164009.tar.gz file in the /tmp directory.
  11. tar zxf /tmp/VMwareTools-4.0.0-164009.tar.gz
  12. Uninstall the previous version of VMware Tools using the uninstaller from the latest version. The uninstall script wasn’t left from the initial install.
  13. /tmp/vmware-tools-distrib/bin/vmware-uninstall-tools.pl
  14. Install the latest version accepting the defaults.
  15. /tmp/vmware-tools-distrib/vmware-install.pl
  16. Reboot the vCMA server.
  17. shutdown -r now
  18. If everything works correctly remove the snapshot.
Posted in Virtualization | Leave a comment

Enabling vSwitch Cisco Discovery Protocol (CDP)

I recently attended the VMware vSphere Troubleshooting course.  It was a very good course and I’d recommend it to anyone responsible for day-to-day maintenance of a virtual infrastructure.  During the course we discussed the vMA (vSphere Management Assistant) and looked at some of the vSwitch configuration options you could set.  One option stood out to me…the CDP Mode of a standard vSwitch.  I knew my vSwitch had the ability to read CDP information, but the class text suggested it could announce its configuration.  Once I had a few minutes to get back into my lab I decided to deploy the vMA and test this configuration.

Downloading and deploying the vMA was rather straight forward.  Once the OVF template was deployed and powered on, a wizard walks you through all of the remaining configuration.  Once the vMA is only, login with the vi-admin user account.  The first step to enabling CDP requires that the host or vCenter be added to the vMA.  I decided to go with each host…so I added it like so:

sudo vifp addserver esxhostname.domain.name
vifpinit esxhostname.domain.name

To verify the existing CDP mode you can type the following command (the name of the vSwitch is case sensitive):

vicfg-vswitch -b vSwitch0

By default a vSwitch will have a CDP mode of listen, but it is very easy to change to both

vicfg-vswitch -B both vSwitch0

The physical switch now shows the following:

show cdp neighbor
Device ID:       esxhostname.domain.name
Local Intrfce:   Gig 1/3
Capability:      Switch
Platform:        VMware ES
Port ID:         vmnic0
Posted in Virtualization | 1 Comment

Linux P2V with VMware Converter Stand Alone

Over the years I have seen thousands of Windows Servers and completed hundreds of Windows P2Vs.  I only have dozens of Linux Servers and have yet to attempt a Linux P2V – until now!

To beign I found an older model physical server.  This physical server had a Dell PERC RAID controller with two RAID-1 volumes.  I installed RedHat Enterprise 4.5 using only the first RAID-1 set and accepting the default partitioning options.  Once the install was complete I added the second drive as the /data mount.  This wasn’t necessary; I just wanted to verify that additional disks would be available for cloning.

mkfs.ext3 /dev/sdb2
mkdir /data
mount /dev/sdb2 /data
vi /etc/fstab

Once the fstab was open in VI, I added the following line at the end of file:

/dev/sdb2  /data  ext3  defaults  1 2

Instructions for adding this second disk as the /data mount were obtained here: http://www.cyberciti.biz/faq/linux-disk-format/

This actual P2V process was completed using the free VMware Converter Standalone.

  1. Specify the machine source.  I’ve allowed root login over SSH, which isn’t a best security practice, but was requested by the converter utility.  Supported Linux distributions are listed on the first screen but include:
    1. Red Hat Enterprise Linux 2.1, 3.0, 4.0, 5.0
    2. Red Hat Linux Advanced Server 2.1
    3. SUSE Linux Enterprise Server 8, 9, 10
    4. Ubuntu Linux 5.x, 6.x, 7.x, 8.x
  2. Specify the vCenter or ESX/ESXi host name, user name and password.
  3. Select a host (if needed), enter information about the VM name, datastore, and hardware version.
  4. A “View/Edit Options” screen appears that allows you to reconfigure the virtual hardware.  In my ‘data to copy’ heading I see all of my volumes with their current size – and am provided the ability to shrink/grow them as needed.  I have also removed CPU/Memory to ‘right-size’ the VM configuration.
  5. The final option on the “View/Edit Options” page asks for the “Helper VM Network” information.  My first attempt to complete this P2V failed because of an invalid configuration on this step.  My test network did not have DHCP available — it turns out that the converter process boots the target VM from an ISO image to assist in the process.  This last step is where the temporary IP information is assigned to the target virtual machine for the cloning process.
  6. Here is a screenshot of the ‘helper’ VM while the cloning operation is going on:
  7. The successful log below gives a better summary of what actually happens:
  8. Once the virtual machine is powered on, a wizard starts to help you with the hardware changes.  In my build I had to press a key to begin and the first screen I saw was really straight forward:
Posted in Virtualization | Leave a comment