Research some of the network and server outage problems experienced during past disasters, such as the 9/11 attacks or a natural disaster such as Hurricane Katrina or Hurricane Sandy.
main challenges or problems
There have been quite a few discussions, blogs, and articles on some of the challenges with deploying and running Exchange on a domain controller. In 2006, Robert Moir wrote a detailed blog titled “Running Exchange on a Domain Controller.” In his blog, he addressed, at a high level, some of the challenges with performance and security when running Exchange and Active Directory on a domain controller. Note that this article is not simply about deploying Exchange Server 2007 in Active Directory (not like what Marc Grote talks about in his article on “how Exchange Server 2007 extends the Active Directory Schema”). The distinction we are making here is that we are running Exchange on the domain controller.
The Small Business Server (SBS) was the first Microsoft product to officially deploy and support Exchange and Active Directory on a domain controller; however, due to the single domain controller nature of the product, it did not face some of the additional challenges described in this article. During the development of Essential Business Server (EBS), which deploys Exchange 2007 on a domain controller in a multiple domain controller environment, we faced a new set of challenges and developed resolutions for them. These resolutions can also be applied to any enterprise-level deployments with Exchange 2007 on a domain controller. In a mid-size and enterprise domain, it is strongly recommended to keep at least one replica domain controller around for fault tolerance and load balancing. However, due to the replication latency between domain controllers, deployment and maintenance of Exchange on a replica Domain Controller becomes more challenging. Moreover, Exchange disaster recovery becomes significantly different than recovery from a member server.
One of the main challenges we faced in running Exchange 2007 on a Domain Controller was its recovery after a disaster. The recovery of Exchange when installed on a replica Domain Controller has a special formula that is quite different from any other topology. Exchange 2007 supports the /mode:RecoverServer that will automatically deploy the server binaries and recover all server settings from Active Directory, however, the Exchange computer object must be preserved and untouched in Active Directory for Exchange recovery to succeed. On the contrary, Active Directory does not recommend “taking over an existing slot” and re-using the existing computer object when promoting a failed Domain Controller. The recommended approach in recovering a failed (or unsuccessfully demoted) Domain Controller is to run ntdsutils.exe and cleanup the leftover computer object, the related NTDS settings object, the replication channels, and all other traces of the failed Domain Controller from Active Directory. Clearly, this approach will render the Exchange server unrecoverable.
In EBS, we solve this problem by examining the state of the Active Directory after disaster, transferring and (if needed) seizing the FSMO roles, and only cleaning up the NTDS setting object of the server. This object can be found under the Sites container in the configuration partition of Active Directory. We keep the computer object untouched and join the new server to the domain with the same name as the failed server. Then we promote the new server to a Domain Controller while preserving all settings and permissions on the computer object and then attempt a recovery of Exchange. With this approach, the domain controller and Exchange server recover successfully. EBS providers this solution under the hood through its recovery process and the users need not to worry about the details.
Another challenge we faced in running Exchange on a Domain Controller was the service dependency between Active Directory and Exchange 2007 services. This dependency can cause some of the core Exchange services (such as the transport service and information Store) to not start properly after a reboot. The reason for this is that during a reboot, all of Active Directory and Exchange 2007 services on that server will be restarted. Upon restart, some of the Active Directory services such as Kerberos Domain Controller take longer to reach the running state if the domain controller is deployed in a domain with other replicas. This is probably associated to the fact that the booting Kerberos Domain Controller must complete a full replication cycle before serving clients and issuing tokens. In the meantime, Exchange Active Directory Topology services will attempt to contact the closest Kerberos Domain Controller, in this case the local host's Kerberos Domain Controller service which results in a hard-to-detect race condition. In a multi domain controller domain, it is very likely that the local Kerberos Domain Controller isn’t responsive in time for the Exchange Active Directory Topology service and unfortunately this service fails to start and gives up. Many key Exchange 2007 services such as transport and Information store depend on the Exchange Active Directory topology service and they all fail to start. Note that this problem is only visible when running Exchange on a domain controller and it is emphasized when running Exchange on a replica domain controller. There are a few ways to solve this race condition. In EBS, we chose to change all Exchange service start types from Automatic to Automatic delay start. This change allows Kerberos Domain Controller service plenty of time to come up before Exchange Active Directory Topology service and other subsequent Exchange services start querying it.
Another problem with installing Exchange on a domain controller surfaces during the installation of Exchange 2007. During installation, Exchange 2007 will first attempt to prepare the Active Directory and extend the schema. For this it will target the schema master domain controller. The schema master is one of the FSMO roles assigned to only one domain controller in the entire forest, which may even be in a remote location. After the schema has been extended, Exchange will proceed with the remainder of the installation, however, there is no guarantee that the remainder of Active Directory changes will also be targeted to the same domain controller, especially when Exchange installation is running on a local host domain controller. Due to delays in replication and depending on the proximity of the FSMO role owners to the local host domain controller, the Exchange 2007 installation can hiccup and fail with “object not found” and “no permission” errors. This is because the changes made during the previous stages of installation haven’t yet made it to the local domain controller. Note that Exchange 2007 installation is very resilient and retry-able, so almost any failure can be retried until it succeeds, however in a multi domain controller domain with scattered FSMO roles, Exchange 2007 installation on a domain controller can leave the administrator baffled with random race conditions, before it succeeds. To eliminate this problem, we (temporarily) gather all FSMO roles on the local domain controller and target the Exchange 2007 installation to the local domain controller. More specifically, after promoting the local domain controller, we ensure a full cycle of replication has completed, and the domain controller is properly advertising and responding, then we transfer the roles over to the local domain controller and ensure all the other domain controllers (at least the ones in local Active Directory site) are aware of the FSMO role ownership change, and then we install Exchange 2007 with the /DC:”local DC name.” At this point, we are completely confident that all Exchange changes will target the local domain controller and Exchange setup will complete faster (eliminating network traffic) and without any random failures.
Even though installing domain controller and Exchange 2007 might be resource intensive on the server, it somewhat helps reduce some network traffic as long as you make the domain controller a global catalog. This is because Exchange 2007 produces plenty of traffic between itself and the closest global catalog in the domain and this network traffic is eliminated if Exchange is installed on a Global catalog. Therefore, it is important to make the local domain controller a GC if you plan on installing Exchange on it.
steps
Securing Domain Controllers Against Attack:
Domain controllers provide the physical storage for the AD DS database, in addition to providing the services and data that allow enterprises to effectively manage their servers, workstations, users, and applications. If privileged access to a domain controller is obtained by a malicious user, that user can modify, corrupt, or destroy the AD DS database and, by extension, all of the systems and accounts that are managed by Active Directory.
Because domain controllers can read from and write to anything in the AD DS database, compromise of a domain controller means that your Active Directory forest can never be considered trustworthy again unless you are able to recover using a known good backup and to close the gaps that allowed the compromise in the process.
Depending on an attacker's preparation, tooling, and skill, modification or even irreparable damage to the AD DS database can be completed in minutes to hours, not days or weeks. What matters isn't how long an attacker has privileged access to Active Directory, but how much the attacker has planned for the moment when privileged access is obtained. Compromising a domain controller can provide the most expedient path to wide scale propagation of access, or the most direct path to destruction of member servers, workstations, and Active Directory. Because of this, domain controllers should be secured separately and more stringently than the general Windows infrastructure.
Physical Security for Domain Controllers
This section provides information about physically securing domain controllers, whether the domain controllers are physical or virtual machines, in datacenter locations, branch offices, and even remote locations with only basic infrastructure controls.
Datacenter Domain Controllers
Physical Domain Controllers
In datacenters, physical domain controllers should be installed in dedicated secure racks or cages that are separate from the general server population. When possible, domain controllers should be configured with Trusted Platform Module (TPM) chips and all volumes in the domain controller servers should be protected via BitLocker Drive Encryption. BitLocker generally adds performance overhead in single-digit percentages, but protects the directory against compromise even if disks are removed from the server. BitLocker can also help protect systems against attacks such as rootkits because the modification of boot files will cause the server to boot into recovery mode so that the original binaries can be loaded. If a domain controller is configured to use software RAID, serial-attached SCSI, SAN/NAS storage, or dynamic volumes, BitLocker cannot be implemented, so locally attached storage (with or without hardware RAID) should be used in domain controllers whenever possible.
Virtual Domain Controllers
If you implement virtual domain controllers, you should ensure that domain controllers run on separate physical hosts than other virtual machines in the environment. Even if you use a third-party virtualization platform, consider deploying virtual domain controllers on Hyper-V Server in Windows Server 2012 or Windows Server 2008 R2, which provides a minimal attack surface and can be managed with the domain controllers it hosts rather than being managed with the rest of the virtualization hosts. If you implement System Center Virtual Machine Manager (SCVMM) for management of your virtualization infrastructure, you can delegate administration for the physical hosts on which domain controller virtual machines reside and the domain controllers themselves to authorized administrators. You should also consider separating the storage of virtual domain controllers to prevent storage administrators from accessing the virtual machine files.
Branch Locations
Physical Domain Controllers in branches
In locations in which multiple servers reside but are not physically secured to the degree that datacenter servers are secured, physical domain controllers should be configured with TPM chips and BitLocker Drive Encryption for all server volumes. If a domain controller cannot be stored in a locked room in branch locations, you should consider deploying RODCs in those locations.
Virtual Domain Controllers in branches
Whenever possible, you should run virtual domain controllers in branch offices on separate physical hosts than the other virtual machines in the site. In branch offices in which virtual domain controllers cannot run on separate physical hosts from the rest of the virtual server population, you should implement TPM chips and BitLocker Drive Encryption on hosts on which virtual domain controllers run at minimum, and all hosts if possible. Depending on the size of the branch office and the security of the physical hosts, you should consider deploying RODCs in branch locations.
Remote Locations with Limited Space and Security
If your infrastructure includes locations in which only a single physical server can be installed, a server capable of running virtualization workloads should be installed in the remote location, and BitLocker Drive Encryption should be configured to protect all volumes in the server. One virtual machine on the server should run an RODC, with other servers running as separate virtual machines on the host. Information about planning for deployment of RODC is provided in the Read-Only Domain Controller Planning and Deployment Guide. For more information about deploying and securing virtualized domain controllers, see Running Domain Controllers in Hyper-V on the TechNet website. For more detailed guidance for hardening Hyper-V, delegating virtual machine management, and protecting virtual machines, see the Hyper-V Security Guide Solution Accelerator on the Microsoft website.
Domain Controller Operating Systems
You should run all domain controllers on the newest version of Windows Server that is supported within your organization and prioritize decommissioning of legacy operating systems in the domain controller population. By keeping your domain controllers current and eliminating legacy domain controllers, you can often take advantage of new functionality and security that may not be available in domains or forests with domain controllers running legacy operating system. Domain controllers should be freshly installed and promoted rather than upgraded from previous operating systems or server roles; that is, do not perform in-place upgrades of domain controllers or run the AD DS Installation Wizard on servers on which the operating system is not freshly installed. By implementing freshly installed domain controllers, you ensure that legacy files and settings are not inadvertently left on domain controllers, and you simplify the enforcement of consistent, secure domain controller configuration.
Secure Configuration of Domain Controllers
A number of freely available tools, some of which are installed by default in Windows, can be used to create an initial security configuration baseline for domain controllers that can subsequently be enforced by GPOs. These tools are described here.
Security Configuration Wizard
All domain controllers should be locked down upon initial build. This can be achieved using the Security Configuration Wizard that ships natively in Windows Server to configure service, registry, system, and WFAS settings on a "base build" domain controller. Settings can be saved and exported to a GPO that can be linked to the Domain Controllers OU in each domain in the forest to enforce consistent configuration of domain controllers. If your domain contains multiple versions of Windows operating systems, you can configure Windows Management Instrumentation (WMI) filters to apply GPOs only to the domain controllers running the corresponding version of the operating system.
Microsoft Security Compliance Toolkit
Microsoft Security Compliance Toolkit domain controller settings can be combined with Security Configuration Wizard settings to produce comprehensive configuration baselines for domain controllers that are deployed and enforced by GPOs deployed at the Domain Controllers OU in Active Directory.
RDP Restrictions
Group Policy Objects that link to all domain controllers OUs in a forest should be configured to allow RDP connections only from authorized users and systems (for example, jump servers). This can be achieved through a combination of user rights settings and WFAS configuration and should be implemented in GPOs so that the policy is consistently applied. If it is bypassed, the next Group Policy refresh returns the system to its proper configuration.
Patch and Configuration Management for Domain Controllers
Although it may seem counterintuitive, you should consider patching domain controllers and other critical infrastructure components separately from your general Windows infrastructure. If you leverage enterprise configuration management software for all computers in your infrastructure, compromise of the systems management software can be used to compromise or destroy all infrastructure components managed by that software. By separating patch and systems management for domain controllers from the general population, you can reduce the amount of software installed on domain controllers, in addition to tightly controlling their management.
Blocking Internet Access for Domain Controllers
One of the checks that is performed as part of an Active Directory Security Assessment is the use and configuration of Internet Explorer on domain controllers. Internet Explorer (or any other web browser) should not be used on domain controllers, but analysis of thousands of domain controllers has revealed numerous cases in which privileged users used Internet Explorer to browse the organization's intranet or the Internet.
As previously described in the "Misconfiguration" section of Avenues to Compromise, browsing the Internet (or an infected intranet) from one of the most powerful computers in a Windows infrastructure using a highly privileged account (which are the only accounts permitted to log on locally to domain controllers by default) presents an extraordinary risk to an organization's security. Whether via a drive by download or by download of malware-infected "utilities," attackers can gain access to everything they need to completely compromise or destroy the Active Directory environment.
Although Windows Server 2012, Windows Server 2008 R2, Windows Server 2008, and current versions of Internet Explorer offer a number of protections against malicious downloads, in most cases in which domain controllers and privileged accounts had been used to browse the Internet, the domain controllers were running Windows Server 2003, or protections offered by newer operating systems and browsers had been intentionally disabled.
Launching web browsers on domain controllers should be prohibited not only by policy, but by technical controls, and domain controllers should not be permitted to access the Internet. If your domain controllers need to replicate across sites, you should implement secure connections between the sites. Although detailed configuration instructions are outside the scope of this document, you can implement a number of controls to restrict the ability of domain controllers to be misused or misconfigured and subsequently compromised.
Perimeter Firewall Restrictions
Perimeter firewalls should be configured to block outbound connections from domain controllers to the Internet. Although domain controllers may need to communicate across site boundaries, perimeter firewalls can be configured to allow intersite communication by following the guidelines provided in How to configure a firewall for domains and trusts on the Microsoft Support website.
DC Firewall Configurations
As described earlier, you should use the Security Configuration Wizard to capture configuration settings for the Windows Firewall with Advanced Security on domain controllers. You should review the output of Security Configuration Wizard to ensure that the firewall configuration settings meet your organization's requirements, and then use GPOs to enforce configuration settings.
Preventing Web Browsing from Domain Controllers
You can use a combination of AppLocker configuration, "black hole" proxy configuration, and WFAS configuration to prevent domain controllers from accessing the Internet and to prevent the use of web browsers on domain controllers.
Research some of the network and server outage problems experienced during past disasters, such as the...