When you are planning site structure, it is important that you understand how replication works. As discussed previously, Active Directory uses two replication models; each of which is handled differently. The intrasite replication model is used for replication within sites and is optimized for high bandwidth connections. The intersite replication model is used for replication between sites and is optimized for limited bandwidth connections. Before I get into the specifics of replication and the replication models, let's look at the way replication has changed between Microsoft Windows 2000 and Windows Server 2003.
The replication model used for Windows Server 2003 has changed in several important ways from the model in Windows 2000. In Windows 2000, the smallest unit of replication is an individual attribute. At first examination, this seems to be what is wanted; after all, you don't want to have to replicate an entire object if only an attribute of that object has changed. The problem with this approach is that some attributes are multivalued. That is, they have multiple values. An example is the membership attribute of a universal group. This attribute represents all the members of the universal group.
In Windows 2000, by adding or removing a single user from the group, you caused the entire group membership to be replicated. In large organizations, a significant amount of replication traffic was often generated because universal groups might have several thousand members. Windows Server 2003 resolves this problem by replicating only the attribute's updated value. With universal group membership, this means that only the users you've added or removed are updated, rather than the entire group membership.
As discussed in the section entitled "Extensible Storage Engine", Active Directory uses transactional processing. When there are many changes, Active Directory processes the changes in batches of 5,000 at a time. This means that Active Directory processes a single transaction or multiple transactions in sequence until it reaches 5,000 changes, then it stops and checks to see if other processes are waiting for the CPU. Because a transaction must be completed before processing stops in this way, this places a practical limit on the number of changes that can be made in a single transaction—that number is 5,000.
In Windows 2000, because all the members of a group were processed any time a group's membership was changed, the limit on transactions also placed a practical limit on the number of members in a group. Again, this value is 5,000. The change in the way Windows Server 2003 replicates multivalued attributes also removes the limitation of 5,000 members for groups.
Note
When a forest is running at Windows Server 2003 functional level or Windows Server 2003 interim functional level, the members of the forest can take advantage of the previously discussed replication enhancements. For Windows Server 2003 functional level, this means that all domain controllers in all domains within the forest must be running Windows Server 2003. For Windows Server 2003 interim functional level, this means that all domain controllers in all domains within the forest must be running either Windows Server 2003 or Microsoft Windows NT.
Other replication enhancements for Windows Server 2003 involve intersite replication. Windows Server 2003 introduces the ability to turn off compression for intersite replication and to enable notification for intersite replication. Windows Server 2003 also has an improved knowledge consistency checker (KCC), which allows Active Directory to support a greater number of sites. These changes affect intersite replication in the following key ways:
In Windows 2000 and Windows Server 2003, all intersite replication traffic is compressed by default. Although this significantly reduces the amount of traffic between sites, it increases the processing overhead required on the bridgehead servers to replicate traffic between sites. Therefore, if processor utilization on bridgehead servers is a concern, and you have adequate bandwidth connections between sites, you may want to disable compression, which Windows Server 2003 allows you to do.
In Windows 2000 and Windows Server 2003, replication between sites occurs at scheduled intervals according to the site link configuration. In Windows Server 2003, you can enable notification for intersite replication, which allows the bridgehead server in a site to notify the bridgehead server on the other side of a site link that changes have occurred. This allows the other bridgehead server to pull the changes across the site link and thereby get more frequent updates.
In Windows 2000, the maximum number of sites you can have in a forest is greatly influenced by the knowledge consistency checker (KCC). As a result, there's a practical limit of about 100 sites per forest. Because the KCC in Windows Server 2003 has been revised, the KCC itself is no longer the limiting factor. This means that you can have many hundreds of sites per forest.
Note
To turn off compression or enable notification, you need to edit the related site link or connection object. See the section entitled "Configuring Site Link Replication Options".
Active Directory replication is a multipart process that involves a source domain controller and a destination domain controller. From a high level, replication works much as shown in Figure 35-3.
The step-by-step procedure goes like this:
When a user or a system process makes a change to the directory, this change is implemented as an LDAP write to the appropriate directory partition.
The source domain controller begins by looking up the IP address of a replication partner. For the initial lookup or when the destination DNS record has expired, the source domain controller does this by querying the primary DNS server. Subsequent lookups can be done using the local resolver cache.
The source and destination domain controllers use Kerberos to mutually authenticate each other.
The source domain controller then sends a change notification to the destination domain controller using RPC over IP.
The destination domain controller sends a request for the changes using RPC over IP, including information that allows the source domain controller to determine if those changes are needed.
Using the information sent by the destination domain controller, the source domain controller determines what changes (if any) need to be sent to the destination domain controller, and then sends the required changes using RCP over IP.
The destination domain controller then uses the replication subsystem to write the changes to the directory database.
Note
For intersite replication, two transports are available: RPC over IP and SMTP. With this in mind, SMTP could also be used as an alternate transport. SMTP uses TCP port 25.
As you can see from this overview, Active Directory replication depends on the following key services:
LDAP
Domain Name System (DNS)
Kerberos version 5 Authentication
Remote Procedure Call (RPC)
These Windows services must be functioning properly to allow directory updates to be replicated. Active Directory also uses File Replication Service (FRS) to replicate files in the System Volume (SYSVOL) shared folders on domain controllers. The User Datagram Protocol (UDP) and TCP ports used during replication are summarized in Table 35-1.
Table 35-1. Ports Used During Active Directory Replication
Service/Component | Port | |
|---|---|---|
UDP | TCP | |
LDAP | 389 | 389 |
LDAP Secure Sockets Layer (SSL) | 686 | |
Global Catalog (LDAP) | 3268 | |
Kerberos version 5 | 88 | 88 |
DNS | 53 | 53 |
Server Message Block (SMB) over IP | 445 | 445 |
Inside Out: How is FRS used with Active Directory?
A lot of people have asked me what exactly FRS is—and it can be confusing to see how it fits in with Active Directory. File Replication Service (FRS) is a replication service that uses the Active Directory replication topology to replicate data files. On behalf of Active Directory, FRS replicates files and folders in the System Volume (SYSVOL) shared folders on domain controllers. The SYSVOL folder contains domain policy, as well as scripts used for log on, log off, shutdown, and startup, and other related files used by Active Directory.
FRS is used to transfer the physical data files themselves. The replication topology used is the one implemented by Active Directory. The way this works is that FRS checks with the KCC to determine the replication topology that has been generated for Active Directory replication, and then uses this replication topology to replication SYSVOL files to all the domain controllers in a domain.
Active Directory's multimaster replication model is designed to ensure that there is no single point of failure. In this model, every domain controller can access changes to the database, and those changes can be replicated to all other domain controllers. When replication occurs within a domain, the replication follows a specific model that is very different from the replication model used for intersite replication.
With intrasite replication, the focus is on ensuring that changes are rapidly distributed. Intrasite replication traffic is not compressed, and replication is designed so that changes are replicated almost immediately after a change has been made. The main component in Active Directory responsible for the replication structure is the KCC. One of the main responsibilities of the KCC is to generate the replication topology—that is, the way replication is implemented.
As domain controllers are added to a site, the KCC configures a ring topology for intrasite replication with pull replication partners. Why use this model? For the following reasons:
In a ring topology model, there are always at least two paths between connected network resources to provide redundancy. Creating a ring topology for Active Directory replication ensures that there are at least two paths that changes can follow from one domain controller to another.
In a pull replication model, two servers are used. One is designated the push partner, the other the pull partner. It is the responsibility of the push partner to notify the pull partner that changes are available. The pull partner can then request the changes. Creating push and pull replication partners allows for rapid notification of changes and for updating once a request for changes has been made.
The KCC uses these models to create a replication ring. As domain controllers are added to a site, the size and configuration of this ring change. When there are at least three domain controllers in a site, each domain controller is configured with at least two incoming replication connections. As the number of domain controllers changes, the KCC updates the replication topology.
When a domain controller is updated, it waits approximately 15 seconds before initiating replication. This short wait is implemented in case additional changes are made. The domain controller on which the change is made notifies one of its partners, using an RPC, and specifies that changes are available. The partner can then pull the changes. Once replication with this partner is completed, the domain controller waits approximately 3 seconds, and then notifies its second partner of changes. The second partner can then pull the changes. Meanwhile, the first partner is notifying its partners of changes as appropriate. This process continues until all the domain controllers have been updated.
Inside Out: Replicating urgent changes
The 15-second delay for replication applies to Windows Server 2003. For Windows 2000, the default delay is 300 seconds. In either case, however, the delay is overridden to allow immediate replication of priority changes. Priority (urgent) replication is triggered if you perform one of the following actions:
Lock out an account, change the account lockout policy, or if an account is locked out automatically due to failed logon attempts
Change the domain password policy
Change the password on a domain controller computer account
Change the relative ID master role owner
Change a shared secret password used by the Local Security Authority (LSA) for Kerberos authentication
Urgent replication means that there is no delay to initiate replication. Note that all other changes to user and computer passwords are handled by the designated primary domain controller (PDC) emulator in a domain. When a user changes a normal user or computer password, the domain controller to which that user is connected immediately sends the change to the PDC emulator. This way, the PDC emulator always has the latest password for a user. This is why the PDC emulator is checked for a new password if a logon fails initially. Once the new password is updated on the PDC emulator, the PDC emulator replicates the change using normal replication. The only exception is when a domain controller contacts the PDC emulator requesting a password for a user. In this case, the PDC emulator immediately replicates the current password to the requesting domain controller so that no additional requests are made for that password.
Figure 35-4 shows a ring topology that a KCC would construct if there were three domain controllers in a site.
As you can see from the figure, replication is set up as follows:
DC1 has incoming replication connections from DC2 and DC3.
DC2 has incoming replication connections from DC1 and DC3.
DC3 has incoming replication connections from DC1 and DC2.
If changes were made to DC1, DC1 would notify DC2 of the changes. DC2 would then pull the changes. Once replication was completed, DC1 would notify DC3 of the changes. DC3 would then pull the changes. Because all domain controllers in the site have now been notified, no additional replication would occur. However, DC2 would still notify DC3 that changes were available. DC3 would not pull the changes, however, because it would already have them.
Domain controllers track directory changes using Update Sequence Numbers (USNs). Any time a change is made to the directory, the domain controller assigns the change a USN. Each domain controller maintains its own local USNs and increments their values each time a change occurs. The domain controller also assigns the local USN to the object attribute that changed. Each object has a related attribute called uSNChanged. The uSNChanged attribute is stored with the object and identifies the highest USN that has been assigned to any of the object's attributes.
To see how this works, consider the following example. The local USN for DC1 is 125. An administrator connected to DC1 changes the password on a user's account. DC1 registers the change as local USN 126. The local USN value is written to the uSNChanged attribute of the user object. If the administrator next edits a group account and changes its description, DC1 registers the change as local USN 127. The local USN value is written to the uSNChanged attribute of the Group object.
Note
With replication, there is sometimes a concern that replication changes from one domain controller may overwrite similar changes made to another domain controller. However, as object changes are tracked on a per-attribute basis, this rarely happens. It is very unlikely that two administrators would change the exact same attributes of an object at the exact same time. By tracking changes on a per-attribute basis, Active Directory effectively minimizes the possibility of any conflict.
Each domain controller tracks not only its local USN but also the local USNs of other domain controllers in a table referred to as an up-to-dateness vector. During the replication process, a domain controller that is requesting changes includes its up-to-dateness vector. The receiving domain controller can then compare the USN values to those it has stored. If the current USN value for a particular domain controller is higher than the stored value, changes associated with that domain controller need to be replicated. If the current value for a particular domain controller is the same as the stored value, changes for that domain controller do not need to be replicated.
As only necessary changes are replicated, this process of comparing up-to-dateness vectors ensures that replication is very efficient and that changes are propagated only when necessary. The up-to-dateness vectors are in fact the mechanism that enables domain controllers with redundant connections to know that they've already received the necessary updates.
Inside Out: Schema changes have priority
Several types of replication changes have priority. If you make changes to object attributes in the schema, these changes take precedence over most other changes. In this case, Active Directory blocks replication of normal changes and replicates the schema changes. Active Directory will continue to replicate schema changes until the schema configuration is synchronized on all domain controllers in the forest. This ensures that schema changes are applied rapidly. Still, it's a good idea to make changes to the schema during off-hours, since schema changes need to be propagated throughout the forest before other changes such as resetting passwords can be made to Active Directory.
The Windows Server 2003 schema adds 25 indexed attributes to the schema directory partition. When you upgrade or install the first Windows Server 2003 domain controller, these changes are replicated throughout the forest. Because of this, it is recommended that Windows 2000 domain controllers have Service Pack 3 or later installed before you upgrade or install the first Windows Server 2003 domain controller.
While intrasite replication is focused on speed, intersite replication is focused on efficiency. The primary goal of intersite replication is to transfer replication information between sites while making the most efficient use of the available resources. With efficiency as a goal, intersite replication traffic uses designated bridgehead servers and a default configuration that is scheduled rather than automatic, and compressed rather than uncompressed.
With designated bridgehead servers, the ISTG limits the points of replication between sites. Instead of allowing all the domain controllers in one site to replicate with all the domain controllers in another site, the ISTG designates a limited number of domain controllers as bridgehead servers. These domain controllers are then the only ones used to replicate information between sites.
With scheduled replication, you can set the valid times during which replication can occur and the replication frequency within this scheduled interval. By default, when you configure intersite replication, replication is scheduled to occur every 180 minutes 24 hours a day. When there's limited bandwidth between sites, you might want to change the default schedule to better accommodate the users who also use the link. For example, you might want to allow replication to occur every 180 minutes 24 hours a day on Saturday and Sunday, but during the week set the schedule to allow more bandwidth during the day. For example, you might set replication to occur every 60 minutes from 6 A.M. to 8 A.M. and from 7 P.M. to 3 A.M. Monday through Friday.
With compression, replication traffic is compressed 85 to 90 percent, meaning that it is 10 to 15 percent of its uncompressed size. This means that even low bandwidth links can often be used effectively for replication. Compression is triggered when the replication traffic is more than 32 kilobytes (KB) in size.
As discussed previously, there are two key ways to change intersite replication, as follows:
Turn off automatic compression if you have sufficient bandwidth on a link and are more concerned about the processing power used for compression.
Enable automatic notification of changes to allow domain controllers on either side of the link to indicate that changes are available. Automatic notification allows those changes to be requested rather than making domain controllers wait for the next replication interval.
Regardless of the site link configuration, replication traffic is sent through dedicated bridgehead servers rather than through multiple replication partners. When changes are made to the directory in one site, those changes are replicated to the other site via the designated bridgehead servers. The bridgehead servers then initiate replication of the changes exactly as was discussed in the section entitled "Intrasite Replication Essentials" earlier in this chapter, except that SMTP can be used instead of RPC over IP if SMTP is being used as a transport. Thus, intersite replication is really concerned with getting changes from one site to another across a site link.
Figure 35-5 shows an example of intersite replication using a single dedicated bridgehead server on each side of a site link. In this example, DC3 is the designated bridgehead server for Site 1 and DC4 is the designated bridgehead server for Site 2.
As you can see from the figure, replication is set up as follows:
DC1 has incoming replication connections from DC2 and DC3.
DC2 has incoming replication connections from DC1 and DC3.
DC3 has incoming replication connections from DC1 and DC2.
DC4 has incoming replication connections from DC5 and DC6.
DC5 has incoming replication connections from DC4 and DC6.
DC6 has incoming replication connections from DC4 and DC5.
If changes were made to DC1 in Site 1, DC1 would notify DC2 of the changes. DC2 would then pull the changes. Once replication was completed, DC1 would notify DC3 of the changes. DC3 would then pull the changes. Because all domain controllers in the Site 1 have now been notified, no additional replication would occur within the site. However, DC2 would still notify DC3 that changes were available. DC3 would not pull the changes, however, because it would already have them.
According to the site link configuration between Site 1 and Site 2, DC3 would notify DC4 that changes were available. DC4 would then pull the changes. Next DC4 would notify DC5 of the changes. DC5 would then pull the changes. Once replication was completed, DC4 would notify DC6 of the changes. DC6 would then pull the changes. Because all domain controllers in Site 2 have now been notified, no additional replication would occur. However, DC5 would still notify DC6 that changes were available. DC6 would not pull the changes, however, because it would already have the changes.
So far, I've talked about designated bridgehead servers but haven't said how bridgehead servers are designated. That's because it is a rather involved process. When you set up a site, the knowledge consistency checker (KCC) on a domain controller that Active Directory has designated the Inter-Site Topology Generator (ISTG) is responsible for generating the intersite topology. Each site has only one ISTG and its job is to determine the best way to configure replication between sites.
The ISTG does this by identifying the bridgehead servers that are to be used. Replication between sites is always sent from a bridgehead server in one site to a bridgehead server in another site. This ensures that information is replicated only once between sites. As domain controllers are added and removed from sites, the ISTG regenerates the topology automatically.
The ISTG also creates the connection objects that are needed to connect bridgehead servers on either side of a site link. This is how Active Directory logically represents a site link. The ISTG continuously monitors connections and will create new connections when a domain controller acting as a designated bridgehead server is no longer available. In most cases, there will be more than one designated bridgehead server, and I'll discuss why in the following section, "Replication Rings and Directory Partitions."