Crossing the Great Divide

Keep your hair from turning gray by understanding how Active Directory access works between domains.

• By Bill Boswell
• April 01, 2002

In spite of our civilized veneer, we are tribal animals at heart. Elementary school kids join clubs. High school students form cliques. Military units have officers and enlisted persons. There are Democrats and Republicans, Eastern and Western football conferences, Orthodox and Reformed sects, lovers of haiku and, well, you get the picture. Everywhere you look, deep divisions form between folks who might otherwise be friends and colleagues.

Take Active Directory as a case in point. Whether you work in a global conglomerate or a 20-node organization with two branch offices, it’s nearly impossible to get different IT groups to trust each other enough to form a single domain. Once you decide to have separate domains, though, arranging user access to shared resources can get a little tricky. Consider, for example, a situation encountered recently by two system administrators, Ann and Frank, who work at a large university.

The AD forest for this university roots at a domain called ad.mortarboard.edu. Ann works for the university hospital, whose domain is hospital.ad.mortarboard.edu. Frank works for the College of Math and Science in the mathsci. ad.mortarboard.edu domain. Figure 1 shows the trust relationships in AD Domains and Trusts.

Figure 1. The domain trust relationships of Mortarboard University. (Click image to view larger version.)

At Mortarboard University, staff and faculty in the hospital often require access to research documents stored on servers in the College of Math and Science. By the same token, pre-med students in the College need access to medical records to do their research. Ann and Frank became frustrated with the university’s convoluted cross-departmental work order system and decided to use the transitive trusts between AD domains to control access to resources in each other’s domains. Here’s how their arrangement works.

Let’s say grad students in the MathSci domain need access to files stored on Ann’s servers in the Hospital domain. Ann protects the shared folders containing these files with NTFS permissions that give read/write access to certain Domain Local groups in her domain. Ann chose to use Domain Local groups because they can have members from other domains. This is also true for Universal groups; but the Hospital domain has not yet been shifted to Native mode, which is necessary for Universal groups to be functional. Ann gave Frank permission to modify the membership of those selected groups in her domain, and Frank uses this privilege to add and remove his users to the groups in response to access requests. Ann and Frank have worked together for years and they trust each other.

The Mystery of the Gray-haired Lady
One day, though, Ann noticed something strange and gave Frank a call. “I’m looking at the membership list for one of my groups and I see a user named Charlotte Johnson from your domain. It’s weird, though, because the icon next to her name has gray hair. All the other icons from your domain and mine have black hair.” She pasted a screenshot into an e-mail message and sent it to Frank.

Frank scanned a listing in AD Users and Computers and told Ann, “I don’t have any users named Charlotte Johnson. Hold on.” He riffled through some paperwork. “I do have a Charlotte Hennessy and it looks like she got married over the summer. There’s a completed work order here for changing her name. Let’s see. Yup. Her maiden name was Johnson. Let me check her group memberships.” He opened the properties page for Charlotte’s object in AD Users and Computers and selected the Member Of tab. “Sorry,” he said, “Must be someone else. Charlotte Hennessey doesn’t show up as a member of any groups in your domain.”

Ann and Frank called a somewhat bewildered Charlotte Hennessey and verified that she, indeed, used to be Charlotte Johnson. She also verified that she had access to her files in the Hospital domain even though the group showed “Charlotte Johnson,” not “Charlotte Hennessy,” as a member.

Because there didn’t appear to be any outright problem other than the difference in the name and the funny icon, Ann and Frank decided to shelve the mystery for a while. Later that day, just before she left for home, Ann checked the membership list for the group again and discovered that Charlotte’s name now displayed as Charlotte Hennessey and the user icon once again had black hair, not gray. She left thinking her mother had been right about studying dentistry rather than Information Systems Management.

To understand what happened to Charlotte’s name and user icon, and to make Ann feel better about selecting a professional career that doesn’t involve getting saliva on her hands, let’s look at the way AD displays and replicates group members.

Forward and Backward
The Group properties displayed in the AD Users and Computers console show the contents of various attributes associated with the group object in AD. Group membership is contained in an attribute called Member. To ensure internal consistency for Group members, AD uses a trick taken from double entry accounting. It designates the Member attribute of a group as a forward link attribute and associates it with a back link attribute called MemberOf on each object that is a member of the group. AD doesn’t actually store entries in a back link attribute. Instead, as shown in Figure 2, it calculates the back link contents on the fly by querying the associated forward links. For example, when you open the properties of a user object in AD Users and Computers and select the Member Of tab, the domain controller searches AD for any groups to which the user belongs and returns that list as if it were the contents of the MemberOf attribute.

Figure 2. Contents of the Member Of attribute are calculated on the fly by querying back links to the Member attribute in groups. (Click image to view larger version.)

If you look at the contents of the Member attribute for a group using an LDAP browser or the ADSI Editor, you would see a list of LDAP distinguished names in the format cn=user_name, ou=some_container,dc=domain_name,dc=domain_root. Actually, though, AD doesn’t store distinguished names in the forward link of a forward/back link pair. Instead, it stores the internal database identifier of each object assigned to the forward link. This maintains referential integrity of the object list.

A Rose by any Other Naming Context…
All this talk of forward and back links might seem like programming exotica, but it goes to the root of the odd behavior of Charlotte’s user icon because of the way AD stores objects from different domains. AD is an LDAP directory service. In LDAP, it’s possible to divide the database into separate partitions, called naming contexts. Each naming context forms a unique replication unit. This reduces the size of the overall database hosted by any individual DC and avoids replicating changes between geographically or politically diverse portions of an organization. If an LDAP server gets a search request for an object in a naming context it doesn’t host, it returns a referral listing the LDAP servers where the client can find the correct naming context.

AD takes advantage of this LDAP feature by placing each domain into its own discrete naming context. Furthermore, an AD DC can only host a read-write replica of the naming context for its own domain. For example, a DC in the Hospital domain would have a full, read-write replica of the Hospital naming context and no replica at all of the MathSci naming context.

This creates a challenge for forward/back link attributes like Member and MemberOf. Here’s why. When Frank originally added the Charlotte Johnson account from the MathSci domain to the group in the Hospital domain, the Hospital DC was forced to account for the entry by adding a database identifier for the “Charlotte Johnson” object into the Member attribute for the group. But the Hospital DC has no replica of the MathSci naming context and therefore has no database identifier for the “Charlotte Johnson” object.

The Phantom
To overcome this challenge, a DC uses another accounting trick. It creates a balancing entry called a phantom record in its naming context to represent the object in the outside naming context. The phantom record consists of the outside object’s Distinguished Name, the object’s SID, and its Globally Unique Identifier, or GUID. Essentially, a phantom record acts like a placeholder at a fancy dinner party that reads, “This place setting belongs to the Tralfamadorian ambassador. Leave any messages under the wine goblet.”

One more bit of information is necessary to explain what happened to Charlotte’s name and icon. Although a standard DC can only host its own domain naming context, AD has a feature called a Global Catalog (GC) that permits selected DCs to host a partial, read-only replica of the naming contexts for all domains in a forest. This permits a GC server to fulfill LDAP search requests for any object in the forest so long as the search involves attributes in the partial attribute set assigned to the Global Catalog. For example, the partial attribute set contains the Distinguished Name and SID attributes for user and group objects but doesn’t contain the Member attribute for Domain Local groups such as those used by Ann to control access to her shared resources.

So, armed with this information, let’s work out an explanation for Ann and Frank as to why Charlotte’s name and user icon behaved so strangely.

When Charlotte’s name in the MathSci domain was changed from Charlotte Johnson to Charlotte Hennessey, this resulted in a change to the Distinguished Name attribute for Charlotte’s AD object in the MathSci naming context. This modified Distinguished Name attribute was then replicated to every DC hosting a replica of the MathSci naming context, including any Global Catalog servers.

Charlotte’s name change had no impact on her security permissions because her Security ID (SID) remained unchanged. This is why Charlotte was still able to access resources in the Hospital domain even though her name didn’t match the listing in the Member attribute of the Domain Local group protecting the resource.

As for the accelerated aging of Charlotte’s icon, this was a result of the way that AD Users and Computers handles the contents of the Member attribute for a group. When Ann opened the properties window for the group and selected the Members tab, AD Users and Computers submitted an LDAP search to a DC in the Hospital domain asking for the contents of the group’s Member attribute. This search returned the Distinguished Names of the group’s members, names that were derived from the database identifiers that constitute the true contents of a forward link attribute such as Member. One of those database identifiers pointed at a phantom record for “Charlotte Johnson” in the Hospital naming context.

AD Users and Computers always verifies the membership list for a group by querying a Global Catalog server for matches to the Distinguished Names on the list. If it doesn’t find a match for a particular Distinguished Name, it changes the associated user icon to an icon with gray hair rather than one with black hair. This is why Charlotte’s icon changed. The name “Charlotte Johnson” in the phantom record no longer existed in the MathSci naming context stored on the GC server. (If AD Users and Computers cannot locate a GC server at all, it displays a warning then shows all the icons with gray hair.)

The Infrastructure Master Did It
That’s not the end of the story, though. Recall that when Ann checked the group members later that day, she discovered that Charlotte’s name had changed from Charlotte Johnson to Charlotte Hennessey and the user icon once again had black hair. This happened thanks to the action of a Flexible Single Master Operations (FSMO) role master called the Infrastructure Master. (A FSMO is a DC that’s been assigned this unique duty, either in a domain or in a forest.)

The Infrastructure Master periodically paws through the phantom records in its domain and verifies that the Distinguished Names of the phantom records match the Distinguished Names of the objects in the source domains. It does this by querying a Global Catalog server. If the Distinguished Name of a phantom record doesn’t exist in the GC, the Infrastructure Master checks to see if the GUID still exists. If so, it creates a new phantom record with a new Distinguished Name and deletes the old one. If the GUID doesn’t exist, it deletes the phantom record entirely. These phantom records then replicate from the Infrastructure Master to the other DCs in that domain.

You may already know that it’s important not to assign the Infrastructure Master role to a Global Catalog server. Here’s the reason. A DC creates phantom records because it needs internal database identifiers for linked attributes that reference objects from outside naming contexts. Because a GC server hosts a replica of all naming contexts in a forest, it has no need for phantom records and won’t create them. Without phantom records, the Infrastructure Master has nothing to check, so the other DCs that aren’t GC servers will not obtain updates to their phantom records. Always be sure to transfer the Infrastructure Master role to a non-GC DC in each domain.

If you have multiple domains in your production forest, you may have already encountered the mysterious gray-haired icons seen by Ann and Frank. The Infrastructure Master checks the phantom records every eight hours, so it might take an icon and the associated user name a long time to change. As we’ve seen, having the old name on the member list doesn’t interfere with operations and requires no additional action on your part. That leaves you free to attend the planning meetings for consolidating your domains once Windows .NET is released. The fun never stops.