Directory Service: Event ID 1480 and 1393, replication halted due to low disk space

Standard

The information provided in event logs is often not too clear but it has definitely gotten better starting in W2K8. I recently encountered an issue where replication delays to certain DC were reported. I immediately looked at the repadmin  replication summary and noticed that my deltas that usually stayed around within an hour had jumped up to 13~ hours.

 

replsum-issues

 

The ‘destination DSA’ section gives you a clue about the DC that’s having issues pulling replicated changes in. I looked at the event logs on the said DC and filtered the DS logs around to 13 hours ago. I noticed event ID 1393, and 1480 sighting the issue with the low disk space and how it had paused the netlogon service. Luckily there was no “user authentication traffic” against this DC as it was a hub site DC with no user subnet tied to this site, otherwise the impact would have been bigger with users not being able to logon to their workstations. The lag on the replication on this instance was reported by an internal application portal that was not reflecting the changes that were made in AD. Nonetheless, it was an issue that had to be fixed.

 

event1393

 

event1480

 

A low disk space issue on a DC is serious issue anyway but in this instance, a FIM job had just run that had imported thousands of new objects incurring a slight NTDS size change. This is one of the reasons I don’t like to put NTDS and SYSVOL on the system partition. I did perform the clean up of some unneeded and temp files but the long term solution is to relocate the DB to a different drive/volume, http://technet.microsoft.com/en-us/library/cc782948%28WS.10%29.aspx.

 

Immediately after the replication was back to normal.

 

replsum-issues2

Auditing Group Membership changes

Standard

I often get this asked this question, “how do I audit group membership changes”. Whereas a lot of AD Change Monitor Tools (Quest, Netwrix etc.) have nice reports that can be generated to look up this information, this question comes up when a change auditor product for AD is not in picture. Let me cover the highlights here.

1. You need to have the Auditing enabled with Group Policy.

Computer ConfigurationPoliciesWindows SettingsSecurity SettingsLocal PoliciesAudit Policy

 

image

2. In order to see on which DC the change was made, you can lookup the metadata via repadmin.

repadmin /showobjmeta test-dc01 "CN=Test Group,OU=Groups,DC=techevan,DC=lab"

Towards the end of the output you see the “absent” in this example on which DC a particular user was removed from this group.

Type     Attribute     Last Mod Time         Originating DSA         Loc.USN          Org.USN Ver        Distinguished Name
===  ========  ===========      =================   ======= ======= === =========================
ABSENT   member        2010-11-05 16:55:28 TestSiteTEST-DC01  749327  749327   2  CN=Rick Sheikh,OU=Users,DC=techevan,DC=lab

 

3.  You can comb the logs on the said DC using EventComb or Event Viewer. Event ID 4729 is logged when a member is removed from a group.

image

Some other important Event IDs for User and Group Auditing in Windows Server 2008 R2 are these:

4727 – A security-enabled global group was created.

4728 – A member was added to a security-enabled global group.

4730 – A security-enabled global group was deleted.

4731 – A security-enabled local group was created.

4732 – A member was added to a security-enabled local group.

4733 – A member was removed from a security-enabled local group.

4734 – A security-enabled local group was deleted.

4735 – A security-enabled local group was changed.

4737 – A security-enabled global group was changed.

4754 – A security-enabled universal group was created.

4755 – A security-enabled universal group was changed.

4756 – A member was added to a security-enabled universal group.

4757 – A member was removed from a security-enabled universal group.

4758 – A security-enabled universal group was deleted.

 

More reading here : http://www.windowsecurity.com/articles/Event-IDs-Windows-Server-2008-Vista-Revealed.html

PowerShell : How do I create Trust ?

Standard

A question was raised on ActiveDir regarding the ability to create Trust via a command line utility. It was discussed how netdom is no longer a supported command line utility to create Trusts. I referenced a snip from AD Cook Book using PowerShell to create Trust via the .Net AD namespace method alternatively.(System.DirectoryServices.ActiveDirectory).

You can use following code to create a two-way transitive trust between the local forest and a remote forest. You will have to run this code on the opposite side of the trust in order for the trust to be fully functional.

$localfor=[System.DirectoryServices.ActiveDirectory.Forest]::getCurrent()
$strRemoteFor='remotedomain.net'
$strRemoteUser='administrator'
$strRemotePass='P@ssw0rd'
$remoteCon=New-Object
System.DirectoryService.ActiveDirectory.DirectoryContext('Forest',
$strRemoteFor,$strRemoteUser,$strRemotePass)
$trustDirection='BiDirectional'$localFor.CreateTrustRelationship
($remoteFor,$trustDirection)

Server Core R2 DC promotion fails due to unavailable ADDS binaries

Standard

I encountered an issue promoting a Server Core R2 to a domain controller. The DCPROMO on Server Core is handled via unattended mode with answer file. The error I received is below. It was due to Server Core’s inability to install/confirm ADDS binaries.

C:UsersAdministrator>dcpromo /unattended:answer.txt
Checking if Active Directory Domain Services binaries are installed…
Failed to detect if Active Directory Domain Services binaries were installed. The error was: An error with no description has occurred.

And the DCPROMOUI.log also shed some light on the nature of the issue.

dcpromoui 504.204 001E 03:01:08.709     Unable to find identity string for package name Microsoft-Windows-ServerCore-Package
dcpromoui 504.204 001F 03:01:08.709   Failed to retrieve the parent package name
dcpromoui 504.204 0020 03:01:08.709   HRESULT = 0x800F0818
dcpromoui 504.204 0021 03:01:08.709   HRESULT = 0x800F0818
dcpromoui 504.31C 0022 03:01:08.709     HRESULT = 0x800F0818
dcpromoui 504.31C 0023 03:01:08.709   Enter GetErrorMessage 800F0818

My attempt to add the binaries via the DISM also failed.

dism /online /enable-feature /featurename:DirectoryServices-DomainController-ServerFoundation

Deployment Image Servicing and Management tool
Version: 6.1.7600.16385

Image Version: 6.1.7600.16385

Error: 0x800f0818

DISM failed. No operation was performed.
For more information, review the log file.

The DISM log file can be found at C:WindowsLogsDISMdism.log  

The OCSETUP attempt also failed sighting the same error.

ocsetup DirectoryServices-Domain-Controller-ServerFoundation

 

image

 

After I had ensured that the OS was up-to-date with patches and updates, I stumbled upon System Update Readiness Tool KB947821. What is System Update Readiness Tool ? it is a patch that helps you find the inconsistencies with system files.

System resources, such as file data, registry data, and even in-memory data, can develop inconsistencies during the lifetime of the operating system. These inconsistencies might be caused by various hardware failures or might be caused by software issues. In some cases, these inconsistencies can affect the Windows servicing store, and they can cause software updates not to work. The System Update Readiness Tool tries to resolve these inconsistencies.

The System Update Readiness Tool creates a log file that captures any issues that the tool found or fixed. The log file is located at the following location:

  • %SYSTEMROOT%LogsCBSCheckSUR.log
  • %SYSTEMROOT%LogsCBSCheckSUR.persist.log

On this Server Core, the CheckSUR.log indicated issues with two files (its an update specific MUM file with its catalog file), that the tool found to be corrupted.

Checking System Update Readiness.
Binary Version 6.1.7600.20822
Package Version 10.0
2011-02-19 20:53

Checking Windows Servicing Packages

Checking Package Manifests and Catalogs
(f)    CBS MUM Corrupt    0x00000000    servicingPackagesPackage_for_KB2207566_RTM~31bf3856ad364e35~amd64~~6.1.1.0.mum        Expected file name Microsoft-Windows-ServerCore-Package~31bf3856ad364e35~amd64~~6.1.7600.16385.mum does not match the actual file name

Checking Package Watchlist

Checking Component Watchlist

Checking Packages

Checking Component Store

Summary:
Seconds executed: 165
Found 1 errors
  CBS MUM Corrupt Total count: 1

Unavailable repair files:
    servicingpackagesPackage_for_KB2207566_RTM~31bf3856ad364e35~amd64~~6.1.1.0.mum
    servicingpackagesPackage_for_KB2207566_RTM~31bf3856ad364e35~amd64~~6.1.1.0.cat

I also came across this Technet forum post, which is similar in nature but is applicable to different corrupt files with different symptoms, however it was this post that gave me the idea to fix my problem.

Solution :

1. I downloaded the KB2207566 (Windows6.1-KB2207566-x64.msu) as indicated by SUR log, and copied it to Server Core (c:temp). Note that this patch already had been installed but for some reason had the mentioned files corrupted.

2. I then extracted the MSU file into a sub folder and extracted the *.cab files.

C:UsersAdministrator>cd c:temp

c:temp>wusa Windows6.1-KB2207566-x64.msu /extract:c:servicingkb2207566

c:temp>cd kb2207566

c:tempkb982214>mkdir files

c:tempkb982214>expand Windows6.1-KB2207566-x64.cab -F:* files

The two files I had to replace were these :

image

The location for the existing files that had to be replaced was c:windowsservicingpackages and an issue I ran into, as it was indicated on the Technet post (when I simply attempted to UNC and copy from another regular W2K8 machine) was that these were protected files and the copy/create option was denied. Lot of systems folders/files in Windows 7 and Windows Server 2008 have a different owner than ‘administrators’ and the TrustedInstaller is set as owner and has full control rights set.

image

3. And I then opted to use the command line option to take the ownership of the folder and assign ‘administrators’ the full rights on the Server Core.

takeown /f c:windowsservicingpackages /r /d y

icacls c:windowsservicingpackages /grant administrators:F /T

4. And lastly I copied the extracted files and replaced.

C:tempkb982214files>copy "package_for_kb2207566_rtm~31bf3856ad364e35~amd64~~6
.1.1.0.cat" c:WindowsservicingPackages

Overwrite c:WindowsservicingPackagespackage_for_kb2207566_rtm~31bf3856ad364e
35~amd64~~6.1.1.0.cat? (Yes/No/All): y
      

1 file(s) copied.

C:tempkb982214files>copy "package_for_kb2207566_rtm~31bf3856ad364e35~amd64~~6
.1.1.0.mum" c:WindowsservicingPackages

Overwrite c:WindowsservicingPackagespackage_for_kb2207566_rtm~31bf3856ad364e
35~amd64~~6.1.1.0.mum? (Yes/No/All): y

1 file(s) copied.

No restart was required and the DCPROMO with the answer file succeeded as it was now able to install the ADDS binaries and the DC promoted successfully.

Note that prior to the fix I also noticed that I was unable to enable the remote management from the SCONFIG (option 4, sub option 3) which also worked afterwards.

PowerShell : How do I clear sIDhistory attribute ?

Standard

What is sIDhistory attribute ?

The sIDhistory attribute is the key attribute that holds the previous SID(s) of Users and Groups objects that facilitate the Active Directory migrations. It contains previous SIDs used for the object if the object was moved from another domain. Whenever an object is moved from one domain to another, a new SID is created and that new SID becomes the objectSID. The previous SID is added to the sIDHistory property. ADMT is one tool that allows you migrate SID from one domain to another, it calls on to the DsAddSidHistory API to accomplish this task.

However, there are times when you look into clearing the sIDhistory attribute after the migration is complete such as when you are attempting to avoid a token size bloat (kerberos token size threshold). You must do your homework before trying to remove sIDhistory, you should check the ACLs of your servers and applications and you should NOT clean up all groups (or users) at once, you should always try accessing the target resources and applications after sIDhistory has been cleared from a few groups/users, only after successful access (of a user that has re-authenticated – i.e. doesn’t have the SIDs from a group’s sIDhistory in his token), you should proceed with a mass removal.

To use PowerShell with Quest Cmdlets,

Get-QADUser user1 | %{Set-QADUser $_ -ObjectAttributes @{sIDHistory=@{delete=$_['sIDHistory']}}}

You may also pass a list of users or groups

Get-QADUser (get-content users.txt) | %{Set-QADUser $_ -ObjectAttributes @{sIDHistory=@{delete=$_['sIDHistory']}}}

PowerShell : How do I set the delegation sensitive flag on users and computers ?

Standard

Kerberos Delegation, constrained and unconstrained is a complex topic, and one that often comes up when Security implications of External/Forests Trusts are discussed. Few days ago, on ActiveDir a similar topic was shed light upon. In brief Brian Arkills sums it up below,

The other security implication that most folks seem oblivious to is the one that comes with unconstrained Kerberos delegation across a trust. When you’ve got a single domain/forest by itself, your domain admin group controls who/what can use delegation. Once you setup a trust to another domain, you extend that ability to the domain admins in the other domain(s). In other words, the domain admins in the other domain can choose to allow a user/computer in their domain to permit unconstrained Kerberos delegation using the user accounts in *your* domain. Almost everyone says unconstrained delegation is bad, and the assumption is that no one will use it. But there are no controls in the product to prevent its use, except by also preventing any delegation (i.e. by marking all or a subset of user accounts as ‘sensitive to delegation’). And of course, unconstrained delegation is the only way to use delegation across a forest trust meaning it’s the only game in town for certain scenarios.

A service that runs under an account that is trusted for Kerberos delegation can assume/impersonate the identity of a client requesting the service. This parameter sets the TrustedForDelegation property of an account object. This value also sets the ADS_UF_TRUSTED_FOR_DELEGATION flag of the Active Directory User Account Control attribute.

It is that marking of the user accounts as “delegation sensitive” with PowerShell is what I would like to share with you.

image

For user and computer accounts, as stated above the setting is enclosed under the useraccountcontrol bits.

http://support.microsoft.com/kb/305144

We take the TRUSTED_FOR_DELEGATION and its associated value in decimal i.e 524288 and assign it a variable.

$NOT_DELEGATED=1048576

Using Quest Cmdlets, we use the Set-QADUser and use the bitwise exclusive operator to set the flag as sensitive (Thanks to Shay Levy for helping me with the correct bitwise syntax)

Get-QADUser user1  |
    Set-QADObject $_ -ObjectAttributes @{userAccountControl= ($_.userAccountControl -bxor $NOT_DELEGATED)}

For computer accounts, there is an explicit parameter for the Set-QADComputer Cmdlet i.e -TrustForDelegation

With ADWS (the native AD Cmdlets) the same parameter (being applicable to both i.e users and computers) for Set-ADUser and Set-ADComputer is called –TrustedForDelegation.

By default, when an account has been granted delegation privileges *every* user account can be “impersonated” in this way. However, with above you can mark certain user accounts as “delegation sensitive”, which means that those user accounts can never be used in a delegation scenario. Marking admin accounts as delegation sensitive is highly recommended. Generally, delegation is used with Kerberos, but there is an option to use protocol transition so that NTLM might be used on some of the hops (this is called S4U). Also note that for service accounts (user accounts), the delegation tab will only appear (allowing for the constrained delegation) when there is a SPN registered against that user account.

image

More reading here :

http://blogs.technet.com/b/askds/archive/2008/03/06/kerberos-for-the-busy-admin.aspx

http://blogs.technet.com/b/askds/archive/2008/06/13/understanding-kerberos-double-hop.aspx

PowerShell : How to lookup Schema version of your forest ?

Standard

The schema version is revealed via the objectversion attribute off of the schema object from your configuration head of the forest i.e “cn=schema,cn=configuration,dc=yourdomain,dc=int”.

So using Quest Cmdlets, you can run this query :

Get-QADObject "cn=schema,cn=configuration,dc=yourdomain,dc=int" -ip objectversion | select objectversion

qadobject_sv

The –ip is the alias for includedproperties.

And, when using the native AD Cmdlets of Server 2008 R2 (ADWS), the syntax is slightly different ;

Get-ADObject "cn=schema,cn=configuration,dc=r2,dc=lab" -properties objectversion

qadobject_sv1

Above, you see the query ran with Quest Cmdlets resulting in objectversion 31 which is against a Server 2003 R2 Forest, whereas the latter is for a Server 2008 R2 Forest i.e Schema version 47.

Reviewing few very useful adds in Quest AD Cmdlets v1.4

Standard

Whereas most of newly added cmdlets focus on  PKI and Email Address Management in v1.4, there are a few cmdlets and handful of new parameters that ought to come in very handy with your AD tasks. Below I review a few that I think are some great adds :

When you are enumerating a large number of objects in shell (without outputting results into a file), you might just want to have a quick idea of the ‘total’ number of objects meeting the criteria of you query.

Get-QADProgressPolicy

“displays a progress bar for long-running commands”

progress_bar

This progress bar overlays (highlights in and out) as your query is running. It also appears when you are performing a count using the measure-object cmdlet or the “.count” switch

progress_bar2

You can set the progress bar setting and its threshold with

Set-QADProgressPolicy -ShowProgress $true -ProgressThreshold 2

The –activity parameter when relying on the progress bar allows you tag each line of progress with a number so that lengthy process is a bit more obvious with respect to the process to one or more cmdlet’s retrieved results.

Some new parameters :

Five new parameters for Get-QADUser

ExpiredFor

Inactive

InactiveFor

NotLoggedOnFor

PasswordNotChangedFor

Four new parameters for Get-QADComputer

Inactive

InactiveFor

NotLoggedOnFor

PasswordNotChangedFor

But what mechanism decides the “inactivity” benchmark to ask cmdlet to retrieve that information ?

You do.

Get-QADInactiveAccountsPolicy

inactivepolicy1

You can change these settings,

Set-QADInactiveAccountsPolicy -AccountExpiredPeriod 0 -AccountNotLoggedOnPeriod 30 –PasswordNotChangedPeriod 120

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }
.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

Note : These settings are profile specific so ones you define these thresholds they will stay there until you change those settings again.

The NotLoggedOnPeriod is probably based on the LastLogonTimeStamp, but I will check and edit this post if its any different. If it is, remember it may not be accurate and should only be used for estimation. The LastLogonTimeStamp gets updated from the LastLogon (DC specific attribute) based on a 9-14 day swing period.

Also :

“This parameter overrides the logon-related inactivity condition of the Inactive or InactiveFor parameter. Thus, if the NotLoggedOnFor value of 60 is supplied in conjunction with the InactiveFor value of 30, the cmdlet searches for accounts that are expired for 30 or more days, or have the password age of 30 or more days, or have not been used to log on for 60 or more days.”

Previously if you had to use the Get-QADGroupMember cmdlet to retrieve the enabled accounts only, you had to pass the LDAPFilter, now you can use the same –enabled and –disabled parameter as you could with Get-QADUser cmdlet since v1.3.

groupmemberdisabled

This and much more. All details can be found here.

The folks who develop these cmdlets and work on adding new parameters do take the feedback very seriously. I have myself asked and gotten couple of requests met. You can do the same.

Quest AD Cmdlets a.k.a Active Roles Management Shell version 1.4 gets released

Standard

From version 1.2 with 49 cmdlets, to version 1.3 with 63 cmdlets and now on to version 1.4 that has 32 new cmdlets making it total of 95.

Here are the new cmdlets in v1.4 :

• Get-QADLocalCertificateStore
• New-QADLocalCertificateStore
• Remove-QADLocalCertificateStore
• Get-QADCertificate
• Where-QADCertificate
• Add-QADCertificate
• Import-QADCertificate
• Show-QADCertificate
• Edit-QADCertificate
• Export-QADCertificate
• Remove-QADCertificate
• Remove-QADPrivateKey
• Get-QADCertificateRevocationList
• Add-QADCertificateRevocationList
• Import-QADCertificateRevocationList
• Export-QADCertificateRevocationList
• Remove-QADCertificateRevocationList
• Get-QADPKIObject
• Publish-QADCertificate
• Unpublish-QADCertificate
• Publish-QADCertificateRevocationList
• Unpublish-QADCertificateRevocationList
• Add-QADProxyAddress
• Set-QADProxyAddress
• Remove-QADProxyAddress
• Clear-QADProxyAddress
• Enable-QADEmailAddressPolicy
• Disable-QADEmailAddressPolicy
• Set-QADProgressPolicy
• Get-QADProgressPolicy
• Set-QADInactiveAccountsPolicy
• Get-QADInactiveAccountsPolicy

With tons of new parameters and bug fixes. All details can be found under ‘ARMS Build History’ text file under the zip file.

http://www.quest.com/powershell/activeroles-server.aspx

Also Dmitry Sotnikov tweeted regarding the updated cmdlet references wiki :

http://wiki.powergui.org/index.php/QAD_cmdlets_reference

Create Active Directory Delegations Report with PowerGUI

Standard

Often times you need to analyze your existing permissions (delegations) on your AD Objects within your domain/forest, perhaps you have just taken over an administrative role over AD and would like to quickly surface information regarding what group and user accounts have certain rights across the board in a pertinent domain. With PowerGUI and Kirk Munro’s “Reporting” PowerPack, you can generate nicely formatted HTML files (that expand and collapse) for each object that has delegated permissions within AD.

1. Download PowerGUI 2.1 from here

2. Get the Advanced Reporting PowerPack from here

3. Launch PowerGUI and import Advanced Reporting PowerPack

ImportPowerPack

4. Click on the root node, go to New and click on the ‘Script Node’ sub-menu option

CreateScriptNode

5. Name your script in the Title bar and type the following cmdlet in the body of the script

Get-QADObject -Type organizationalUnit -SecurityMask dacl | Get-QADPermission

 

TypeCmd

6. By hitting OK the report will run. From the Action Pane (right) click on the ‘Create Report’ link, name the report and add the desired attribute you would like to export on the report

Report

IncludeReport

7.  Hit OK and and your HTML based report will be saved by default in your DocumentsPowerGUI Exports folder.

ReportHTML

You can create all sorts of reports from your AD, do any modifications to your scripts, the format how the Report Pack creates the HTML report and how it generates the data. Download PowerGUI and the Reporting PowerPack and start playing with it.