Crowd manages a cache of LDAP directory information stored in the Crowd database, to ensure fast recurrent access to user and group data. We call this 'database-backed LDAP caching'.

This page describes the caching of user and group information in the Crowd database. For a description of the other types of caching offered by Crowd, please refer to Overview of Caching.

On this page:

Features of LDAP Caching in Crowd

For all LDAP directories with caching enabled, Crowd will keep an up-to-date cache of user and group information retrieved from the LDAP directory. Use of the cache should improve performance particularly in directories which are slow or off site.

Please refer to the notes below, especially regarding the number of users for which the caching is optimized.

Summary of the caching functionality:

• The caches are held in the Crowd database.
• When you add the directory connector to Crowd, Crowd will start a synchronization task in the background to copy all the required users, groups and membership information from LDAP to the Crowd database. This task may take a while to complete, depending on the size and complexity of your user base.
• Crowd will perform a periodic synchronization to update the database with any changes made to LDAP. The default sync interval, or polling interval, is one hour (60 minutes). You can change the polling interval on the directory connector configuration screen.
• You can manually synchronize the database-backed cache if necessary.
• Whenever an update is made to the users, groups or membership information via Crowd, Crowd will update both the database-backed cache and the LDAP directory immediately.
• For all authentication requests, Crowd performs calls to the LDAP directory itself. The Crowd database-backed cache does not store user passwords.
• Crowd performs all other queries against the database-backed cache.

The diagram below gives a conceptual overview of the caches supported by Crowd, including the LDAP database-backed caching discussed on this page. For a description of the other types of caching offered by Crowd, please refer to the overview of caching.

Supported LDAP Directories

Crowd's database-backed caching is available for all the LDAP directories that Crowd supports. See Configuring an LDAP Directory Connector for the list of supported directories.

Configuring the Cache

1. In the top navigation, click Directories. 
2. Click on the directory for which you want to configure cache. 
3. In the Details tab, select Cache Enabled. 
4. In the Connector tab, set the polling interval option
   The polling interval is the period of time, in minutes, that Crowd will wait between its requests for updates from LDAP.

• • The length of your polling interval depends on the length of time you can tolerate stale data, the amount of load you want to put on Crowd and the LDAP server, and the size of your user base. If you poll more frequently, then your data will be more up to date. The downside of polling more frequently is that you may overload your LDAP server with requests.
  • If in doubt, we recommend that you start with an interval of 60 minutes (this is the default setting) and reduce the value incrementally. You will need to experiment with your setup.

Finding the time taken to synchronize

You can find the time take to synchronize on the Details tab of your directory. 

The directory connector's 'Details' tab shows information about the last sync operation, including the length of time it took.

Manually Synchronizing the Cache

Screenshot: Manually syncing the cache

You can manually synchronize the cache by clicking the Synchronize Now button on the the directory connector's Details tab. If a sync operation is already in progress, you cannot start another until the first has finished.

Notes

General Notes

1. Be aware of the optimal number of users. We have optimized the database caching for directories containing approximately 10 000 (ten thousand) users. If your directory is significantly larger, the new caching may not be as beneficial. For really large user bases, we recommend that you leave the caching disabled.
2. You can reduce the number of LDAP users visible to Crowd. You can narrow the LDAP user/group filter to control the size of the userbase visible to Crowd.
3. Delegated Authentication directories are not cached. Delegated Authentication directories are not cached, because only the authentication is delegated to the directory, and authentication itself is not cached.
4. Synchronization errors are shown in the logs. If there are any errors during the synchronization process, they will appear in the logs (not the UI). If one user fails to sync for some reason, the process will write the error to the logs, skip that user and continue with the remaining users.

Additional Notes for Active Directory

When Crowd synchronizes with Active Directory, Crowd requests only the changes from the LDAP server rather than the entire user base. This optimizes the synchronization process and gives much faster performance on the second and subsequent requests.

On the other hand, this synchronization method results in a few limitations:

1. Externally moving objects out of scope or renaming objects causes problems in AD. If you move objects out of scope, this will result in an inconsistent cache. We recommend that you do not use the external LDAP directory interface to move objects out of the scope of the sub-tree, as defined on Crowd's Directory Connector screen. If you do need to make structural changes to your LDAP directory, manually synchronize the directory cache after you have made the changes to ensure cache consistency.
2. Syncing between AD servers is not supported. Microsoft Active Directory does not replicate the uSNChanged attribute across instances. For that reason, Crowd does not support connecting a single directory to different AD servers for syncing.
3. You must restart Crowd after restoring AD from backup. On restoring from backup of an AD server, the uSNChanged timestamps are reverted to the backup time. To avoid the resulting confusion, you will need to flush the directory cache after a Active Directory restore operation.
4. Obtaining AD object deletions requires administrator access. Active Directory stores deleted objects in a special container called cn=Deleted Objects. By default, to access this container you need to connect as an administrator and so, for Crowd to be aware of deletions, you must use administrator credentials. Alternatively, it's possible to change the permissions on the cn=Deleted Objects container. If you wish to do so, please see this Microsoft KB Article.

Our Test Results

We performed internal testing of synchronization with an AD server on our local network consisting of 10 000 users, 1000 groups and 200 000 memberships.

We found that the initial synchronization took about 5 minutes. Subsequent synchronizations with 100 modifications on the AD server took a couple of seconds to complete.

Please keep in mind that a number of factors come into play when trying to tune the performance of the synchronization process, including:

• Size of userbase. Use LDAP filters to keep this to the minimum that suits your requirements.
• Type of LDAP server. We currently support change detection in AD, so subsequent synchronizations are much faster for AD than for other LDAP servers.
• Network topology. The further away your LDAP server is from your application server, the more latent LDAP queries will be.
• Database performance. As the synchronization process caches data in the database, the performance of your database will affect the performance of the synchronization.
• JVM heap size. If your heap size is too small for your userbase, you may experience heavy garbage collection during the synchronization process which could in turn slow down the synchronization.