Missing groups from Crowd and/or embedded Crowd due to 1000 users/groups limit

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform Notice: Server and Data Center Only - This article only applies to Atlassian products on the server and data center platforms.

 

Summary


After configuring an LDAP connector within Crowd or an application that uses the embedded Crowd component (ex. Confluence), only 1000 users/groups are synchronized even though there are more configured within LDAP.

Environment

  • Confluence Server or Data Center
  • Crowd Server or Data Center
  • LDAP connector

Diagnosis

atlassian-confluence.log shows 1000 users and/or 1000 groups when synching with the external directory like the example below:

atlassian-confluence.log
2020-05-28 13:10:43,590 INFO [CrowdUsnChangedCacheRefresher:thread-1] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 1000 ] remote users in [ 185ms ]
2020-05-28 13:10:43,596 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned and compared [ 1000 ] users for delete in DB cache in [ 3ms ]
2020-05-28 13:10:43,597 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned for deleted users in [ 4ms ]
2020-05-28 13:10:43,601 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] getUsersToAddAndUpdate scanning [ 1000 ] users to add or update
2020-05-28 13:10:43,606 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers scanned and compared [ 1000 ] users for update in DB cache in [ 8ms ]
2020-05-28 13:10:43,609 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addUsers adding [ 1000 ] users
2020-05-28 13:10:43,858 INFO [CrowdUsnChangedCacheRefresher:thread-2] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 1000 ] remote groups in [ 453ms ]
2020-05-28 13:10:49,348 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addUsers added [ 1000 ] users successfully in [ 5739ms ]
2020-05-28 13:10:49,356 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers synchronised [ 1000 ] users in [ 5758ms ]
2020-05-28 13:10:49,363 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedGroupsNotIn scanned and compared [ 1000 ] groups for delete in DB cache in [ 5ms ]
2020-05-28 13:10:49,365 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedGroups scanning [ 1000 ] groups to add or update
2020-05-28 13:10:49,371 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] findGroupsToUpdate scanned and compared [ 1000 ] groups for update in DB cache in [ 6ms ]
2020-05-28 13:10:50,461 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addGroups added [ 1000 ] groups successfully in [ 1086ms ]
2020-05-28 13:10:50,468 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedGroups synchronized [ 1000 ] groups in [ 1103ms ]

In Crowd, similar logging appears like this:

atlassian-crowd.log
2020-07-17 11:49:32,969 CrowdUsnChangedCacheRefresher:thread-2 INFO [directory.synchronisation.cache.UsnChangedCacheRefresher] found [ 1000 ] remote groups in [ 66ms ]

(info) If Synchronise group memberships when logging in is set to Every time a user logs in, it is possible that users will not show up in the UI after the sync. After they login their user and groups will be synced as a separate query.

Cause

LDAPs (especially Active Directory) often limit the number of results per LDAP query to 1000. 

Solution

To return more than 1000 results use paged results to break up the synch into multiple "pages":

  • Go to Admin > User Management > User Directories > User Directories
  • Find your LDAP directory and click Edit
  • Go to Advanced Settings and check the option Use Paged Results, setting it to 1000 results
  • Synchronize the directory

Notes

  • The same issue happens in Jira (and presumably other Atlassian applications linked to LDAP that use the embedded Crowd library): Jira KB - LDAP only syncs 1000 users when should sync more
  • If you deselect the Cache enabled setting for the directory and then perform a search for the missing group or user based on a given pattern, then a search result will most likely return as along as the result set is less than 1000.  This is because Crowd or the embedded Crowd component will perform a search against AD using the search criteria that is provided and bypass its own cache.


Last modified on Jul 22, 2020

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.