Missing groups from Crowd and/or embedded Crowd due to 1000 users/groups limit
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
After configuring an LDAP connector within Crowd or an application that uses the embedded Crowd component (ex. Confluence), only 1000 users/groups are synchronized even though there are more configured within LDAP.
Environment
Confluence Server or Data Center
Crowd Server or Data Center
LDAP connector
Diagnosis
atlassian-confluence.log shows 1000 users and/or 1000 groups when synching with the external directory like the example below:
atlassian-confluence.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
2020-05-28 13:10:43,590 INFO [CrowdUsnChangedCacheRefresher:thread-1] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 1000 ] remote users in [ 185ms ]
2020-05-28 13:10:43,596 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned and compared [ 1000 ] users for delete in DB cache in [ 3ms ]
2020-05-28 13:10:43,597 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned for deleted users in [ 4ms ]
2020-05-28 13:10:43,601 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] getUsersToAddAndUpdate scanning [ 1000 ] users to add or update
2020-05-28 13:10:43,606 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers scanned and compared [ 1000 ] users for update in DB cache in [ 8ms ]
2020-05-28 13:10:43,609 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addUsers adding [ 1000 ] users
2020-05-28 13:10:43,858 INFO [CrowdUsnChangedCacheRefresher:thread-2] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 1000 ] remote groups in [ 453ms ]
2020-05-28 13:10:49,348 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addUsers added [ 1000 ] users successfully in [ 5739ms ]
2020-05-28 13:10:49,356 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers synchronised [ 1000 ] users in [ 5758ms ]
2020-05-28 13:10:49,363 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedGroupsNotIn scanned and compared [ 1000 ] groups for delete in DB cache in [ 5ms ]
2020-05-28 13:10:49,365 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedGroups scanning [ 1000 ] groups to add or update
2020-05-28 13:10:49,371 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] findGroupsToUpdate scanned and compared [ 1000 ] groups for update in DB cache in [ 6ms ]
2020-05-28 13:10:50,461 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addGroups added [ 1000 ] groups successfully in [ 1086ms ]
2020-05-28 13:10:50,468 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedGroups synchronized [ 1000 ] groups in [ 1103ms ]In Crowd, similar logging appears like this:
atlassian-crowd.log
1
2020-07-17 11:49:32,969 CrowdUsnChangedCacheRefresher:thread-2 INFO [directory.synchronisation.cache.UsnChangedCacheRefresher] found [ 1000 ] remote groups in [ 66ms ]ℹ️ If Synchronise group memberships when logging in is set to Every time a user logs in, it is possible that users will not show up in the UI after the sync. After they login their user and groups will be synced as a separate query.
Cause
LDAPs (especially Active Directory) often limit the number of results per LDAP query to 1000.
Solution
To return more than 1000 results use paged results to break up the synch into multiple "pages":
Go to Admin > User Management > User Directories > User Directories
Find your LDAP directory and click Edit
Go to Advanced Settings and check the option Use Paged Results, setting it to 1000 results
Synchronize the directory
Notes
The same issue happens in Jira (and presumably other Atlassian applications linked to LDAP that use the embedded Crowd library): Jira KB - LDAP only syncs 1000 users when should sync more
If you deselect the Cache enabled setting for the directory and then perform a search for the missing group or user based on a given pattern, then a search result will most likely return as along as the result set is less than 1000. This is because Crowd or the embedded Crowd component will perform a search against AD using the search criteria that is provided and bypass its own cache.
Was this helpful?