Missing groups from Crowd and/or embedded Crowd due to 1000 users/groups limit
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
After configuring an LDAP connector within Crowd or an application that uses the embedded Crowd component (ex. Confluence), only 1000 users/groups are synchronized even though there are more configured within LDAP.
Environment
- Confluence Server or Data Center
- Crowd Server or Data Center
- LDAP connector
Diagnosis
atlassian-confluence.log shows 1000 users and/or 1000 groups when synching with the external directory like the example below:
2020-05-28 13:10:43,590 INFO [CrowdUsnChangedCacheRefresher:thread-1] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 1000 ] remote users in [ 185ms ]
2020-05-28 13:10:43,596 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned and compared [ 1000 ] users for delete in DB cache in [ 3ms ]
2020-05-28 13:10:43,597 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedUsersNotIn scanned for deleted users in [ 4ms ]
2020-05-28 13:10:43,601 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] getUsersToAddAndUpdate scanning [ 1000 ] users to add or update
2020-05-28 13:10:43,606 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers scanned and compared [ 1000 ] users for update in DB cache in [ 8ms ]
2020-05-28 13:10:43,609 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addUsers adding [ 1000 ] users
2020-05-28 13:10:43,858 INFO [CrowdUsnChangedCacheRefresher:thread-2] [directory.ldap.cache.UsnChangedCacheRefresher] call found [ 1000 ] remote groups in [ 453ms ]
2020-05-28 13:10:49,348 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addUsers added [ 1000 ] users successfully in [ 5739ms ]
2020-05-28 13:10:49,356 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedUsers synchronised [ 1000 ] users in [ 5758ms ]
2020-05-28 13:10:49,363 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] deleteCachedGroupsNotIn scanned and compared [ 1000 ] groups for delete in DB cache in [ 5ms ]
2020-05-28 13:10:49,365 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedGroups scanning [ 1000 ] groups to add or update
2020-05-28 13:10:49,371 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] findGroupsToUpdate scanned and compared [ 1000 ] groups for update in DB cache in [ 6ms ]
2020-05-28 13:10:50,461 INFO [Caesium-1-3] [atlassian.crowd.directory.DbCachingRemoteChangeOperations] addGroups added [ 1000 ] groups successfully in [ 1086ms ]
2020-05-28 13:10:50,468 INFO [Caesium-1-3] [atlassian.crowd.directory.DirectoryCacheImplUsingChangeOperations] addOrUpdateCachedGroups synchronized [ 1000 ] groups in [ 1103ms ]In Crowd, similar logging appears like this:
2020-07-17 11:49:32,969 CrowdUsnChangedCacheRefresher:thread-2 INFO [directory.synchronisation.cache.UsnChangedCacheRefresher] found [ 1000 ] remote groups in [ 66ms ]
If Synchronise group memberships when logging in is set to Every time a user logs in, it is possible that users will not show up in the UI after the sync. After they login their user and groups will be synced as a separate query.
Cause
LDAPs (especially Active Directory) often limit the number of results per LDAP query to 1000.
Solution
To return more than 1000 results use paged results to break up the synch into multiple "pages":
- Go to Admin > User Management > User Directories > User Directories
- Find your LDAP directory and click Edit
- Go to Advanced Settings and check the option Use Paged Results, setting it to 1000 results
- Synchronize the directory
Notes
- The same issue happens in Jira (and presumably other Atlassian applications linked to LDAP that use the embedded Crowd library): Jira KB - LDAP only syncs 1000 users when should sync more
- If you deselect the Cache enabled setting for the directory and then perform a search for the missing group or user based on a given pattern, then a search result will most likely return as along as the result set is less than 1000. This is because Crowd or the embedded Crowd component will perform a search against AD using the search criteria that is provided and bypass its own cache.