Users can't login to Stash - LDAP response read timed out

Miscellaneous

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

Symptoms

Users are unable to login to Stash.

The following appears in the atlassian-stash.log:

2014-08-26 22:26:35,892 ERROR [clusterScheduler_Worker-2]  c.a.c.d.DbCachingDirectoryPoller Error occurred while refreshing the cache for directory [ 229377 ].
com.atlassian.crowd.exception.OperationFailedException: Error looking up attributes for highestCommittedUSN
	at com.atlassian.crowd.directory.MicrosoftActiveDirectory.fetchHighestCommittedUSN(MicrosoftActiveDirectory.java:847) ~[crowd-ldap-2.7.2.jar:na]
	at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseChanges(UsnChangedCacheRefresher.java:80) ~[crowd-ldap-2.7.2.jar:na]
	at com.atlassian.crowd.directory.DbCachingRemoteDirectory.synchroniseCache(DbCachingRemoteDirectory.java:993) ~[crowd-core-2.7.2.jar:na]
	at com.atlassian.crowd.manager.directory.DirectorySynchroniserImpl.synchronise(DirectorySynchroniserImpl.java:75) ~[crowd-core-2.7.2.jar:na]
	at com.atlassian.crowd.directory.DbCachingDirectoryPoller.pollChanges(DbCachingDirectoryPoller.java:50) ~[crowd-core-2.7.2.jar:na]
	at com.atlassian.stash.internal.crowd.DirectoryPollerJobRunner.runJob(DirectoryPollerJobRunner.java:71) [stash-service-impl-3.2.0.jar:na]
	at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:127) [atlassian-scheduler-core-1.2.2.jar:na]
	at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:101) [atlassian-scheduler-core-1.2.2.jar:na]
	at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:80) [atlassian-scheduler-core-1.2.2.jar:na]
	at com.atlassian.scheduler.quartz1.Quartz1Job.execute(Quartz1Job.java:32) [atlassian-scheduler-quartz1-1.2.2.jar:na]
	at org.quartz.core.JobRunShell.run(JobRunShell.java:223) [quartz-1.8.6.jar:na]
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) [quartz-1.8.6.jar:na]
	... 13 frames trimmed
Caused by: org.springframework.ldap.UncategorizedLdapException: Uncategorized exception occured during LDAP processing; nested exception is javax.naming.NamingException: LDAP response read timed out, timeout used:120000ms.; remaining name '/'
	at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:217) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	at org.springframework.ldap.core.LdapTemplate.executeWithContext(LdapTemplate.java:810) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	at org.springframework.ldap.core.LdapTemplate.executeReadOnly(LdapTemplate.java:793) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	at org.springframework.ldap.core.LdapTemplate.lookup(LdapTemplate.java:822) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	at com.atlassian.crowd.directory.ldap.LdapTemplateWithClassLoaderWrapper$3.call(LdapTemplateWithClassLoaderWrapper.java:88) ~[crowd-ldap-2.7.2.jar:na]
	at com.atlassian.crowd.directory.ldap.LdapTemplateWithClassLoaderWrapper.invokeWithContextClassLoader(LdapTemplateWithClassLoaderWrapper.java:54) ~[crowd-ldap-2.7.2.jar:na]
	at com.atlassian.crowd.directory.ldap.LdapTemplateWithClassLoaderWrapper.lookup(LdapTemplateWithClassLoaderWrapper.java:85) ~[crowd-ldap-2.7.2.jar:na]
	at com.atlassian.crowd.directory.MicrosoftActiveDirectory.fetchHighestCommittedUSN(MicrosoftActiveDirectory.java:823) ~[crowd-ldap-2.7.2.jar:na]
	... 12 common frames omitted
Caused by: javax.naming.NamingException: LDAP response read timed out, timeout used:120000ms.
	at com.sun.jndi.ldap.Connection.readReply(Connection.java:483) ~[na:1.7.0_67]
	at com.sun.jndi.ldap.LdapClient.getSearchReply(LdapClient.java:639) ~[na:1.7.0_67]
	at com.sun.jndi.ldap.LdapClient.search(LdapClient.java:562) ~[na:1.7.0_67]
	at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1985) ~[na:1.7.0_67]
	at com.sun.jndi.ldap.LdapCtx.doSearchOnce(LdapCtx.java:1934) ~[na:1.7.0_67]
	at com.sun.jndi.ldap.LdapCtx.c_lookup(LdapCtx.java:1028) ~[na:1.7.0_67]
	at com.sun.jndi.toolkit.ctx.ComponentContext.p_lookup(ComponentContext.java:544) ~[na:1.7.0_67]
	at com.sun.jndi.toolkit.ctx.PartialCompositeContext.lookup(PartialCompositeContext.java:177) ~[na:1.7.0_67]
	at javax.naming.InitialContext.lookup(InitialContext.java:415) ~[na:1.7.0_67]
	at org.springframework.ldap.transaction.compensating.manager.TransactionAwareDirContextInvocationHandler.invoke(TransactionAwareDirContextInvocationHandler.java:92) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	at org.springframework.ldap.core.LdapTemplate$9.executeWithContext(LdapTemplate.java:824) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	at org.springframework.ldap.core.LdapTemplate.executeWithContext(LdapTemplate.java:807) ~[spring-ldap-core-1.3.1.RELEASE.jar:1.3.1.RELEASE]
	... 18 common frames omitted

Cause

There can be multiple causes for this error:

  1. The LDAP directory is too huge and Stash fails to find the user before the timeout.
  2. There have been cases where the enabled 'Follow Referral' option causes the same behavior.
  3. Stale connections are not being closed and when used the next time, they immediately fail
    1. If you're seeing authentication attempts immediately fail as opposed to after the 120000ms timeout, this could be the cause
    2. An improvement for this is being tracked at  CWD-4297 - Getting issue details... STATUS

Resolution

Increase LDAP Read Timeout

  1. Go to Administration > User Directories
  2. Edit the LDAP directory
  3. Increase the value of Read Timeout

Disable Follow Referral

  1. Go to Administration > User Directories
  2. Edit the LDAP directory
  3. Disable the Follow Referral option

Set a Timeout to Automatically Close Stale LDAP Connections in the Pool

  1. Stop Stash
  2. Modify <Stash Installation>/bin/setenv.sh to add the following parameter to the JVM_SUPPORT_RECOMMENDED_ARGS:

    JVM_SUPPORT_RECOMMENDED_ARGS="-Dcom.sun.jndi.ldap.connect.pool.timeout=300000"

     

    1. This will timeout idle LDAP connections from the pool after 5 minutes and help clear out any stale connections from being reused
  3. Start Stash
Last modified on Mar 30, 2016

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.