Dashboard loads slowly after upgrade to Jira server 7.1 and up

Still need help?

The Atlassian Community is here for you.

Ask the community

This article only applies to the Atlassian server platform. Learn more about the differences between cloud and server.

Problem

  1. Accessing dashboards in JIRA 7.1 or newer is taking a long time.
  2. Logging in using the dashboard is taking a long time. 

Diagnosis

Similar Symptoms

You may be experiencing the symptoms described in this knowledgebase article as well: https://confluence.atlassian.com/jirakb/how-to-fix-gadget-titles-showing-as-__msg_gadget-813697086.html

Environment

  • JIRA 7.1 or newer.
  • Affects new installations or upgraded installations.

Diagnostic Steps

  • Generating a HAR file while the dashboard is still loading and opening the HAR file in http://www.softwareishard.com/har/viewer/ , we are able to see an extremely long duration spent in the Receiving state for the HTTP request to load the dashboard:

 

  • In the example above, the dashboard took 36 seconds to load, and the entire time was spent in "Receiving" state.
  • By generating Thread Dumps during the slow dashboard load (one dump every 5 seconds is a good start) and doing an analysis of long-running threads with Thread Dump Analyzer (TDA), we can see the following thread stuck at attempting a connection to the JIRA base URL to retrieve gadget rendering data: 

    "http-nio-8080-exec-XX uri:/secure/Dashboard.jspa username:XXXXXXX" #152 daemon prio=5 tid=0x00007f34cc408000 nid=0x63f7 runnable [0x00007f34ab8cd000]
       java.lang.Thread.State: RUNNABLE
    	at java.net.PlainSocketImpl.socketConnect(Native Method)
    	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    	- locked <0x00000007a40a5db0> (a java.net.SocksSocketImpl)
    	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    	at java.net.Socket.connect(Socket.java:589)
    	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:337)
    	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
    	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
    	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    	...
    	at com.atlassian.gadgets.renderer.internal.http.HttpClientFetcher.fetch(HttpClientFetcher.java:85)
    	at org.apache.shindig.gadgets.DefaultMessageBundleFactory.fetchBundle(DefaultMessageBundleFactory.java:138)
    	at org.apache.shindig.gadgets.DefaultMessageBundleFactory.getNestedBundle(DefaultMessageBundleFactory.java:111)
    	...
    	at org.apache.shindig.gadgets.DefaultMessageBundleFactory.getBundle(DefaultMessageBundleFactory.java:79)
    	at org.apache.shindig.gadgets.variables.VariableSubstituter.substitute(VariableSubstituter.java:47)
    	at com.atlassian.gadgets.renderer.internal.GadgetSpecFactoryImpl.getGadgetSpec(GadgetSpecFactoryImpl.java:127)
    	at com.atlassian.gadgets.renderer.internal.GadgetSpecFactoryImpl.getGadgetSpec(GadgetSpecFactoryImpl.java:79)
    	at sun.reflect.GeneratedMethodAccessor1358.invoke(Unknown Source)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	...
    	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
    	at com.sun.proxy.$Proxy2003.getGadgetSpec(Unknown Source)
    	at com.atlassian.gadgets.dashboard.internal.impl.GadgetFactoryImpl.createSpecificationBasedGadget(GadgetFactoryImpl.java:120)
    	at com.atlassian.gadgets.dashboard.internal.impl.GadgetFactoryImpl.access$000(GadgetFactoryImpl.java:40)
    	at com.atlassian.gadgets.dashboard.internal.impl.GadgetFactoryImpl$1.visit(GadgetFactoryImpl.java:72)
    	at com.atlassian.gadgets.dashboard.internal.impl.GadgetFactoryImpl$1.visit(GadgetFactoryImpl.java:69)
    	at com.atlassian.gadgets.GadgetState.accept(GadgetState.java:132)
    	at com.atlassian.gadgets.dashboard.internal.impl.GadgetFactoryImpl.createDashboardItem(GadgetFactoryImpl.java:69)
    	at com.atlassian.gadgets.dashboard.internal.impl.StateConverterImpl.convertStateToGadget(StateConverterImpl.java:28)
            ...
    	at com.atlassian.gadgets.dashboard.internal.rest.representations.RepresentationFactoryImpl.createDashboardRepresentation(RepresentationFactoryImpl.java:48)
    	at com.atlassian.gadgets.dashboard.internal.velocity.DashboardEmbedder.dashboardToJsonObject(DashboardEmbedder.java:47)
    	at com.atlassian.gadgets.dashboard.internal.velocity.DashboardView.getLayoutsJson(DashboardView.java:164)
    	at com.atlassian.gadgets.dashboard.internal.velocity.DashboardView.writeTo(DashboardView.java:77)
    	at com.atlassian.jira.web.action.Dashboard$1.render(Dashboard.java:231)
    	at com.atlassian.jira.web.tags.RenderTag.doStartTag(RenderTag.java:37)

Cause

JIRA is having network issues while trying to connect to itself (its own base URL) to retrieve gadget data to render the Dashboard.

The reason for JIRA needing to connect to itself to get gadget data is for performance improvements. By offloading the processing to browsers instead of doing the rendering on the JIRA server, this effectively reduces load on the JIRA server itself. More details in https://confluence.atlassian.com/jirakb/how-to-fix-gadget-titles-showing-as-__msg_gadget-813697086.html

When there is a network issue where the JIRA server has trouble connecting to its own base URL, e.g. JIRA is hosted behind a Reverse Proxy, this can slow down the dashboard loading times.

Another root cause would be the following parameter configured on the setenv.sh file:

JVM_SUPPORT_RECOMMENDED_ARGS="-Dcom.atlassian.gadgets.dashboard.ignoreCache=true"

 

Resolution

Ensure that the JIRA server is able to connect to its own base URL and there is no JVM parameter for the gadget dashboard cache in the setenv.sh file. 

(warning) In case JIRA is able to connect to its IPv6 address but the problem persists, ensure that JIRA is able to connect to its IPv4 address as well. This is because IPv6 is currently not supported:  JRA-36676 - Getting issue details... STATUS

To confirm if this is working correctly, try running ping commands from the JIRA server to the base URL. 

(info) Please contact your network administrator if you are having issues getting this to work. Atlassian Support would be limited in what we can do to help with network problems specific to your environment.

 

 

Last modified on Sep 25, 2019

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.