Installing Insight

Make sure that Jira is configured to handle the amount of data you plan to import into Insight. If you plan to use a lot of scheduling tasks and import a large amount of data we recommend that you follow the table below.

Remember to always test the memory consumption in a test environment for a huge data set, because it's not always the number of objects, but the content of the object attributes as well at play. 

On large installations of Insight, there are some options you can tweak to increase performance. 

Garbage collection

To decrease the pause time we recommend that you switch the garbage collection (GC) algorithm to G1 from the default. You can configure it by adding the following JVM argument to your Jira setenv script.

-XX:+UseG1GC

Insight parallelism

Based on your environment, other options to tune the G1 may be required. Check out the information from Oracle (http://www.oracle.com/technetwork/tutorials/tutorials-1876574.html) and tune your JVM based on your requirements. 

Insight executes tasks in parallel (e.g. reindex, imports). On a large instance, you can increase the number of threads that perform the tasks. You can see this here: Configuring global Jira settings. By default, the parallelism is configured to be equal to the amount of cores available to the JVM. 

Database pool

If you have increased the number of threads(Insight parallelism), it is recommended to check the database connection pool monitor and determine if an increased connection pool is required as well.

Jira shutdown

On Jira shutdown Insight persists the index on disk for faster startup. In large instances, the index file fails to be saved on disk in time before the force shutdown command is sent. This will cause Insight to do a database reindex on each startup. To prevent this change, you need to change the timeout on force shutdown in the stop-jira script.

Change the value 20 (the unit is seconds) to a value that is more reasonable based on the amount of data in your installation. Below is a snippet from the standard Jira stop-jira.sh and it is the value 20 that should be changed at two places.

if [ -z "$JIRA_USER" ] || [ $(id -un) == "$JIRA_USER" ]; then
    echo executing as current user
  
    exec $PRGDIR/shutdown.sh 20 -force $@
  
elif [ $UID -ne 0 ]; then
  
    echo JIRA has been installed to run as $JIRA_USER so please sudo run this to enable switching to that user
    exit 1
  
else
  
    echo executing using dedicated user
    if [ -x "/sbin/runuser" ]; then
        sucmd="/sbin/runuser"
    else
        sucmd="su"
    fi
    $sucmd -m $JIRA_USER -c "$PRGDIR/shutdown.sh 20 -force $@"
  
fi

If you have configured your heap with the CATALINA_OPTS in setenv, make sure that the JVM configuration (JVM_MAXIMUM_MEMORY) is not the same value as the max heap configured in CATALINA_OPTS. 

If you are using Insight to accommodate millions of objects and plan to use importers heavily, (e.g, with the Insight Discovery product),  then we recommend that you move to Data Center version if you are not already using it. This will offer you the possibility to isolate the importers to only one import node with the other nodes being the user nodes. With one import node, the import process will have less impact on user interaction . It will also be more robust since any failing import or any over consumption of memory will only affect the import node and not any of your user nodes.

It is difficult for us to set any exact object size or import size recommendation since it's about the data stored on your objects, as well as how often the imports occur. But in general, when reaching millions of objects or memory usage over 32 Gb or if you don't ever want to impact any user interaction, then you should consider using the Data Center version of Insight.  

During imports and re-index with Data Center, we need to send messages to other nodes to update the index. This is done through provided Atlasssian functionality and the clustermessage table is the DB table where all nodes push and pull. The problem we have seen is that Atlasssian have a retention period of 30 days for this table. We think this is a bit too much and recommend that you should remove all Insight related data rows which are older than 24 hours with your own retention scripts.