Storing backups in Amazon S3
Before you begin
If you're considering using Amazon S3 for storing backups, first read through the configuration requirements and current limitations to make sure this storage method is suitable for you.
Learn more about Amazon S3 configuration
How S3 backup storage is set up in Jira
Backup storage in Amazon S3 is configured in the filestore-config.xml file that should be located in Jira <localhome>.
To use S3 as a target location for backup data, the filestore attribute in the filestore-config.xml must match the s3-filestore ID.
If you’re already storing avatars in S3, you can configure the filestore-config.xml file to store backups there as well. To do this, add a new association element with the target backups.
If you want to store backups in the same bucket as avatars, the filestore attribute should point to the same bucket where avatars are located.
To use separate buckets for backups and avatars, define multiple <s3-filestore> elements and reference each of them in the respective association targets.
If you want to use a Jira <sharedhome> directory to store backups but don’t want to delete the configuration file, just remove the association element with a target of backups. Jira will default to storing backups locally if S3 isn't configured in filestore-config.xml or if this file is missing from the <localhome> directory.
Connecting to Amazon S3 dual-stack endpoints
If your Jira installation is on an IPv6-only network, you need to connect to Amazon S3 dual-stack endpoints. Learn more about Amazon S3 dual-stack endpoints
To set up the connection, you need to override the default endpoint with the dual-stack endpoint like this:
<?xml version="1.1" ?>
<filestore-config>
<filestores>
<s3-filestore id="backupBucket">
<config>
<bucket-name>dualstack-bucket</bucket-name>
<region>us-east-1</region>
<endpoint-override>https://s3.dualstack.us-east-1.amazonaws.com</endpoint-override>
</config>
</s3-filestore>
</filestores>
<associations>
<association target="backups" file-store="backupBucket" />
</associations>
</filestore-config>You can also use this configuration option to store backups in a third-party object store that exposes an S3-compatible API. However, we’re not providing direct support for backups that are stored in an object store other than Amazon S3.
Configure Amazon S3 to store backups
If you’re ready to proceed with Amazon S3 configuration, follow the instructions from the Configuring Amazon S3 as your data storage method.
Connect your S3 bucket with Jira
After you configure S3 object storage, you need to connect the created S3 bucket with your Jira instance:
In the Jira application home directory of one of your Jira installation nodes, create a
filestore-config.xmlfile. The Jira application home directory should be set to the value of theJIRA_HOMEenvironment variable.
Learn about the contents of the Jira application home directory
In thefilestore-config.xmlfile, define which S3 bucket will be used by Jira to store backups and avatars.
Learn more about storing avatars in S3
Samplefilestore-config.xmlfile:<?xml version="1.1" ?> <filestore-config> <filestores> <s3-filestore id="backupBucket"> <config> <bucket-name>example-co-jira-backup-bucket</bucket-name> <region>ap-southeast-4</region> </config> </s3-filestore> </filestores> <associations> <association target="backups" file-store="backupBucket" /> </associations> </filestore-config>If you’re running a clustered installation, copy the
filestore-config.xmlfile to the Jira application home directory of the other nodes.Start or restart all Jira nodes.
When Jira starts up, it will check your bucket configurations, such as bucket connectivity, name and region validity, and bucket permissions. Learn about potential errors and how to fix them
To verify that Jira is using Amazon S3 object storage:
In the upper-right corner of the screen, select Administration > System.
Under IMPORT AND EXPORT settings (the left-side panel), select Backup system.
Next to the “The backup file will be placed here”, you'll see that backups are stored in S3. Here you can also check the S3 bucket name and region.\
Restore process
For restoring, the backup file needs to be manually downloaded from the S3 bucket and put into <sharedhome>/import directory. Learn how to restore data from an XML backup
Write-only mode
We recommend granting access to the S3 bucket in write-only mode, which means that application has only access to write new files to S3 Storage.
Here is an example of how Identity and Access Management (IAM) policy provides appropriate permissions:
{
"Version": "2012-10-17",
"Id": "PolicyForS3Access",
"Statement": [
{
"Sid": "StatementForS3Access",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:user/JiraS3"
},
"Action": [
"s3:PutObject"Additionally, we recommend enabling versioning in this bucket to avoid backup file overriding.
Jira every hour for every node tries to create, read, and finally remove small files (~60 bytes) from S3 storage to confirm connectivity. With write-only mode, Jira won’t be able to remove files, which leads to the accumulation of data (less than 1 MiB yearly per node).
To avoid this:
- Remove regularly all files from
{backupBucket}/backups/Healtchecklocation. Add Object Lifecycle configuration.
<LifecycleConfiguration> <Rule> <ID>Remove Health checks files</ID> <Filter> <Prefix>backups/Healthcheck/</Prefix> </Filter> <Status>Enabled</Status> <Expiration> <Days>5</Days> </Expiration> </Rule> </LifecycleConfiguration>
Troubleshooting S3 backup storage
When Jira starts up, it runs a series of checks to make sure there are no issues with the filestore-config.xml file. If there are any errors during the file parsing, Jira won't start and will display an error message.
If there are problems connecting to or performing operations on S3, Jira will also detect them and flag the Backup Filestore instance health check as failing.
The following sections list the problems that can happen during S3 configuration along with the resolution steps. The issues are mainly related to improper S3 configuration, permissions, or authentication.
Jira startup failures
You can also find more details about the problems by checking Jira logs at <localhome>/log/atlassian-jira.log. Learn how to access Jira logs
| Problem | Root cause and resolution |
|---|---|
Couldn’t parse
| The Ensure that the file consists of valid XML. |
Invalid
| The If you provide an |
Missing
| An |
Health check failures
Learn more about the instance health checks and how they are performed
| Problem | Root cause and resolution |
|---|---|
Unable to write to backup storage
| The
|
Able to read data from backup storage
| To increase security, we recommend using write-only mode. The
|
Able to delete data from backup storage
| To increase security, we recommend using write-only mode. The
|
Mitigating S3 connection issues for slow or unreliable networks
Jira uses the Amazon S3 CRT asynchronous client to connect to Amazon S3.
The Amazon S3 CRT-based client allows controlling multiple simultaneous threads for a single connection for read and write operations. A Jira admin can configure the client settings to mitigate upload and download failures if the Jira server’s internet connection is unstable or slow.
The Amazon S3 CRT-based client allows tuning several parameters, the most important of which is max-concurrency. By default, max-concurrency is set to 100 threads. It’s been introduced to the Jira filestore configuration filestore-config:
<?xml version="1.1" ?>
<filestore-config>
<filestores>
<s3-filestore id="backupBucket">
<config>
<bucket-name>dualstack-bucket</bucket-name>
<region>us-east-1</region>
<max-concurrency>100</max-concurrency>
</config>
</s3-filestore>
</filestores>
<associations>
<association target="backups" file-store="backupBucket" />
</associations>
</filestore-config>The growing number of concurrent connections can increase client throughput. This also implies that your application has to manage more resources simultaneously.