How to programmatically archive Jira Projects with a Bash Script

Platform Notice: Cloud and Data Center - This article applies equally to both cloud and data center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Summary

For long-lived Jira Sites, with many Projects & much activity, performance issues can arise due to data that would best be archived.

This article talks about how to programmatically archive Jira Projects using a Bash Script. The reason this is needed is because the API Endpoint that Archive Projects only takes 1 Project at a time.

Due to this, bulk archival can be laborious without software to cycle through the Projects that need to be archived.

The script in this KB Article can do this, as well as offering a few quality of life features for making that process easier.

The shell commands utilized in this script are:

  1. curl - Making Web Requests
  2. jq - Processing JSON Payloads
  3. The basics: echo, tee, printf, read, base64, sed, let, wc, tr, awk, shift, getopts, & cat.
  4. Other conventions like: case, if, logical operators, & for.

All of those must be available in the terminal environment for the script to work.

Environment

Jira Cloud or Jira Datacenter - Either work so long as the Script is adjusted to point to the appropriate Endpoints.

Solution

First, I will post the script in its entirety here in a collapsed code block. Then I will give a brief overview of its functionality, followed by an analysis of what it achieves.

Click here to expand...
ArchiveProjects.sh
#!/bin/bash
# This script is provided as-is. It utilizes either a CSV or Site & Date/Time zone information to archive projects programmatically.
# The CSV File should have 1 row and that row should contain all projects separated only by commas. Project IDs or Project Keys are acceptable.
# BR,ONKEY,SD,PROJ or 10000,10301,10402,11003 are examples.
# Alternatively, provide a Date & Time zone and the Script will get all Projects with a lastIssueUpdateTime before that.
# "3 days ago" & "US/Pacific" are examples. The Time zone is not required and will default to your time zone.
# The File Option & the Date/Time zone Option are mutually exclusive, and if both are provided, the File will be deferred to.

# Base Variables...
# Default argument conflict
argConflict=false
# Default Log File
logFile="./ArchiveProjects.log"
# Default Time zone
TZ=$TZ
# Verbose Logging
_V=0
# Handling Exits when needing to check other variables first.
exitTime=0

# Credentials...
# User Email the API Token is for. Replace <john@doe.com> with something like name@domain.com.
# Remove the <>.
apiEmail="<john@doe.com>"
# API Token generated from https://id.atlassian.com/manage-profile/security/api-tokens.
# Replace <API-Token> with the string provided from there.
# Remove the <>.
apiToken="<API-Token>"
# Processes the Email & Token into a base64 String for Basic Authentication to the REST API.
# The -w 0 is to remove the linebreaks base64 creates.
authBasicHeader=$(echo -n "$apiEmail:$apiToken" | base64 -w 0)

# URI Variables
# https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-projects/#api-rest-api-3-project-search-get
searchUri="rest/api/3/project/search"
# We will replace the {projectIdOrKey} with the information we gather. It is a placeholder.
# https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-projects/#api-rest-api-3-project-projectidorkey-archive-post
archiveUri="rest/api/3/project/{projectIdOrKey}/archive"

usage() { # Basic Usage Information.
	echo "Usage: $0 [-h] [-v] <-f csv file> [-t time zone] <-d relation date> <-s site>"
	echo "Either the -f flag with a CSV file path (./keys.csv) or the -d flag with a date time relation (3 months ago) must be provided."
	echo "Options:"
	echo " -h  Help - This text."
	echo " -v  Verbose Logging - Makes it so that more information is provided as the script runs."
	echo " -f  File Path - Use this or Date as one of the two are required. The file must be a CSV with a single row of comma-separated Project Keys or IDs. It must have a .csv extension."
	echo " -t  Time zone - Optional. This will default to your system time zone. Either in country/region format like US/Pacific or time zone name with offset like UTC+5."
	echo " -d  Date Cutoff - Use this or File as one of the two are required. Provided in this format: \"6 years ago\" relative to now. | See: https://man7.org/linux/man-pages/man1/date.1.html#DATE_STRING for more info. Example: 3 months ago"
	echo " -s  Site URL - In this format: https://[site].atlassian.net | replace [site] with the actual name such as: https://johndoe.atlassian.net"
}

check_proceed() { # This checks if you want to proceed via User Input.
	read -p "Are you sure? Y/N "$'\n' -n 1 -r # Asks User if they want to continue.
	printf "\n" # Fixes post-script entry
	if [[ ! $REPLY =~ ^[Yy]$ ]]; then # If the character typed anything but Y or y, exit.
		verbose "Canceling process. User responded with $REPLY."
		[[ "$0" = "$BASH_SOURCE" ]] && exit 1 || return 1
	fi
}

verbose() { # Handles Verbose Logging by echoing the string provided to Terminal & also logging it to the Log File.
	if [[ $_V -eq 1 ]]; then
		echo $1 | tee -a "$logFile"
	fi
}

# Arguments & Flags checking ...
# If no Arguments & Flags are provided, exit.
[[ -z "$@" ]] && usage && exit 1
# If no File or Date are provided, exit. One of either is required.
[[ "$@" != *"-f"* && "$@" != *"-d"* ]] && usage && exit 1

while getopts "h?vf:t:d:s:" opt; do # Handling the Arguments & Flags. There should at least be 2 since Site is required and either File or Date is required.
	case "$opt" in
		h|\?) # Help
			usage
			exit 0
		;;
		v) # Verbose Mode Handling.
			echo "Verbose Mode On."
			_V=1
		;;
		d) # Date Cutoff - Time Comparison Variable for API process.
			 # Examples: "3 years ago", "1653 minutes ago", "21 weeks ago", etc.
			process="API"
			flagDate="$OPTARG"
			[[ "$@" == *"-f"* ]] && argConflict=true # If the -f flag is provided, alert regarding the conflict.
			verbose "Flag Date Relation: $OPTARG"
		;;
		f) # File provided - must be a CSV with a single row and the Keys or IDs for the Project separated by only commas. Must also be accessible.
			process="CSV"
			fileCSV="$OPTARG"
			[[ ! -f $OPTARG ]] && echo "File does not exist, or it is inaccessible." && exitTime=1
			[[ "$@" == *"-t"* || "$@" == *"-d"* ]] && argConflict=true # If the -t or -d flags are provided, alert regarding the conflict.
			verbose "File realpath: $(realpath $OPTARG) | Entered string: $OPTARG"
		;;
		t) # Time zone - See here for options: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
			 # Examples: US/Pacific; country/region, UTC+5; time zone+offset
			TZ="$OPTARG"
			[[ "$@" == *"-f"* ]] && argConflict=true # If the -f flag is provided, alert regarding the conflict.
			verbose "Time zone: $OPTARG"
		;;
		s) # Site Information - Applies validation and prepares to exit if the input fails to validate.
			 # Examples: https://site.atlassian.net and all variations of that domain name with other subdomains.
			siteUrl="$OPTARG"
			[[ "$siteUrl" != "https://"*".atlassian.net" ]] && echo "Invalid input to site: \"$siteUrl\" is not a usual Atlassian Site." && exitTime=1
			verbose "Site: $OPTARG"
		;;
	esac
done
[[ $exitTime == 1 ]] && exit 1

# Build the URL to Archive the Projects
archiveUrl="$siteUrl/$archiveUri"
# Build the URL to Search the Projects. expand=insight is required. orderBy is not.
searchUrl="$siteUrl/$searchUri?expand=insight&orderBy=lastIssueUpdatedTime"

authorization_check() { # This verifies the API credentials will work.
	response=$(curl -s --request POST \
		--url "$(sed 's/{projectIdOrKey}/INVALIDPROJECT1234567890DONOTUSE/g' <<< $archiveUrl)" \
		--header 'Accept: application/json' \
		--header "Authorization: Basic $authBasicHeader")
	# If the response mentions archival, then authentication worked.
	# A Project can only have a Key up to 10 characters long, so the above is never going to be valid.
	[[ "$response" == *"You cannot archive this project."* ]] && echo "Credentials valid, proceeding."
	# If the response mentions authentication, then authentication failed.
	if [[ "$response" == *"You are not authenticated."* ]]; then
		echo "Credentials failed. Check your API Token at https://id.atlassian.com/manage-profile/security/api-tokens."
		echo "If the API Token shows activity, that's not the issue. If it doesn't show activity, verify it is correct."
		echo "If you generated your API Token in a different location, such as the Org Admin panel, it will not work."
		verbose "Credentials failed: Check API Token and that it maches the User: $apiEmail." && exit 1
	fi
}

# Logging Date Time
verbose "Script Run Time: $(date)"
# Logging User for API Authentication | Will not log Token or base64 string for security
verbose "API User: $apiEmail"
# Must have Site URL. - -z returns true if length of string is zero.
[[ -z $siteUrl ]] && echo "Site Required via -s Argument." && usage && exit 1
# Argument Conflict Alert.
[[ $argConflict == "true" ]] && echo "Using -d for date cutoff & -f for file conflict. This script will ignore it, deferring to File."
# Shift through the provided Arguments.
shift $((OPTIND-1))

# Gets the final cutoff date using the Time zone & Date relation. Defaults to System Time zone.
# -n returns true if length of string is non-zero.
[[ -n $flagDate ]] && cutoffDate=$(TZ=$TZ date -d "$flagDate" +%Y-%m-%dT%H:%M:%S.%3N%z)
# Logs the CSV File Path if provided.
[[ -n $fileCSV ]] && process="CSV" && verbose "CSV File: $(realpath $fileCSV)"

# Final flight checks...
# If the Date returned is not valid, setup exit time. -z returns true if length of string is zero.
[[ -z $cutoffDate ]] && [[ -z $fileCSV ]] && echo "Invalid input to date: \"$flagDate\" resulted in no selected Date. Try something like \"30 hours ago\"." && exitTime=1
# If the File is not a valid CSV File, setup exit time. -n returns true if length of string is non-zero.
[[ -n $fileCSV ]] && [[ "$fileCSV" != *".csv" || $(cat "$fileCSV"| wc -l) -gt 1 ]] && echo "Invalid file: $fileCSV is not a +valid+ CSV. It must end in .csv & have a single line of comma-separated Project keys or ids." && exitTime=1
# If exit time, exit.
[[ $exitTime == 1 ]] && exit 1
# If both cutoffDate & flagDate are not zero, log them.
[[ -n $cutoffDate && -n $flagDate && "$process" == "API" ]] && verbose "Calculated Date: $cutoffDate ($flagDate)"

# Checks against the Project Archival Endpoint to verify Authorization via API Credentials.
authorization_check

case $process in
	API)
		# Gets the 1st page.
		search=$(curl -s --request GET \
			--url "$searchUrl" \
			--header 'Accept: application/json' \
			--header "Authorization: Basic $authBasicHeader")
		verbose "RAW Search: $search"
		# Gets the lastPage boolean from the curl response.
		lastPage=$(jq '.isLast' <<< "$search")
		# Get the Total results from the JSON.
		total=$(jq '.total' <<< "$search")
		# Grab values from the JSON as an Object Array [{{},{},{}}]
		search=$(jq '.values' <<< "$search")
		# Count the pages.
		iteration=1
		# Starting Page count, the maxResults defaults to 50.
		resultCount=50
		echo "Total: $total"
		verbose "Parsed Search: $search"
		verbose "Page $iteration pulled. Results: ${total:-0}. Last Page: $lastPage."
		# Reminder occurs when the initial search returns 0 results. This can happen when authorization fails.
		# Or when there just aren't any Projects in the given timeframe. Either/or could be true.
		# Check your API Token at https://id.atlassian.com/manage-profile/security/api-tokens
		if [[ $search == "[]" ]]; then # If it shows activity, that's not the issue. If it doesn't show activity, verify it is correct.
			echo "Reminder: This API can be accessed Anonymously, but it will return 0 results if done so."
			echo "Verify API Token is actually valid & authenticating." | tee -a "$logFile"
		fi

		while ! $lastPage; do # Pagination cycling if the initial search is not the last page.
			# Iterative variable for counting pages.
			(( iteration++ ))
			pageRaw=$(curl -s --request GET \
				--url "$searchUrl&startAt=$resultCount" \
				--header 'Accept: application/json' \
				--header "Authorization: Basic $authBasicHeader")
			# Combines the two json.
			search=$(echo -e "$search\n$(jq '.values' <<< "$pageRaw")" | jq -s 'add')
			# Capture the boolean's value.
			lastPage=$(jq '.isLast' <<< "$pageRaw")
			# Add 50 to get the next page.
			let resultCount=$resultCount+50
			# Get total from 
			verbose "Page $iteration pulled. Results: ${total:-0}. Last Page: $lastPage."
			# 1 second between requests.
			sleep 1
		done
		
		# This variable contains the IDs of the Projects that have a lastIssueUpdateTime value before the provided cutoff Date for Archival.
		archiveProjectKeys=$(jq -r '.[] | select (.insight.lastIssueUpdateTime < "'$cutoffDate'") | .key' <<< "$search")
		verbose "Project Keys: ${archiveProjectKeys:-None}"
		
		# This turns something like ABC\nXYZ\nGHI into ABC,XYZ,GHI for visibility/review.
		formattedProjects=$(tr -s '\r\n' ',' <<< "$archiveProjectKeys" | sed -e 's/,$/\n/')
		# This counts the number of projects that will be archived.
		countedProjects=$(wc -w <<< "$archiveProjectKeys")
		# If 0 Projects would be archived, let the User know and exit.
		[[ $countedProjects == 0 ]] && echo "REST API returned 0 Projects that fell in the timeframe before $cutoffDate ($flagDate) for Site $siteUrl." && exit 1
		# It's go time.
		echo "Archiving $countedProjects of ${total:-0} Projects matching criteria: "$formattedProjects" sourced via REST API for Site: $siteUrl."
		check_proceed

		# https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-projects/#api-rest-api-3-project-projectidorkey-archive-post
		for archiveProjectKey in $archiveProjectKeys; do
			archiveResponse=$(curl -s --request POST \
				--url "$(sed 's/{projectIdOrKey}/'$archiveProjectKey'/g' <<< $archiveUrl)" \
				--header 'Accept: application/json' \
				--header "Authorization: Basic $authBasicHeader")
			verbose "$archiveResponse"
			# If the Response contains the string errorMessages...
			if [[ $archiveResponse == *"errorMessages"* ]]; then
				# Post failure to terminal, log, & exit.
				echo "Received non-200 response from Archive API Endpoint. Response: $archiveResponse" | tee -a "$logFile" 
				# The archival attempt failed.
				exit 1
			fi
			# Post success to terminal & log.
			echo "Successfully Archived Project: $archiveProjectKey" | tee -a "$logFile"
			# 1 second between requests.
			sleep 1
		done
	;;
	CSV)
		# Get the number of projects from the CSV.
		countedProjects=$(awk -F '[,]' '{print NF}' "$fileCSV")
		# It's go time.
		echo "Archiving "$countedProjects" Projects: "$(cat $fileCSV)" sourced from CSV: $fileCSV for Site: $siteUrl." | tee -a "$logFile"
		check_proceed

		IFS=',' read -ra archiveProjectKeys < "$fileCSV"
		# Cycle through the Project Keys from the CSV - may be IDs instead which would work as well.
		for archiveProjectKey in "${archiveProjectKeys[@]}"; do
			archiveResponse=$(curl -s --request POST \
				--url "$(sed 's/{projectIdOrKey}/'$archiveProjectKey'/g' <<< $archiveUrl)" \
				--header 'Accept: application/json' \
				--header "Authorization: Basic $authBasicHeader" | tee -a "$logFile")
			if [[ $archiveResponse == *"errorMessages"* ]]; then
				# Post failure to terminal, log, & exit.
				echo "Received non-200 response from Archive API Endpoint. Response: $archiveResponse" | tee -a "$logFile"
				# The archival attempt failed.
				exit 1
			fi 
			# Post success to terminal & log.
			echo "Successfully Archived Project: $archiveProjectKey" | tee -a "$logFile"
			# 1 second between requests.
			sleep 1
		done
	;;
	*)
		echo "Process not clearly defined. Exiting. Check with Atlassian Support for an investigation." | tee -a "$logFile" && exit 1
	;;
esac
echo "Finished. Closing Script."

Usage

This Script takes either a Date or a File. If given a Date, it will look up the Projects on the given Site, and any who lack Issues updated after the Date will be archived. If given a File, it will just archive the Projects in said File. Examples:

1. ./ArchiveProjects.sh -f ./Projects.csv -s "https://site.atlassian.net"
2. ./ArchiveProjects.sh -d "3 years ago" -s "https://site.atlassian.net"
3. ./ArchiveProjects.sh -d "1 year ago" -t "US/Pacific" -s "https://site.atlassian.net"
4. ./ArchiveProjects.sh -v -d "1 hour ago" -t "UTC+5" -s "https://site.atlassian.net"
5. ./ArchiveProjects.sh -h

The Script does require being updated with the credentials that will be used to access the Site (Email Address & API Token).
Note the permissions required by the User as mentioned here: Jira KB - Archive a project.

Optionally, the Time zone can be specified via -t if hours matter.

The 1st takes the file "Projects.csv" which should be a single line with comma-separated project keys (like ABC,XYZ,PROJ,ITSM) and archives those for the given site.
The 2nd gets the list of projects that don't have issues with updates newer than 3 years ago and archives those.
The 3rd does the same but for 1 year and uses the specified time zone, overriding your system time zone.
The 4th archives any projects that lack updates on their issues within the last hour and verbosely logs that process, and while overriding the system time zone.
The last just outputs the help usage text.

Analysis

The Script goes through this logic, in words:

  1. Establishes some baseline Variables like verbose logging, the logFile, etc along with setting up defaults.
  2. Creates a few functions we'll be utilizing to "make the magic happen":
    1. The first is just a Usage to explain how to use the script.
    2. The second is a simple check to make sure the user wants to proceed with the action being taken, based on the information provided to the script.
    3. The third is a simple verbose checker & logging for it.
    4. The fourth/last (after the option checking) verifies that the credentials are correct by making an API Request using them to see what is returned.
  3. There are a few lines to verify that the appropriate flags have been provided along with their values.
  4. The flags provided are then cycled through to get some setup done, such as setting some base variables to a value (like process=CSV or process=API), along with logging said values.
  5. After building the URL we'll be using, some final logging & checks are done along with some variables being set. Like the Site being provided and the Date being parsed.
  6. Lastly, we actually run the authorization check before diving into the cycling that makes this process work.

I'm going to separately go over the logic in the final case function because it is a bit in-depth:

  1. For the final case that actually archives the projects, we start with checking which Process we ended up going with between CSV or API. For API:
    1. We get the first page of Project results.
    2. We check if it's the last page, and set some variables based on what we received.
    3. We do some simple checking to make sure we didn't receive 0 results and warn if we did.
    4. We cycle through grabbing more pages if the first page isn't the only page, building the results from there until we get to the last page.
    5. We process those results into a simple JSON Object of the Keys only, then format the content into a comma-separate string of keys.
    6. We get a count of the total projects returned, do another check to make sure that returned >0, then do a final check to make sure the user wants to proceed with archiving what was returned.
    7. The For cycles through each Key in the list, sending the request to archive it, checking for an error message, and stating the result.
  2. For CSV:
    1. We get a count of projects from the provided CSV.
    2. We do a check to make sure the user wants to proceed with archiving what was returned from processing the CSV.
    3. The For cycles through in the same manner as API, archiving the project, checking for an error, and stating the result.
  3. For neither of those, we error out and convey next steps.

Further Reading

Last modified on Oct 17, 2024

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.