How To Estimate FishEye Repository Index

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible


The purpose of this article is to provide you with steps to estimate the length of time it will take to index your FishEye repository.

This page is based on getting estimates using Git.

Step 1 - Identify Repository Size

The most important factors that determine slurp time are the total number of commits (reachable both from remote heads and tags).

Run the following commands to retrieve the corresponding values:

  • total commits reachable
  • number of paths changed in every commit
  • number of tags
  • number of paths for the most recent tag

$ git rev-list --all --count
$ git show-ref --heads --no-abbrev | wc -l
$ git show-ref -d --tags | wc -l
$ git ls-tree -r origin/<recent_tag> | wc -l

Step 2 - Identify the Number of Commits Fisheye Can Process Per Day

(info) This information is only available if DEBUG logging is enabled on Fisheye/Crucible. 

To enable debug logging

  •  Go to Administration > Global settings > Server
  •  Under Debug logging, click on Turn debug logging ON

Retrieve the FishEye logs and execute the following command:

grep "<- Processing" atlassian-fisheye-YYYY-MM-DD.log | awk 'BEGIN {total = 0}{total += $15}END{printf("%d commits processed in %dms total (%.1fms avg)\n", NR, total, total / NR);}'
1000000 commits processed in 75525706ms total (42.9ms avg)

Of which, we look for the unique commits that were processed:

grep "<- Processing" atlassian-fisheye-YYYY-MM-DD.log | awk '{print $13}' | sort | uniq | wc -l

Step 3 - Calculate Estimate

From the above, we now know that we have 1 million commits within our repository. Of those, only 50 thousand were able to be processed in a single day.  This is 5% of the total number of commits we need to be indexed. This means that it would take approximately 20 days to index this repository. 

Bonus - Check (Git) Index Progress

Fisheye doesn't provide any indication of a Git repository indexing progress via its UI, but one can hit a REST endpoint that returns a number of commits processed so far:

"name": "processor-sdk-linux-local",
"displayName": "processor-sdk-linux-local",
"state": "RUNNING",
"enabled": true,
"indexingStatus": {
"linesOfContentIndexingInProgress": false,
"indexingStateCounts": {
"UNKNOWN": 33958,
"initialScanningComplete": false,
"fullRepositorySlurpDone": false,
"incrementalIndexingInProgress": false,
"message": "Processing commit dc335d9735220b3a9ece5ec2d95864b1e8ff06a0",
"error": false,
"fullIndexingInProgress": true,
"crossRepositoryRescanInProgress": false

The number we're interested in is indexingStatus.indexingStateCounts.UNKNOWN and can be compared to a total number of commits in the repo (git rev-list --all --count) to get a rough estimate of indexing progress.

Last modified on Mar 5, 2024

Was this helpful?

Provide feedback about this article
Powered by Confluence and Scroll Viewport.