Summary

• When a user tries to test the functionality of Rate Limiting in Jira, they notice that the X-Ratelimit-Remaining stays the same and believe that the functionality is not working

Environment

Jira 8.x , 9.x

Diagnosis

• Let us have Rate Limit Configuration like following with exemption for a user. Requests allowed for that user are set to 5 every 1 minute and Max requests that user can send are 15.

• Let us invoke Jira via REST operation in a loop(in interval of 1 second) with a counter of 18 which is more than Max requests configured in Rate Limit Configuration. 

• If we check results we would see that user can send 16 request which are success while Max requests is configured to 15 in a minute.

  ❯ for i in $(seq 18); do echo ---- $i ----;curl -sI -u cpandey:admin 'http://localhost:8080/rest/api/2/myself';sleep 1;done > header.txt
  
  ❯ grep -c "HTTP/1.1 200" header.txt
  16
  
  ❯ grep -c "HTTP/1.1 429" header.txt
  2

• Important Response header we should be aware of are following. These are well described in documentation Improving instance stability with rate limiting.

  

• Reason for this behavior is user is receiving a new token in constant interval. This constant interval is Time interval(in second) divided by Request allowed. Thus every 12 second user would receive a token which would allow him to send another request successfully.
• If we notice following headers, we find that X-RateLimit-Remaining response header for request 12 and 13 are same, which means that user received first token in 12 second since first request was invoked. Thus he is allowed to send one more request.
• Further if we analyse 16th request we find header retry-after is set to 8 second which mean user can try again in 8 second to make a successful request. Within 8 second all request will fail with HTTP status 429 for that user. 
• Another question could be why retry-after is set to 8 second in 16th request, answer is that token was received in 13th request and there is a difference of ~4 second between 13th and 16th request. Thus next token(or user will be allowed to send another success request) would be in 8 + 4 seconds.

  
  

  ❯ egrep "X-RateLimit-Remaining|HTTP|Retry*|Date|----*" header.txt
  ---- 1 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 14
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:22 GMT
  ---- 2 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 13
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:23 GMT
  ---- 3 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 12
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:24 GMT
  ---- 4 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 11
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:25 GMT
  ---- 5 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 10
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:26 GMT
  ---- 6 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 9
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:27 GMT
  ---- 7 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 8
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:28 GMT
  ---- 8 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 7
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:29 GMT
  ---- 9 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 6
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:30 GMT
  ---- 10 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 5
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:31 GMT
  ---- 11 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 4
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:32 GMT
  ---- 12 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 3
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:33 GMT
  ---- 13 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 3
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:34 GMT
  ---- 14 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 2
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:36 GMT
  ---- 15 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 1
  Retry-After: 0
  Date: Tue, 11 Apr 2023 13:03:37 GMT
  ---- 16 ----
  HTTP/1.1 200
  X-RateLimit-Remaining: 0
  Retry-After: 8
  Date: Tue, 11 Apr 2023 13:03:38 GMT
  ---- 17 ----
  HTTP/1.1 429
  X-RateLimit-Remaining: 0
  Retry-After: 6
  Date: Tue, 11 Apr 2023 13:03:39 GMT
  ---- 18 ----
  HTTP/1.1 429
  X-RateLimit-Remaining: 0
  Retry-After: 5
  Date: Tue, 11 Apr 2023 13:03:40 GMT

Cause

Since the rate limiting operates on a token per request basis and has a fill rate, you would get a additional token based on the fill rate if you back off and start over. For example, if you have the following setting,

X-Ratelimit-Fillrate: 30 
X-Ratelimit-Interval-Seconds: 60
X-Ratelimit-Limit: 30
X-Ratelimit-Remaining: 29

1. Now the fill rate is 30 per 60 seconds, essentially 1 token will be provided every 2 seconds when you back off for two seconds.
2. If you send all of them in a burst then you will exhaust your tokens and no more requests will be allowed and you would receive a 429.
3. If you back off for sometime, based on the fillrate you will start accumulating tokens to be used for request.
4. Irrespective of options of Seconds, Minutes & Hours, Jira Always translates time interval in seconds.
5. Rate Limit configuration Per time irrespective of options of Seconds, Minutes and Hours, Jira translates time interval in seconds i.e. token is sent to user in constant interval(in seconds) which allows him to send next successful request.
6. Rate Limit works on token bucket algorithm.
7. Also note that these Rate limit headers would only be available with authenticated requests, one would not be able to see these headers within response headers of anonymous user.

Solution

Testing whether rate limiting works before getting any traffic can be done as follows with the above rate limit settings

The first one below will reduce the counter and the last output would show 20 remaining after exhausting the 10 tokens in a burst of requests with no time for filling up of the token for request.

for i in $(seq 10); do curl -v -u user:pass http://localhost:8080/rest/api/2/myself ; done 

The second one below would have refilled the tokens resulting it in the same 29 remaining after the request every time.

for i in $(seq 10); do curl -v -u user:pass http://localhost:8080/rest/api/2/myself ; sleep 2;done