The Jira or JSM mail handler fails to connect to the GMAIL server after some time when using Oauth 2.0
Platform Notice: Data Center - This article applies to Atlassian products on the Data Center platform.
Note that this knowledge base article was created for the Data Center version of the product. Data Center knowledge base articles for non-Data Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Since Jira 8.10.0 (or in Service Management 4.10.0), it has become possible to configure incoming Mail Handlers using the Oauth 2.0 Authentication instead of the Basic Authentication, which will be eventually deprecated by Google and Microsoft.
If a Jira or a Service Management (JSM) Mail Handler is configured using a GMAIL server with Oauth 2.0, the connection between the Jira application and GMAIL will fail after a certain amount of time (1h or 30 days later), due to 2 known bugs. This knowledge base article is meant to list the known bugs which are related to the GMAIL + Oauth 2.0 integration, how to identify them, and how to work around them.
Environment
- Jira Server/Data Center on 8.10.0 or any higher version
- Or Jira Service Management (JSM) on 4.10.0 or any higher version
Diagnosis
- An Oauth 2.0 integration was configured in the page below, with Google as the provider
- In ⚙ > System > Oauth 2.0 (for Jira versions 8.10.x-8.21.x)
- Or in ⚙ > System > Application links (for Jira versions 8.22.x+)
- An incoming Jira (or JSM) mail handler was configured using a GMAIL mail server along with the Oauth 2.0 integration:
- In ⚙ > System > Incoming Mail (in case of a Jira Mail Handler)
- In Project Settings > Email requests (in case of a JSM Mail Handler)
- If any the following observation is made, then Root Cause 1 is relevant
- The Mail Handler is configured with the SECURE_IMAP protocol
- The connection between Jira and the Mail Server fails about 30 days later
The following error is found in the file atlassian-jira-outgoing-mail.log:
2023-01-03 16:27:00,008+0000 ERROR [] Caesium-1-2 anonymous GMAIL Mail Handler [c.a.m.s.managers.jira.OFBizMailServerManager] OAuth2 token not found for id=<TOKEN_ID>.
- If any the following observation is made, then Root Cause 2 is relevant
- The Mail Handler is configured with the SECURE_POP protocol
- The connection between Jira and the Mail Server fails 1 hour later
The following error will is found in the file atlassian-jira-incoming-mail.log:
2023-01-12 08:40:00,261+0000 ERROR [GMAIL Server] Caesium-1-3 anonymous GMAIL Mail Handler GMAIL Mail Handler[10100]: Messaging Exception in service 'com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl' when getting mail: Open failed javax.mail.MessagingException: Open failed; nested exception is: java.io.IOException: STAT command failed: [AUTH] Invalid credentials. at com.sun.mail.pop3.POP3Folder.open(POP3Folder.java:220) [jakarta.mail-1.6.5-atlassian-2.jar:1.6.5-atlassian-2] at com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl.getAndProcessMail(MailFetcherService.java:160) [jira-api-8.20.15.jar:?] at com.atlassian.jira.service.services.mail.MailFetcherService.processMessages(MailFetcherService.java:388) [jira-api-8.20.15.jar:?]
If the debugging package com.atlassian.mail.auth is enabled with the DEBUG level, the following error will be found in the file atlassian-jira-incoming-mail.log:
2023-01-12 08:40:00,126+0000 DEBUG [GMAIL Server] Caesium-1-3 anonymous GMAIL Mail Handler DEBUG POP3: AUTH XOAUTH2 using one line authentication format 2023-01-12 08:40:00,227+0000 DEBUG [GMAIL Server] Caesium-1-3 anonymous GMAIL Mail Handler DEBUG POP3: AUTH XOAUTH2 failed, THROW: 2023-01-12 08:40:00,227+0000 DEBUG [GMAIL Server] Caesium-1-3 anonymous GMAIL Mail Handler java.io.EOFException: OAUTH2 authentication failed: {"status":"400","schemes":"Bearer","scope":"https://mail.google.com/"}
Cause
Root Cause 1
The Jira/JSM Mail Handler is configured using SECURE_IMAP, and we are facing the known bug linked below. Because of this bug, if a Mail Handler is using the combination GMAIL + IMAP + OAUTH 2.0, the Mail Handler will eventually fail to connect to the GMAIL mail server after 30 days, and the Jira admin will be forced to re-authorize the mail server from the UI.
JRASERVER-74666 - Getting issue details... STATUS
Root Cause 2
The Jira/JSM Mail Handler is configured using SECURE_POP, and we are facing the known bug linked below. Because of this bug, if a Mail Handler is using the combination GMAIL + POP + OAUTH 2.0, the Mail Handler will eventually fail to connect to the GMAIL mail server after 1h, and the Jira admin will be forced to re-authorize the mail server from the UI.
JRASERVER-74728 - Getting issue details... STATUS
Solution
Solution for Root Cause 1
Workaround 1 - Short Term workaround
After you observe that the Mail Handlers associated to the problematic GMAIL Server fail to fetch emails, do the following:
- Edit the Mail Server configuration in ⚙ > System > Incoming Mail
- Re-select the Oauth 2.0 integration in the authentication method
- Re-authorize the mail server, Re-test the connection, and save the configuration
Workaround 2 - Long Term workaround
A long term workaround is to force update the expiration date of the Refresh token to many years in the future, for example the year 2099 by using the Epoch Timestamp 4101054453000.
The steps are listed below, and need to be repeated for each Jira Mail Server that is already configured, and also to any Jira Mail Server that will be configured in the future:
Run the following SQL query after replacing <NAME> with the name of the Mail Server:
select * from mailserver where name = <NAME>;
- Check the auth_conf column and take the value from the tokenId field
Run the following SQL query after replacing <TOKEN_ID> with the Token ID obtained from the previous step:
update "AO_723324_CLIENT_TOKEN" set "REFRESH_TOKEN_EXPIRATION" = 4101054453000 where "ID" = <TOKEN_ID>;
Solution for Root Cause 2
Switch from the SECURE_POP to the SECURE_IMAP protocol.
Note that, by switching to the SECURE_IMAP protocol, even though the mail connection will no longer fail after 1h, it will instead fail after 30 days (due to the other bug mentioned in Root Cause 1). To prevent this other bug from occurring, please apply the workaround mentioned in the section Solution for Root Cause1 > Workaround 2 - Long Term workaround.