Jira Mail Gateway

Deprecated

http://confluence.atlassian.com/display/JIRAEXT/Jira+Extendable+Mail+Handler provides more features and is more actively developed.

(Peter Burkholder, April 2009)

Description/Features

A SOAP-based mail handler with support for issue handling and external users.  (Compare to Jira Extendable Mail Handler)
It's designed as a replacement for Jira's IMAP/POP or FileService mail handling and has the following advantages over those systems:

1. No delay: Messages are processed as soon as they arrive at the mail gateway machine. For our work network, the delay between client send and Jira receipt is a couple of seconds
2. Instant Feedback: Mail that can't be processed generates error messages over SMTP. If the Jira server is down, the mail handler returns a TEMPFAIL which will cause the delivery agent to retry the message later. If the message is malformed (e.g. unknown user) then the sender is informed of the problem. Jira's mail handling, in constrast, just leaves unprocessable messages in the delivery mailbox, or sends them to the administrator (another headache for you).
3. Command support: Issue creation and comment messages can include issue attributes (e.g. Assignee, IssueType) or commands (e.g. Resolve).
4. Mail support for Developer-level comments

The primary downside is that installation requires the following (detailed in the install section):

1. a patch to the SOAP RPC library
2. mailer configuration to support piped messages in 'alias'
3. Ruby 1.8.5+ and several Ruby gems (jira4r, tmail and, optionally, spec)

Jira Mail Gateway shares many project goals with Jira Extendable Mail Handler, and was developed independently around the same time. This project is in production at the National Institutes of Health, and test-driven development has resulted in 105 (and counting) tests for the core mail-handling code.

The documentation on this page is still incomplete. If you're comparing this project to Jira Extendable Mail Handler, I suggest you leave a comment here on the issue tracker requesting more background.

User Documentation

Jira Mail Gateway provides the end-user with the ability to create issues and comment on them, and to include issue commands and attributes.

As with Jira's built-in mail handling, new issues are created from messages with no issue key in the subject line and comments to issues have the issue key included (typically this is done automatically when replying to a message with your mail client).

You can include commands and attributes in the first lines of a mail message. The Jira Mail Gateway parser recognizes commands that start with the '@' (at) symbol as the first character on a line. Values are given after a ':' (colon) or '=' (equal) symbol. Commands are not recognized after any non-blank lines that aren't of the '@command:' format.

The requirement for the '@' symbol will go away in the next release

The current version of Jira Mail Gateway supports the following commands:

• @Assignee:username
  • Assign issue to userusername(alias: '@Assign').
• @Reporter:username
  • Set issue reporter to userusername. Useful when forwarding a message to Jira. By default, the reporter is the username preceding the '@' symbol of the From: address of the mail message
• @Summary:summary text
  • Set new 'Summary' attribute (alias: '@Subject')
• @IssueType:named_issue_type
  • Set the issue's type, e.g.Bug,New Feature,Improvement(aliases: '@Issue Type', '@Type')
• @Component:project component
  • Add a project component to an issue. Multiple components can be added with numbered component commands as '@Component1:', '@Component2:', as in the examples below.
• @Priority:priority_name
  • Set the issue priority, e.g.Blocker,Major,Trivial@Resolve:status_nameTransition the issue to 'Resolved' with the indicated status, e.g.Fixed,Not a Bug. Note that you can create and resolve a new issue with one email message; handy when you want to document that something was noted, and fixed. (alias: '@Resolved')
• @Custom_Field:field value
  • Set the text-based custom field value.

Differences from Jira's mail handling

1. Quoting. Messages with the quote '>' (greater_than) character at the start of each line keep the first level of quotation. This can make for some long comments when replying carelessly, but lets one comment directly on mailed text.
2. Attachments. Attachments are added by the 'mailgateway' Jira user instead of the sender. This is required by the addAttachment bug, and will be rectified with the 3.13.3 release.
3. Developer-level comments: The jira/mail administrator can set up a mail alias such as project-comments and call the maildumper script with the -D option to add comments with developer-level only visibility (Enterprise edition of Jira)

Configuration comparison

The FileService (and I presume the IMAP/POP Service) configuration takes three step:
1) create the alias
2) create the INBOX/directory
3) set up the Jira Services to collect the mail

With the mailgateway, all you need to do is set up the alias to pipe into the maildumper.rb script, with the appropriate command switches. If a project needs an issue_type or quote_level set, that currently needs to be added to
the config.yaml, but that will be a command-line option in a future release.

Mishandled mail

Another feature of the mailgateway is that mail doesn't get 'stuck' in the inbox when it can't be processed. For instance, I had with the old service two messages still in the inbox. jjjj

When you run them the mailgateway:

cat 10705 | /export/home/JIRA/local/jira_mailgw/maildumper.rb -p NB -e production

and look at logs/mailgateway.log, you see that :

W, [2008-12-04T22:00:56.564334 #20450]  WARN -- : SOAP::Fault error: FAILED. When trying to add message to Jira, got...

com.atlassian.jira.rpc.exception.RemoteAuthenticationException: Invalid username or password.

That's because the Jira user is 'lastname', and he's sent his mail as
fnamelastname'. Instead of mail just being stuck, the user can now either fix the problem, or ask for help.

User Examples

Start a new issue for the 'Propulsion Systems' and 'Jet Car' components, and set the custom field value for 'Fuel System' to 'Oscillation Overthruster'. The reporter will be set to 'buckaroo' based on the 'From:' field of the email header. The description is the body of the message after the commands:

To: jira-rocket@yoyodyne.com
From: buckaroo@yoyodyne.com
Subject: Insufficient lift

@Assignee: hikita
@Type: Bug
@Priority: Blocker
@Component1: Propulsion Systems
@Component2: Jet Car
@Fuel System: Oscillation Overthruster

I can't reach the 8th dimension

Resolve Buckaroo's issue. Jira Mail Gatweay will keep the '>' quoted line in the comment.

To: jira-rocket@yoyodyne.com
From: hikita@yoyodyne.com
Subject: [JIRA] Created: (RCKT-118) Insufficent lift

@Resolved: Fixed

> I can't reach the 8th dimension
You can now.  Thanks for pointing this out.  Good luck

New Features and Bug Reports

If you would like to report a bug or suggest a new feature (or vote on ones already in the system), please browse the JiraMailGateway project at http://developer.atlassian.com/jira/browse/JMGW

Configuration

JMGW is configured via a YAML file. A sample file:

# The top level elements designate your various jira environments.  maildumper.rb defaults
# to using 'preview' unless the --environment option used

production:
    jira_server:     http://127.0.0.1:8080

    # username of the mailgateway service account.
    # this account must have the allowImpersonate: yes property
    jira_user:       service_account
    password:        secret

    # Ruby Logging
    # log stream is a file path or STDOUT
    log_stream:     /home/JIRA/logs/mailgateway.log

    # Probably want Logger::INFO or Logger::WARN
    log_level:      Logger::INFO
    fqdn:           jira.example.com
    smtp_server:    smtp.example.com
    smtp_port:      25

    # Projects default to JiraMessage::DefaultIssueType (2, usually)
    # projects only need to be defined if the defaults aren't appropriate
    projects:

External Users

If a user is not recognized it's possible on a per-project basis to accept mail via a proxy user and map the original mail address to a custom field.

Install

You'll need the Ruby prerequisites, below.

• Untar the package on the mail server that will handle sending mail to Jira. This could be localhost, or your central mail gateway.
• Configure the config.yaml file based on the settings in config.yaml.sample
• Make sure the maildumper.rb works with a sample input file. See that file for usage
• Set up the mail aliases. E.g.:

tkt-jiraproject:      "|/export/home/JIRA/local/jira_mailgw/maildumper.rb
  -p JPROJ -B /export/home/JIRA/JMGBACKUP -e production"

tkt-jiraproject-comment:      "|/export/home/JIRA/local/jira_mailgw/maildumper.rb
  -p JPROJ -B /export/home/JIRA/JMGBACKUP -e production --developer-only"

Ruby Requirements

• soap4r
• rspec
• tmail
• Jira4r
• get the source from http://svn.rubyhaus.org/jira4r/trunk
• Follow the README.txt to build the gem and install it

Jira Modifications

Jira needs a patch to the RPC plugin to let the Jira Mailgateway user insert comments as other users. See e.g. http://jira.atlassian.com/browse/JRA-13176
http://jira.atlassian.com/browse/JRA-12522

Run the rpc/Makefile with  (make) to build the new jar.
Note that you'll need maven installed.

Jira Configuration

1. Enable the SOAP RPC
2. Create a mailgateway user
3. To that user, add the property: +AllowImpersonate: true+
4. To all relevent *permission schemes*, allow the mailgateway user to +Modify Reporter+ If there's just one permission scheme, that should only take a moment.

Backup and Reporting

The mailgateway script, maildumper.rb, uses a mail backup directory. All messages are saved first to /export/home/JMGBACKUP/pending. If there's an error in processing, they are moved to JMGBACKUP/error, and on success, moved to JMGBACKUP/done.

I have a script in JIRA/local/jira_mailgw/util/jmg_report that tries to match up the files in the backup directory to corresponding log entries. 'jmgreport.rb' will report on yesterday's error, 'jmgreport -t' will report on today's. It's an ugly hack, but suffices for keeping tabs on what the mailgateway has been up to. It does give false postiives.

I've been running jmgreport out of cron every night. You might want to also.

License

With the exception of certain third-party files (see ), this software is a "United States Government Work" under the terms of the United States Copyright Act. It was written as part of the authors' official duties as United States Government employees and thus cannot be copyrighted. This software is freely available to the public for use. The National Library of Medicine and the U.S. Government have not placed any restriction on its use or reproduction.

Although all reasonable efforts have been taken to ensure the accuracy and reliability of the software and data, the NLM and the U.S. Government do not and cannot warrant the performance or results that may be obtained by using this software or data. The NLM and the U.S. Government disclaim all warranties, express or implied, including warranties of performance, merchantability or fitness for any particular purpose.

Please cite the authors in any work or product based on this material.

Author: Peter Burkholder <pburkholder@pobox.com> Contact: Jira Mail Gateway <tkt-jmg@ncbi.nlm.nih.gov>

Version History

Open Issues