See Integrate JIRA With A Test Case Management Tool for a list of tools JIRA integrates with. The discussion below is now out-dated.
Previous Discussion
There is no reasonably priced Test Case Management tool on the market - there is a big opportunity here to provide an obvious add on to Jira. Things like Mercury Test Director are enormously expensive when compared to Jira, and really, they dont do that much.
I work in a medium sized software firm - about 60 employees. We have a dedicated testing team, of about 6-10 people. They mostly do manual functional testing, with detailed test cases which are derived from Use Cases and other specs.
A couple of years ago we went down the path of writing our own test case management tool, because there was nothing reasonable in the market, either commercial or open source. We are now looking at redeveloping the tool, as it has maintainence problems. There is still nothing in the market.
Test case management has an obvious integration with defect/issue management, so any tool would need to integrate with Jira.
Much better still would be if it was within Jira. It is the same users, projects, and generally there is a One-To-Many between test case and defect.
There are three parts to this - creating test cases, executing them, and reporting on them.
Its pretty simple really - A project has many test cases, which may be linked to a Use Case (Issue type? - Jira for Requirements Management as well?) and maybe a version.
Each test case has mutiple steps, which have an index and a description, plus some other minor fields.
You need a nice effecient interface to create these steps.
Each test case can have many runs, or executions.
When running a test case, a tester is presented with another UI that efficiently allows them to work through the steps of the test case, and mark each step as completed, or not. Where a step cannot be completed a defect may be created, which is automatically linked to the the run.
The other crucial thing is a set of reports that allows a project manager or test manager to see how many and which tests are passing, how many runs have been done, etc.
As I said, we are currently looking at redeveloping our test case tool, but would much rather use an externally developed product - and an add on to Jira would be the ideal solution. We could possibly help with the detailed requirements of such an addition to Jira. Please let me know if there is any interest.
Jon Nermut
Project Manager
Dytech Solutions
Hobart
btw, I have created a feature request for this: http://jira.atlassian.com/browse/JRA-11059
Comments (4)
Oct 05, 2007
Matt Doar says:
Notes on Using Jira for Test Tracking These rough notes describe how I impleme...Notes on Using Jira for Test Tracking
These rough notes describe how I implemented a test case and test result tracking system using Jira.
Note: the changes below mean that the Enterprise edition is necessary.
Tool Requirements
A test case manager must support the following requirements:
The following requirements are useful but not essential:
"If data is hard to enter, it won't be entered"
Existing Test Case Managers
Implementation Overview
Define a new issue type named "TestCase", with the following custom fields:
A new test case is defined by creating a new TestCase issue in Jira and filling in the required fields such as Summary, Description and Component.
For each release where this test is going to be run, a subtask of the issue is created and the Affects Version field is set to the release. The priority of the test for this release can also be added here.
When a test is run, the results are entered or updated in the subtask,not in the parent TestCase issue. Comments about the test run can also be entered if appropriate.
When a report is created for the current state of a release, we search for all TestCase issues with the Affects Version field set to the required release. The resulting set of issues can then be sorted and counted by number passed, failed, not run etc. Producing historical reports can be done with the Timecharts plugin for Jira, which shows a graph of how the results in a report change over time.
We can also create a new link type to connect test cases to the bugs that they created or are verifying. This is the biggest advantage that I see of using an issue tracker to track test cases and bugs together. We could also add a box to all bugs to indicate whether a Test Case is expected by someone for a bug.
Example
Test Case TC-1234
------------------
Summary: Check that the Citrix installer works
Description: instructions on how to check that the installer works,
including the environment and other assumptions.
Links:
TC-1235
TC-1236
TC-1237
TC-1235
-------
Affects Version: 3.0
Date Last Run: 8/8/07
Build Run Against: 3.0.0.4322
State: PASSED
TC-1236
-------
Affects Version: 3.1
Date Last Run: 12/12/07
Build Run Against: 3.1.0.9992
State: FAILED
TC-1237
-------
Affects Version: 3.2
Date Last Run:
Build Run Against:
State: Not Yet Run
Example Report
Date: 12/15/07
Release: 3.1
Number of test cases: 200
Number of tests not run: 20 (10%)
Number of tests passed: 90 (50%)
Number of tests failed: 90 (50%)
I yhought about putting tests in their own project, but linking wouldn't be as good?
Field Changes
You could add a new tab for tests. The field changes I made were:
1) Test Case
Required: Summary, Component, Description, but not Affects Version or Finder
No Affects or Fixed Version
Assignee is person who defines the test
2) Test Result
Required: Summary, Component, Description, but not Finder
Component should be inherited from the parent test case, or not used?
Assignee is person who runs the test, or is this QA contact?
No Fixed Version for test result, just Affects Version
Resolution not used?
Add the following custom fields:
Date Last Run
Build
Custom issue icon images exist too somewhere on this wiki
Screens
Create a Field Configuration for each issue type
Create a new Field Configuration scheme and add each issue type with its Field Configuration
Associate the new Field Configuration scheme with the project PS.
Workflow
Add new statuses: Not Wanted or Inactive, Not Yet Run, Passed, Failed, Error, Needs Rerunning
The intent is that a test should Pass or Fail, and an Error is when you couldn't decide whether it passed or failed because of a problem with the test. Or, "error is when the test itself fails, not the device under test"
Add new workflow scheme
Add new workflow for TestResults
Define new workflow
To remove the default initial state of Open, edit the Open step, and
change the transition to use the new state
Associate a project with new workflow scheme, migrate (takes a couple of minutes)
This is hard to change while active!
Link Types
Test, "is a test for", "is tested by"
Running Reports
Choose the project and Issue Type.
Conclusion
This hasn't been through a release cycle yet, but is attractive since it:
Jun 15, 2007
Adam Sandman says:
Hi JIRA Community Based on the need identified in this forum, we at Inflec...Hi JIRA Community
Based on the need identified in this forum, we at Inflectra Corporation have just added a connector from our affordable (starting at $299.99) test management system - SpiraTest - to JIRA.
SpiraTest is a Test Management system that provides integrated Requirements, Test Case and Defect Management. With the connector activated, any defects logged during a test execution get routed to JIRA using the SOAP API. Any changes to these issues in JIRA then get synchronized back into SpiraTest using our SOAP API.
To learn more about this, please visit http://www.inflectra.com/SpiraTest
Regards
Adam Sandman
Director, Sales & Marketing
Email: sales@inflectra.com
Tel: 1-866-572-5878 (toll-free)
+1 202-558-6885 (outside US)
Aug 31, 2007
Charles West says:
Your post speaks to the very reason I created Qualify, a test case management so...Your post speaks to the very reason I created Qualify, a test case management solution that doesn't break the bank but competes with the core features provided by Rational and Mercury. Please take a minute to check it out at http://www.plan9software.com . Incidently, one of my clients just asked about integration with JIRA - I'm working on it. Expect integration by 2008!
Regards,
Charles West
Jul 14
Jite-staff says:
Jite Test Management Jite is a test management tool to manage and execute tests{...Jite - Test Management
Jite is a test management tool to manage and execute testsrelated to any kind of application or device (for example a software application, an electronic system, a logistic system or a process.) that needs to be tested and certificated before to issue it to the end costumer.
Main Features of Jite:
More info
http://www.prismasw.it/ita/jite/indexEng.html
jite@prismasw.it