Xcode support is undergoing testing and as such is unsupported by Atlassian at this time. If you would like to see it officially supported, please vote for this improvement request.

 

Prerequisites

  • Apple Xcode 4 – or later version
  • Certificates and provisioning profiles – You must install all required developer certificates and provisioning profiles on every machine that Bamboo will use to run your build. See the App Distribution Guide for more information.
  • Bamboo Xcode support plugin –  The latest Xcode plugin installed in your Bamboo server.
  • ios-sim (optional when building Mac applications) – a command line utility used to launch the iOS Simulator from the command line. If you have homebrew installed, you can install it by running brew install ios-sim. For other installation methods, see the ios-sim website.
  • Cocoapods (optional if you do not have a Podfile in your project) – Cocoapods is the library dependency manager for Mac OS X. In order for Bamboo to install dependencies from your Podfile (if you have created one), Bamboo will need it installed on all systems where the build should run.

Testing iOS applications

To have tests automatically run on the iOS Simulator and reported within Bamboo you must make some changes to your Xcode project's test bundles and add the Xcode build task to your Job within Bamboo.

Configuring your Xcode project automated simulator tests

Without modifications, Apple does not support running unit tests in the simulator using the xcodebuild terminal utility which Bamboo uses to automate builds and tests.

Using ios-sim and a small modification to the RunUnitTests script phase in the test bundle its possible to overcome this limitation.

 

 

Change the content of the script to:

if [ "$RUN_UNIT_TEST_WITH_IOS_SIM" = "YES" ]; then
test_bundle_path="$BUILT_PRODUCTS_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION"
ios-sim launch "$(dirname "$TEST_HOST")" --setenv DYLD_INSERT_LIBRARIES=/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection --setenv XCInjectBundle="$test_bundle_path" --setenv XCInjectBundleInto="$TEST_HOST" --args -SenTest All "$test_bundle_path"
echo "Finished running tests with ios-sim"
else
"${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"
fi

Configuring the Xcode task for testing

To configure a Xcode to test an iOS project task:

  1. Navigate to the Tasks configuration tab for the job (this will be the default job if creating a new plan).
  2. Click the name of an existing Xcode task, or click Add Task then Xcode to create a new task.

  3. Complete the following settings:
     

    Task DescriptionA description of the task, which is displayed in Bamboo.
    Disable this taskCheck, or clear, to selectively run this task.
    Apple SDK

    The Apple SDK to target during the build.

    Report test results

    Report and store any OCUnit/SenTestKit results run during the build.

    Run tests in iOS simulatorProvides the RUN_UNIT_TEST_WITH_IOS_SIM variable used in the custom build phase to run the unit tests on the simulator.
  4. Click Save.

Updating the available SDKs when Xcode is upgraded

When you upgrade Xcode you may need to update Bamboo with the correct SDK information.

If you use local agents:

  1. Login as an administrator
  2. Go to Administration -> Server Capabilities
  3. Click Detect server capabilities

If you are using remote agents:

  1. Run xcode-build -showsdks from the command line
  2. Login as an administrator
  3. In Administration -> Agents, pick the agent you wish for the new SDK capability to be present on
  4. Click Add Capability and pick Xcode SDK from the Capability Type field
  5. Set a name for the SDK (e.g. OS X 10.9)
  6. Set a SDK label (e.g. macosx10.9)

 

 

  • No labels

16 Comments

  1. b

    Oct 01, 2013

    I love the support for unit testing with xcode. Can you start investigating the XCTest framework in xcode 5? We are starting from scratch on unit tests and this is the framework provided by xcode 5.

    1. James Dumay

      Oct 02, 2013

      Hi Bob,

      Would you be able to email me some of the test output from XCTest? I'd like to take a look at it and see if I can build a parser. I am james at atlassian dot com.

      Thanks
      James 

      1. Andrew Herman

        Oct 16, 2013

        Just checking in here–did support for XCTest move forward?  Thanks!

        1. Anonymous

          Oct 16, 2013

          Just sent James some of the output.

           

          -Bob

        2. James Dumay

          Oct 17, 2013

          I started work on this today and was able to get some tests parsing. I still need some time to put together a Task and have the person who sent me the test output to approve me including them in the test code.

          If anyone has any extra XCTest output they can send me it would go a long way to ensuring that there are no bugs in the parser.

          1. Andrew Herman

            Oct 28, 2013

            Hey James, I actually pulled down your code on Friday, and built it into my own plugin–things are looking good, but I have 2 notes:

            1. It doesn't seem to be reporting Failed Results.  Explicitly, it detects that tests ARE failing, but doesn't report which.  If you contact me I can send you a bunch of logs/screenies/whatever you need.
            2. I discovered where the actual test results are stashed, which may be more reliable since they are formatted into a nice PLIST (XML).  I'd be happy to collaborate with you on how we could use this instead of parsing log output, but there may be some other issues here.

            I'd love to help out here.  I love the product, so I'm eager to see this updated for XCode 5 and XCTest.

             

            1. James Dumay

              Nov 22, 2013

              Andrew Herman if you could send me more information, my email is james@atlassian.com 

              Any help would be very appreciated (smile)

  2. Anonymous

    Nov 26, 2013

    Hi James,

    Can you please provide some date on when XCTest support will be available. We are planning to implement unit tests using XCTest framework. Need to update my manager with this info.

    Thanks.

  3. Matt Deir

    Dec 06, 2013

    This plugin works great for building iOS apps, but we are facing a challenge when trying to use the Test Flight task.  We've used the Test Flight API before without issue with other systems.

    I'm not sure if this is the appropriate place for raising this issue or not - I couldn't see anywhere on the bitbucket page to open one.

    When trying to post to the Test Flight API we are getting an error 400:

     

    simple	05-Dec-2013 20:39:55	Starting task 'Test Flight Upload' of type 'com.atlassian.bamboo.plugins.xcode.bamboo-xcode-plugin:testflight'
simple	05-Dec-2013 20:39:55	Uploading '/Users/matt/bamboo-agent-home/xml-data/build-dir/FT-ITASDF-JOB1/build/Debug-iphoneos/Test.ipa' to TestFlightApp...
error	05-Dec-2013 20:40:05	POST response to 'http://testflightapp.com/api/builds.json' was 400
simple	05-Dec-2013 20:40:05	Finished task 'Test Flight Upload'

     

    As per the Test Flight api -> https://testflightapp.com/api/doc/ the URL is http://testflightapp.com/api/builds.format , but from my interpretation of the code on bitbucket -> https://bitbucket.org/atlassian/bamboo-xcode/src/4565f0a2a6b167e635adfd4556623f5a4fd46657/src/main/java/com/atlassian/bamboo/plugins/xcode/testflight/TestFlightTask.java?at=default#cl-28 the URL is now pointing here -> http://testflightapp.com/api/builds.json .  I'm not sure if this is related or not.  Is there a way to turn on more verbose logging to see what exactly the task is trying to post?

     

    Thanks,

    Matt.

     

     

    1. James Dumay

      Dec 11, 2013

      The appropriate place to file feature requests is on jira.atlassian.com.

      Ill ping the TestFlightApp team to see if their API changed recently.

  4. Ben

    Jan 08, 2014

    Hi,

    I've got unit testing working well with Xcode 5 and iOS 7 using the SenTestingKit framework.

    I'm now trying to unit test a cocoapod in bamboo using the same technique. However it doesn't seem to work.

    I've created the project as a static library so there is no app to install into the simulator when unit testing and I think this is where is fails. What's confusing is that in Xcode the unit tests work fine.

    Is there a a way to unit test static libraries with Bamboo?

    Thanks,

    Ben

  5. Ben

    Jan 10, 2014

    Hi again,

    I've been messing about with the Xcode command line tools to try and understand this a bit more.

    Just wondering why ios-sim is required at all? Building and testing using xcodebuild on the command line seems ok and remove any requirement for ios-sim. What advantage does using ios-sim bring?

    If the final bamboo Xcode plugin could be built without this dependency that would simplify it somewhat, no?

    Thanks,

    Ben

    1. b

      Jan 14, 2014

      If your iOS unit tests are running then Xcode is using the simulator. Check: make sure the sim is quit, then run the tests. The sim opens.

      1. Ben

        Jan 14, 2014

        ios-sim is an open source os-x app that launches the iOS simulator. It's not the simulator itself. According to the instructions on this page, it needs to be downloaded and installed for the post build script (also provided above) to launch the simulator and run the tests.

        My point was that you can build and test in the simulator directly using xcodebuild without using ios-sim. So my question is about why ios-sim and the post build script are required at all. Not having to using this would make the build configuration a lot simpler.

        1. James Dumay

          Jan 14, 2014

          At the time, it was difficult to set up Xcode 4 to run tests in the simulator. I believe this is now easier with Xcode 5 and we will need to revisit it.

          1. Ben

            Jan 15, 2014

            Thanks for the update, James.