As described in the AtlasCamp Europe 2012 talk on plugin quality, there are a number of patterns of bugs that we see with plugins.

Here are the valid user scenarios that usually lead customers into encountering these common bugs.

Data

  • Non-English text
    • Especially character encoding around AJAX calls.

    • If you don't have other keyboards set up on your test machine, one quick way to get good test data is to copy-paste it from the Wikipedia pages in that language.

  • Special characters
    • Ones that are important to HTML, eg. ' " < > &
    • Ones that are important to JSON, e.g. " { } ,
    • Ones that are important to regular expressions, e.g. ] [ ) ( ^
  • Large data sets
    • Lots of projects, issues, spaces, pages, users... whatever data the plugin interacts with.
    • Anything that's paged, make sure there's at least 2 pages of data.
  • Long strings
    • Make sure every value at least wraps to a new line, and every page is long enough to scroll.
    • The full text of Pride and Prejudice (legally obtained from Project Gutenberg) makes great test data - paste it everywhere!

Use Cases

  • Upgrades
  • Clean installations
  • Session timeouts
    • Easy to reproduce - just open another tab in the same browser and log out of the app there, then try the action in your plugin.
  • WebSudo timeouts
    • WebSudo is the extra login prompt you get when you try to do an admin action. Its timeout is much shorter than a session timeout (about 10 mins).
    • To reproduce,
      • First make sure you're not running in dev mode.
      • Do an admin action and log in at the WebSudo prompt.
      • Open another tab in the same browser and click the link in the banner to drop WebSudo access.
      • Then try whatever admin action you intended to perform.
  • Data being deleted/moved/edited
    • Data in the underlying app can and will disappear out from underneath you, as people use the system. 
  • Anonymous access 
    • Regardless of what action is being performed, the current user can be null.  

Client Environments

  • Web browsers
    • In general, Atlassian products support the two latest IE versions and the latest FireFox, Chrome and Safari.
    • Layout issues
    • Broken functionality
    • Client-side performance
      • Even though a feature may work in all browsers, in some older browsers it can be unusably slow with larger data sets, for example, if you're doing lots of client-side rendering. 
    • Behaviour in unsupported browsers
      • Even if the plugin doesn't support a particular browser, do a quick check of it:
        • To make sure the base application or other plugins are not broken.
        • To recognise the symptoms of how it behaves in the unsupported browser so you can recognise them quickly in customers' complaints.
  • Screen resolution
    • Try with 1024x768 to make sure none of the screens horizontally-scroll. 

Server Environments

  • HTTPS
    • Mixed content warnings are a usability killer due to frequent annoying popups in some browsers
    • Try the plugin with HTTPS to make sure it's not delivering HTTPS and HTTP content on the same page.
  • Different databases
    • Raw SQL is dangerous.
    • If it's absolutely necessary to write SQL(e.g. for an upgrade task), test across multiple database environments.
      • For example, even a simple "DELETE FROM my_table_name" will fail if the database is not using the default schema.
  • Running outside of dev mode
    • Different batching, minification rules
    • No WebSudo - what you see when you access admin screens is not the same as what admins see.
  • Context paths
    • By default in dev mode, most apps have a context path (e.g. "/jira"). It's common to experience bugs that only occur when there is no context path (e.g. http://jira.atlassian.com). 

 

  • No labels