How TestRail designed, built and deploys universal add-ons for JIRA Software Server & Cloud

Reading Time: 8 minutes

More and more teams adopt and migrate to JIRA Software Cloud to benefit from the various advantages of Atlassian’s hosted platform, such as easier distributed access, great performance, less maintenance hassle, no upfront costs, and automated updates. JIRA Cloud also provides great opportunities for add-on developers, as it offers better control of the user experience, easier upgrades, faster deployments, choice of development platform as well as recurring revenue.

At the same time there’s a huge existing JIRA Server customer base. If you are building a new tool or integration for JIRA, you ideally want to target both server and cloud customers to reach as many users as possible. So when we designed and planned our all-new JIRA test management integration for TestRail 5.0, one of our design goals was therefor to offer the same rich integration options to JIRA Cloud and Server customers. To make things even more complicated, we also offer TestRail server and cloud editions to choose from.

So how did we build the integration to work seamlessly with all possible integration combinations? We came up with a universal integration approach for both JIRA Server and Cloud. We noticed that the approach we designed works great for many kinds of tools, integrations and add-on ideas. So we decided to share our approach to help other developers overcome the same challenges we faced and I will explain the details of our approach in this article.

JIRA Cloud integration with Atlassian Connect

Building a JIRA Cloud integration is pretty straightforward by using the Atlassian Connect platform. Especially for TestRail, but also for a lot of other application integrations and add-on ideas, Atlassian Connect is a great fit. Instead of trying to extend the internals of JIRA and rendering elements and pages with code that runs on the JIRA server, Atlassian Connect provides various clearly defined integration points to embed external pages and page elements in JIRA’s user interface (in addition to providing powerful APIs and web hooks to access and manipulate data stored in JIRA).

So building a new Atlassian Connect add-on mainly consists of telling JIRA which page elements, full pages, and menu items you want to embed. And then building (or extending) an external application to render and provide these elements. When I say external application, this can either be a standalone application such as TestRail, which comes with its own user interface and provides various additional page elements to embed in JIRA. Or it can be an application that provides its full user interface through JIRA, ultimately hiding the fact from users that it’s a separate application at all. In both cases, elements integrated in JIRA will look and feel like native page elements rendered by JIRA, so users wouldn’t notice a difference to classic server plugins.

In TestRail’s case, and this is true for many applications that provide a multi-tenant environment, customers can choose their own domain name for their application instance (for example, Additionally, we also want to enable JIRA Cloud users to integrate with TestRail Server instances (and vice versa). So the Atlassian Connect add-on cannot just hard code the address of page elements to embed, but let the user configure the address of their TestRail instance to link to (see below).

So in TestRail’s case, the Atlassian Connect add-on actually provides just a thin communication layer between JIRA and TestRail. It provides and stores all the configuration options on a per JIRA instance level and embeds the pages of the configured TestRail address. To implement this in a universal way, the Atlassian Connect add-on needs to render its own iframes and implement certain JavaScript functionality (e.g., to automatically resize the frame based on the content height as well as displaying dialogs). The result is a fast and universal integration that allows JIRA Cloud users to embed any TestRail cloud or server instance.

JIRA Server integration with a Plugins 2 add-on

When we looked at the various options to develop the same integration for JIRA Server (on-premise) instances, we first reviewed the typical way add-on developers would approach this: building a Plugins 2 add-on. It’s important to note that we only ever really considered a Plugins 2 add-on implementation for JIRA Server. Trying to use Atlassian Connect with JIRA Server has never been a good idea as it’s not officially supported. And Atlassian also announced that they don’t have plans to extend Atlassian Connect for local servers for now (which makes sense, as it’s just not a good fit).

Developing a Plugins 2 add-on would involve rendering any page elements and implementing any custom logic via Java as part of the JIRA add-on. JIRA provides rich APIs and many add-on options to extend the application. In general, if Atlassian can use a certain feature or extend JIRA in a certain way, add-on developers can benefit from the same flexible customization options. For many add-ons, including our TestRail integration, displaying data also involves requesting the data from external sources first, usually via REST APIs.

In TestRail’s case this would mean test results, details about test cases that are linked to JIRA issues, and the underlying metrics to render rich testing reports. Building a separate Plugins 2 add-on in addition to the Atlassian Connect add-on would create multiple issues. The biggest issue with this approach is the need to implement the full JIRA add-on features and interface twice. First as embeddable pages and elements that TestRail provides for the Atlassian Connect integration in JIRA Cloud. And then we would have to implement all the same functionality as part of Plugins 2 add-on for JIRA Server.

We wanted to avoid implementing the entire integration twice. So instead of requesting and rendering all the data in the server add-on, why not embed the same page elements in JIRA Server that we embed with Atlassian Connect? What if we could reuse the same integration points, with small adjustments, instead of rebuilding everything? And that’s exactly what we did. We identified all relevant integration points and page elements and built a Plugins 2 add-on that uses iframes similar to Atlassian Connect to embed all relevant TestRail page elements.

Atlassian Connect provides various libraries and conventions to make it easier to embed external pages and page elements so they look and behave similar to page elements rendered by JIRA. For example, iframes aren’t usually resized automatically based on the embedded content size. Likewise, accessing methods to show dialogs or messages isn’t directly possible from referenced pages due to browsers’ same-origin policy. In order to emulate Atlassian Connect’s behavior, we had to build a similar thin JavaScript layer to automatically resize iframes, show dialogs, or access other JavaScript methods of the host. Implementing (and securing) this via JavaScript messaging is very straightforward, and just a small price to pay compared to implementing the entire integration twice.

Deploying and configuring the integration

Deploying both the cloud and server add-ons, as well as configuring the integration, is very different for both platforms. For JIRA Software Cloud our Atlassian Connect add-on, (which is basically a standalone application that sits between TestRail and JIRA) runs on our highly available AWS-based infrastructure to render the add-on configuration page for each JIRA instance. Whenever a JIRA Cloud instance embeds a page element from our add-on, the add-on looks up the add-on configuration for the particular JIRA instance and returns a simple page with another iframe to include the actual TestRail pages (and passes relevant configuration and security parameters). By using this approach, even TestRail Server instances behind a firewall can be easily embedded as only the user’s browser needs to be able to reach both JIRA and TestRail.

For JIRA Server instances and our Plugins 2 add-on we developed and render pretty much the exact same configuration page as part of the add-on code. The configuration page also allows JIRA administrators to configure their TestRail web address so the add-on knows the page elements it needs to embed. In case of the Plugins 2 add-on, the add-on can directly render the iframes for the relevant page elements. For example, to embed test results for a JIRA issue on the issue page, the add-on registers the relevant web panel and injects an iframe with the external page reference. TestRail returns the same page elements regardless of which JIRA edition is embedding the pages, making it very easy to extend and improve embedded page elements in future updates.

One big advantage of this approach is that we can update and improve the TestRail integration for JIRA without the need to deploy new add-on versions at all. Unless we need to add new integration points for the add-ons or change the configuration pages, we only have to update the embedded pages and page elements on TestRail’s side. If we have to actually build and deploy new add-on versions, we can simply deploy a new version of our Atlassian Connect add-on on our servers and upload a new build of the Plugins 2 add-on to the Atlassian Marketplace.

Final thoughts and recommendations

After reviewing the general approach on building universal add-ons for both JIRA server and cloud, let’s review some quick tips and recommendations that make it easier to design and develop such add-ons. One important thing we figured out quickly is to start with a simple foundation for both add-ons and gradually add new features to both add-on editions in parallel. It’s much easier to start with a minimally viable add-on for both editions and extending the functionality as you go. By building both add-ons in parallel you can identify potential limitations of the platform earlier and can make design changes faster.

Another important lesson we learned is that if you can avoid requiring a direct connection between JIRA and your particular integration, application, or external data source, it makes the integration across editions and servers much easier and more robust. For TestRail’s JIRA add-ons, no direct connection between TestRail and JIRA is required. This means that no firewall, NAT, or security rules can affect the functionality of our add-on. If you need to access JIRA’s REST API to store important configuration settings or update conditions, accessing the API via JavaScript in the context of the JIRA user can make things much easier.

Last but not least, it’s easy to fall into the common trap of using your application’s existing stylesheets and code to render elements and data inside JIRA. Don’t do this — it will result in a sub-par user experience inside JIRA, as different sections and elements will look and behave differently. Instead, use the AUI library for both page elements embedded from your application as well as the page elements rendered directly by the add-on (i.e., the configuration pages) so your integration has the same look and feel as the rest of JIRA.

Building and designing universal add-ons for JIRA can make it much easier and less time-consuming to target both JIRA Software Cloud and Server customers at the same time with a common code base. If you are interested in learning more about TestRail, our JIRA integration, and how it’s used by teams to boost their testing efforts, make sure to give TestRail a try.