As an Atlassian partner, META-INF works with Atlassian on multiple levels: our business is based on our Atlassian expertise, and we use Atlassian products to do our work. Initially, we stored our source code on Stash and then on Bitbucket Server which was complemented by Bamboo. Since migrating to Bitbucket Cloud, we have continued to build, deploy, and orchestrate various DevOps operations, such as setting up Atlassian on-premise test environment management and everything else necessary for Atlassian app development as a Platinum Partner.
Before: Identifying the challenge
In our Atlassian stack, we would sometimes run into features that didn't exist. When this happened, we would look for an app, or if one didn’t exist, we’d create it! This was the start of many of our Marketplace apps including Email This Issue, Content Exporter, Glass Documentation.
With Bitbucket Cloud, we faced the following challenges:
- Pipelines
- were not flexible or interactive enough with the user during the execution
- did not support enough options for input fields (mandatory, regex, multi-select, etc.)
- sometimes required a separate guide for execution
- Scheduling
- did not allow for the specification of variables or overriding the default, necessitating pipeline code duplication with different default values
- Did not allow selection of a specific commit, only a branch
- Once a schedule was set, it could not be edited
We had recently released our first Forge app (ACNC) when we heard Forge would be available on Bitbucket in an Early Access Program (EAP). With our Forge experience and the desire to solve the challenges above, we began working on our first Bitbucket Forge app.
We aimed to create a flexible pipeline control app that would meet our internal needs (regular testing, managing Jira test environments) and, if it worked well for us, could also be helpful to others.
Launching Pipeline Forms
From that spark of an idea came Pipeline Forms (PFO), an app that integrates with Bitbucket to enhance pipeline management. Pipeline Forms creates a pipeline-forms.yml file—nestled next to your bitbucket-pipelines.yml—that allows us to define sophisticated forms and fields that echo the behavior of Jira screens.
Pipeline Forms provides customizable input fields with field types including basic text and checkboxes, as well as advanced regex-validated text and remote fields. Fields can include attributes such as read-only, hidden, required, default value, and secure.
The true power of Pipeline Forms lies in simplifying long and complex pipeline configurations with the pipeline-forms.yml file. By filling out user-friendly forms, end users can dynamically supply parameters to pipelines, enabling varied execution scenarios directly from an intuitive interface. This not only reduces the complexity inherent in bitbucket-pipelines.yml but also broadens the spectrum of possibilities for pipeline execution.
Pipeline Forms also includes scheduling capabilities, allowing for hourly, daily, or specific day executions. Scheduled executions, like manual ones, use parameters entered in the forms. These scheduled tasks can be viewed (along with their parameters), turned off, edited, and deleted.
Lastly, extensive logging captures every change and execution, ensuring full transparency and control over pipeline activities.
After: Benefits from Pipeline Forms
The impact of Pipeline Forms on our workflow cannot be overstated. Initially, our approach to creating Atlassian test environments was efficient yet somewhat limited, relying on a customizable script launched through pipelines. It served its purpose, enabling those in the know to spin up environments as needed, but it lacked the ease and flexibility we strived for.
Once we built Pipeline Forms, we created several forms that not only simplified configurations but also significantly broadened the range of possibilities:
- Readonly Fields: Ensuring that certain critical values remain unchanged (but visible), thereby maintaining consistency and reliability.
- Regex Validation: This brought a new level of precision to naming environments, ensuring names meet predefined patterns for better organization and identification.
- Dynamic Value Population: By automatically fetching Jira versions from DockerHub, we ensured that users could access the latest releases.
- Expiration Dates: The ability to assign expiration dates to environments introduced a level of temporal control, crucial for resource management and security.
Reducing configuration complexity for the bitbucket-pipelines.yml file while creating a well-organized pipeline-forms.yml file was a game changer for our DevOps team. This change meant users could tailor their environments with less effort and greater accuracy. This ease of use and the system’s enhanced fault tolerance resulted in a more resilient and user-friendly platform.
Technology Challenges
One of the distinctive features of the Forge platform is that customer data does not leave the customer’s instance, a principle that applies to Pipeline Forms as well. The app integrates into the repository, and following user authorization, it allows users to run Form-based pipelines on their behalf while maintaining security.
When we joined the Early Access Program for Bitbucket Forge apps, some fundamental important features were still missing. However, by the General Availability phase, we were excited to see that creating a product as mature as Pipeline Forms became possible! We are keen to route billing for the app through the Atlassian Marketplace, and this is one of our remaining feature requests for Atlassian.
Conclusion
Bitbucket Pipeline is a solid tool, but with the help of Pipeline Forms, we’ve transformed it into a truly powerful and elegant DevOps solution. Now, we execute approximately 2,000 customized executions monthly in our production environment. These executions include a variety of operations, such as:
- Occasional and scheduled system and other tests of our products
- Setting up, backing up, and deleting Atlassian on-premise test environments
- A daily standup bot to support our morning standups
- A script for detecting new versions of Jira applications
- Processing logs of our Forge apps
- Running additional custom scripts from a user-friendly interface
In conclusion, Pipeline Forms has proven to be a game-changer for us, transforming how we think about and interact with Bitbucket Pipelines. It’s not just a tool; it’s a catalyst for innovation, enabling us to create more robust, scalable, and user-friendly DevOps solutions. As we continue to explore its potential, we’re excited about the future possibilities and the next wave of challenges we can overcome with Pipeline Forms at our side.
Pipeline Forms EAP can be installed from here! By installing, you automatically agree to the Terms of Use.Extensive documentation can be found here, with help in syntax, examples, and more.