Building Intelligent Pipelines with mabl and GitHub Actions 2.0
At mabl, we have long envisioned a world where it was easy to configure and operate intelligent CI/CD pipelines. In that world, new code changes are automatically tested and assessed for their quality and risk. If the risk is too high or the quality is too low, the changes are automatically held back until the team takes appropriate action--by fixing bugs, adding test coverage, reviewing changes more thoroughly, or otherwise. If the risk is low and the quality is high, the changes are moved along in the pipeline quickly and efficiently.
In order to achieve that vision, we recognized that testing tools would need to become more intelligent. For the past two years, mabl has focused on building the most intelligent end-to-end testing service on the planet. Today, people can use mabl to create tests in minutes, run them on all major browsers, and receive insights about their application quality, including functional correctness, visual presentation, performance, and more.
To achieve the vision of intelligent CI/CD, we also recognize that pipelines would need to become more integrated. Today, GitHub launched Actions 2.0, which we believe will serve as a leading platform for smarter, more integrated CI/CD workflows. We jumped on the opportunity to integrate with Actions 2.0 and GitHub more broadly. In this post, we'll show you how those integrations work.
Overview of the
GitHub / mabl workflow
At the highest level, the workflow starts with you making code changes within a given GitHub repo, building them, and deploying them to an environment using Actions 2.0. At that point, a second Action triggers end-to-end tests within mabl. If mabl discovers problems during the test runs, it creates corresponding GitHub issues within the relevant repo.
Our challenge was to automate the most common activities related to testing in a CI/CD pipeline; kicking off automated tests and tracking issues. We wanted to deliver those capabilities in a manner that was easy to configure for new users but was also flexible enough to support the needs of power users.
Testing new deployments
name: mabl Test
- uses: actions/checkout@master
- name: Run mabl tests against deployment
browser-types: [‘firefox’, ‘chrome’]
When mabl receives a deployment event, it automatically triggers the relevant tests based on the parameters that are specified in the Action. The Action then polls mabl to get the status of each of the triggered tests until they are all complete. At that point, the Action completes.
By default, if any of the triggered tests fail, the Action returns a failure and stops the pipeline. Alternatively, you can configure the action to continue on failure appending the following:
Part of the power of this model is its flexibility. The mabl action uses the outputs feature of Actions 2.0 to output a number of statistics about your test results. You can use these statistics to control which subsequent actions are executed. For example, rather than failing the workflow if any test fails, you can customize the action to only fail if more than five tests fail by appending the following:
- name: Do something on failure
run: exit 1
if: (job.steps.mabl-test-deployment.outputs.journeys_failed > 5)
You can even configure your pipeline to automatically promote the build to the next environment if the tests all pass, so that you don’t need to manually tag or trigger a push to production. In this example, we call another action to deploy the change to production:
- name: Deploy to prod
When the Action receives information on test results, it also provides you with deep links into mabl to view the detailed logs, screen shots, DOM information, and more.
Actions log with links back to mabl output
We are so excited about this integration because you can be up and running with the core functionality really quickly, but it also gives you the flexibility to tailor the workflow to meet the needs of your team over time.
Creating issues automatically
In order to streamline issue management, we created a simple mabl GitHub app that you can configure to read your mabl event stream and create issues automatically based on the results of tests. This saves you the time of collecting the relevant information and manually creating each issue by hand.
GitHub integration settings
The issues include screenshots from the test and links back to the detailed test results in order to help you triage and resolve issues as quickly as possible:
A GitHub issue with the latest screenshot and a link to the full mabl test output
Bringing it all together
In order to make the experience truly seamless, we also wanted to make it easy for you to navigate across the GitHub and mabl interfaces. Fortunately, GitHub makes it easy to maintain the appropriate context across the integrations; the mabl action passes PR, committer, and branch information along to mabl automatically. As of today, you can filter your mabl test runs based on Commit information from GitHub.
Filter mabl test results based on commit information from GitHub
We're working hard to add deeper links between mabl test results, pull requests, branches, and more.
This is the first in a series of integrations and enhancements that we have planned to help you realize the potential of intelligent CI/CD pipelines, so please follow us on Twitter to stay up to date. In the meantime, you can sign-up for the GitHub actions Beta and get your free trial of mabl to try it out for yourself!
Watch a quick overview of the new mabl GitHub Integrations: