So your organization is doubling down on agile processes, and you’ve been told that as part of the effort, you need to start automating your tests. This makes sense, as testing automation can increase test coverage, and you can do a lot with them. For example, you can integrate them into the continuous delivery pipeline and use them as an automated quality stage gate. You can have a robust test suite to determine the quality of each release candidate, and map regressions the more the test suite runs.
The problem is that you don’t exactly have coding experience, and you’re not sure where to start. A logical step forward that many have taken is to adopt a record and playback automated testing tool.
The promise is great - all you need to do is step through your web app, and all your actions will be recorded and turned into a test script for you, right? Well… it’s not that easy.
In this post we’ll cover:
- The folly of most record and playback tools
- The most effective scenarios to use them in in order to get an ROI
- How next-generation record and playback tools are fixing the problem that traditional record and playback tools have
Record and Playback: Promised Land or Oasis?
So you found the tool you want to use, and you go ham on creating tests for every single test case that you have documented, and oh boy! record and playback automation tools definitely make test creation easy. The hard part now, is test maintenance. Consider how record and playback tools are capturing the activity in your test. Most rely on element descriptors like XPATHs or CSS, like element IDs, to identify objects you’re taking actions on. This is fine if your web application is at a mature stage where UI changes are seldom made. But if your organization is trying to embrace the fail-fast mindset of agile and DevOps, then it’s likely that your application is changing frequently, sometimes multiple times a day. The test scripts that you’re now left with break often, leaving you with unreliable test suites that are hard to diagnose because the test scripts are so convoluted from automatically generated code.
The promise of test automation slowly slips from your grasp like the soul from the husk of your body as you realize that you’re spending more time fixing and re-recording tests than testing. A little morbid? Maybe so, but this is the life of many who tread this road naively. The best ways to use a traditional record and playback tool is to get acquainted with testing automation (record a user flow and review the generated test script), or for testing simple, short test cases that don’t change. Limiting traditional record and playback tools to these two scenarios will ensure that the tests stay manageable and beneficial.
Next-Generation AI-backed Record and Playback Tools
But it’s 2018 guys. In the age of AI and extensive automation, we can harness machine learning to build practical solutions to overcome many problems—including the steep learning curve of automated test scripting. mabl and other “third wave” automation tools employ machine learning methodologies to testing automation, so that everyone—developers, testers, and even operations and support teams—can create reliable, and resilient tests in a few minutes.
How so?
1. Recording more than just UI locators
record and playback tools can’t rely on element IDs or XPATHs anymore, especially with modern AJAX apps that have dynamically loaded content which contain non-unique locators and IDs by nature. With the growth of SaaS and cloud storage, next-generation record and playback tools can collect dozens of attributes for every element in a test to make locating objects a breeze, even if one or several attributes have changed.
2. Updating tests automatically as your apps evolve
Next-generation record and playback tools should update tests automatically, reducing test maintenance almost completely. Once we have a collection of attributes for every element in a test, we can update our knowledge about those elements at every code release iteration. Just like how a codebase is a living, breathing organism, tests should also evolve with the application in development.
3. Using machine intelligence to track regressions
With cloud and machine learning frameworks making machine intelligence and analytics so accessible, we can model characteristics of our applications through regression testing. These can range from performance anomaly detection, to traditionally computationally intensive processes, like tracking visual changes. Using robust analytics engines available in the cloud will let us rate the health of a release, not just based on test failures, but on the trajectories created from our regression testing.
4. Plugging functional UI tests into your software delivery pipeline
Now that we have a good formula for a better record and playback tool, we can plug reliable functional and UI tests into your software delivery pipeline. record and playback tools aren’t simply for automating simple test cases anymore, but they can become a legitimate helper in increasing test coverage in the agile and DevOps age, just as you’ve always dreamed of.
mabl is an automated testing service that runs in the cloud, and uses machine learning to make record and playback test automation just as described above. You can try mabl for free at www.mabl.com.
