Many teams report that their developers produce software that is 95% complete, and yet it doesn’t integrate well and is not shippable to customers. This is the reason why pair programming is given serious consideration in many software development organizations. We wrote about pair testing which is most valuable in teams for which there is at least some tolerance for deeper thinking, exploration, and some creativity. The ideal is to find two team members who are critical thinkers—one who is creative and the other with the capacity for disruption. As in many other aspects of human experience, two minds can be much more productive than one.

While pair testing can be very beneficial—especially for complex challenges—it quite inefficient for most development teams that continue to crank out many new features and changes. Often testers must perform manual exploratory testing on a significant proportion of changes, and then turn to manually update and reconfigure the regression suite. On many development teams, much of the conventional test automation is tedious, time-consuming, and faulty.

The burdens of conventional test automation

To improve productivity, many software QA teams rely heavily on conventional automation frameworks such as Selenium and Appium. More than 80 percent of the respondents to a recent mabl survey currently use Selenium. While frameworks such as Selenium have boosted QA velocity in comparison with manual testing, there are a number of significant disadvantages such as:

• Each framework require specialized scripting skills—which are in short supply. Most teams don’t have much capacity for additional QA automation.
• Test results don’t provide much context, leaving the team with little information to properly address problems and bugs.
• At scale, a conventional automation framework needs its own infrastructure, which requires resources for provisioning and operation.
• There is tight coupling with changing attributes of the product, such as XPaths, and CSS. This results in constant maintenance and false positives.

As the pace of innovation continues to increase, these disadvantages become acutely painful. It becomes so intense that some teams limit the use of conventional tools. Since they don’t have the capacity for retraining or increasing QA staff, this often results in a decrease in product quality. 

Pair testing should have a positive ROI

As we wrote previously, pair testing involves one person that does the testing—while another person observes, inquires, clarifies, records notes, and spots defects that would otherwise go unnoticed. Pair testing can be especially effective when one programmer sits together with a tester. The dual oversight of this pairing can yield many benefits, thought it is arguably an inefficient use of human capital. No wonder that it is used sparingly.

Pair testing accomplishes the following:

• Finds more bugs
• Saves time
• Eliminates the communication gap between testers and developers
• Is an excellent opportunity for very efficient exploratory testing
• Provides visceral, in-person learning opportunities

Ideally, pair testing should give each participant an opportunity to take the wheel. When the tester is driving, the programmer can gain deeper, more valuable insights on how a tester uses and perceives the software. While the programmer is driving, the tester can gain a deeper understanding of how the software has been built.

Pair testing is especially effective during development. Many problems will be found at such early explorations, and this early identification will likely result in a much easier solution. Even better, if possible, would be to invite a business analyst to pair testing sessions, and this upstream triad is sure to reveal any lurking logic, design, usability, or functionality issues. Further downstream in the delivery pipeline, post-development pair testing collaboration can be valuable for improving development and testing practices.

What if you could easily learn and manage a learning machine that functions as a testing partner? How much could you benefit from a configurable system that provides not only regressive testing—but also additional testing oversight across the entire development pipeline?

Can a machine function as a testing partner?

While a machine isn’t going to look over your shoulder, you can enlist and configure automated computing systems to substitute as a pair testing partner—at least to some extent. A robust, AI test automation system can test various permutations and combinations that might otherwise go untested. With such as tool, you can leverage the power of machine learning to find bugs that might not otherwise be found and work faster to perform regression tests that will automatically adapt and update with each new release.

The best of the new wave of automation tools can save significant amounts of testing effort which can then be spent doing more exploratory testing and finding more ways to automate to accommodate new features and changes.

How mabl is taking test automation to the next level

Quickly evolving requirements and specifications, ever more rapid feature development, short release cycles, and nebulous accountability affect overall quality. All of these combine to burden QA with serious and intensive challenges. New tools like mabl are emerging and are making far-reaching headway in mitigating these challenges. Let’s consider a few.

Create and manage tests more easily

Unlike current automation solutions, next-gen QA tools like mabl don’t require scripting expertise. Scripting will largely be replaced by visualized interfaces that enable rapid test creation, easy and automatic test adjustments, and simple dashboard test management. Test coverage can expand significantly and tests will evolve fluidly with changes and new features.

AI will adapt tests automatically

Because they employ adaptive, robust methods of simulating user interaction, next-gen QA tools don’t exhibit the brittleness of conventional test automation toolsets. It’s no longer necessary to keep the bar low with tests that remain closely bound to volatile front-end elements. Solutions like mabl leverage machine intelligence to create and maintain elaborate yet elastic test models.

Run QA faster in the cloud

Many of the most innovative QA tools today exploit rapid-provisioning cloud computing resources for much more efficient parallel test execution. It’s easy to extend this approach and tap into highly powerful processing and analytics engines to scrutinize every last test result detail and deliver more meaningful QA insights.

Integrate QA tightly into the delivery pipeline

Cutting edge AI testing solutions like mabl integrate tightly with automated CI/CD delivery pipelines that are managed by CircleCI, Jenkins, CodeShip, Spinnaker, and any others. One wonderful outcome of this seamless integration is that tests can be set to trigger automatically on frequent, granular product changes. Notifications are sent immediately to key staff when potential concerns arise. Far fewer bugs will make it downstream, and it’s much easier to execute tests—comparing results among various environments and builds. Some will contend that the most glorious result is that release, deployment, promotion, and rollback decisions can truly be much less stressful.

QA paradigms will transform

Beyond merely validating particular assertions, the new generation of QA frameworks will use machine intelligence to automatically detect, pre-analyze, and enumerate probable regressions. Quality assurance thought patterns will morph away from pass/fail paradigm, as technical leaders increasingly seek to assess the extent to which a change will have a positive or negative impact on overall user experience. The major focus will shift from code coverage to product coverage—toward a holistic approach that is continually assessing the risk of releasing each successive build.

mabl helps you effectively test rapidly evolving applications

Let’s look at some specific ways that mabl can be a effective virtual pair testing partner. We wrote about how our users had been pushing the mabl Trainer to do all the things they could do in Selenium. Of course, mabl has much more capability that goes beyond Selenium, but we have responded fully to user feedback and made a big push in this direction.

Today, the mabl Trainer enables users to interact in many different ways with their applications:

• Assertions - we have lots of different assertion types for elements of a page, including present, equals, <, >, starts with, and more. These are often used to confirm the existence of one or more buttons, menu items, search box, etc., on a page
• Searching for elements on a page - by CSS or XPath, and choosing the first, last, or any element meeting the criteria specified. Once we locate an item, you can also decide what you want mabl to do - select, make an assertion, click on.  This type of search is often used by customers who are choosing something from a list, which may be dynamic and change on a daily basis. A couple examples of this would be an airline reservation system or calendar booking application.
• Create and use variables - numeric, alpha, or alphanumeric variables calculated by JavaScript functions. Using randomly created variables are very popular in filling out sign up forms, contact us forms, or other input oriented portions of web applications.
• Adding wait times - up to 60 seconds added in between steps of a test. Though mabl already has automated retry and wait times built in so that users don’t have to add waits for elements to load, we still allow users to configure wait times for any particularly long-running background processes.
• Flows - a series of steps which can be saved and re-used across tests. Flows are very useful to append to parts of tests so you don’t have to re-create the same steps in different tests. Examples of this include login flows, check-out flows, creating new user flows and search flows.
• Data-driven testing - being able to use specific data from a file as inputs or variables for a test. This is a very common use case for Selenium users. Users can copy and paste their data from a file into a mabl grid and use that as part of their tests.
• Hovers and iFrames - being able to track user behavior for menu items that have hovers or applications which have iFrame designs
• Custom Javascript - inserting a custom Javascript snippet to take an action as part of a test. Users can add custom JavaScript to accomplish things that are not natively supported in the Trainer.

mabl is a high-value intelligent test partner

Software QA has never been easy. The accelerating pace of software development is creating more complexity that will only serve to intensify existing challenges and raise new obstacles. Thankfully, mabl is part of a small group of distinguished testing platforms that have emerged to deal with the complexity of today—and the future. At mabl, we continue to innovate our Trainer by exploiting new advances in cloud computing, machine intelligence, and tight CI/CD coupling. The result is a dramatic improvement in QA efficacy for our customers. Let us know if you’d like to try us out.