0% Complete

 

Introduction

Before taking this lesson

Make sure you are comfortable with the following lesson(s):

  • mabl Application Introduction
  • Test Results Best Practices

Learning Objectives โœ”๏ธ

By the end of this you will:

  • Understand the value of Visual Testing ๐Ÿ’ฐ
  • Know how Visual Testing works ๐Ÿ› 
  • Understand how Visual Tests are created ๐Ÿง 
  • Know where to review Visual changes ๐Ÿ‘€

Key Benefits of Visual Testing ๐Ÿ—

  • Functional testing is important, but Visual Testing is a very important complementary feature that can ensure there are no issues with the look of the application.
  • Visual Testing helps identify visual changes in static sections of the app.
  • mabl not only shows you the visual changes, but can also provide insights about visual changes, as well as highlighted side-by-side comparison of differences.
  • Beyond complementing functional testing, many of our users leverage Visual Testing to monitor their production environment without impacting application performance.

Overview ๐Ÿ—ป

There are two types of visual change detection for trainer-created tests in mabl:

  1. Machine Learning-based change detection - mabl learns to differentiate expected visual changes from the unexpected using an established baseline, where visual change Insights are triggered. (Note: applies only to tests where Visual Change Learning is enabled in the Plan - you'll learn how to do this in this lesson).
  2. Run-by-run change detection - mabl notices any and all changes between the current test run and the previous (this the default setting when Machine Learning is NOT enabled in the Plan).

Getting Started

How does Machine Learning-based visual testing work? ๐Ÿง

  • Using machine learning, mabl creates a visual model of your appโ€™s UI to help you identify visual changes in static sections of the app.
  • We create a fixed baseline after ~10 test executions for a Plan. That baseline can easily be updated to adjust for intended changes (check out how to rebaseline here).
  • Since mabl knows whatโ€™s dynamically changing (ads, animations, videos, alternating images for products/offerings, etc...) we alert you with notifications and include insights and highlighted comparisons of those differences.

Quick disclaimers before we get started

  1. Visual changes do not impact the pass or fail of a test (however, you will be notified of any visual changes that can be reviewed under Insights)
  2. To create visual tests that check for correct loading (as well as visual changes, broken links, JavaScript errors and network activity issues across browsers), you'll want to create Visual Smoke Tests, which are covered in a separate lesson in this series, but in the meantime, click here to learn more about creating Visual Smoke Tests

How to get started with Visual Testing โœ๏ธ

It's almost too easy to get started: just create any Test under a Plan and enable Visual Change Learning. That's all you have to do, then mabl takes care of the rest.

 

Once that Test executes, machine learning automatically creates a visual model of your appโ€™s UI (some of these screenshots will appear in the test output).

Any additions, deletions or edits of steps (including Flows) in the test will automatically be detected and mabl will adjust the visual model of the UI

It's important to note: detected visual changes will not fail a test - they are treated as warnings and displayed in test output.

mabl decides what changes are important and generates insights for those changes.

Insights are in a few places: ๐Ÿƒโ€โ™€๏ธ

On the left-hand side, you'll find the Insights bell icon.
insightsFrom there you can filter by "Insight Type" to only see Visual changes and click through to the Test Output, or you can click the "Visual changes" tile at the top of the page to be taken to the Visual Explorer that will automatically apply the Visual changes filter. These will be explained in the next few pages.

Please note: a link to this Insights page is also included in an emailed Report that you can configure in the app (Reporting is covered in another lesson)

Let's take a look at the Test Output ๐Ÿ‘€

To get there from the Insights page, click the More actions ellipses (three dots) on the right side, then select View Insights Details, then Results.

When you get to the test results page, mabl will automatically filter the steps to show ones that have the corresponding visual changes.

 

Here you can see the Current or Baseline version, or a side-by-side comparison. The differences will be highlighted.

As shown in the accompanying image, you have the option to 

(A) see where mabl identified dynamic content areas (if available) by clicking the Highlighting dropdown and selecting Dynamic content

(B) you can choose to rebaseline the current step, or all steps, or download a screenshot for further review/distribution by clicking the Actions dropdown.

test_output

Visual Explorer ๐Ÿš€

If you click the Visual changes tile at the top of the Insights page, you'll be taken to the Explorer page (which can also be accessed by clicking the Explorer icon on the left side of the app) and the Visual changes filter will be automatically applied.

 

In the above video, you can see you're able to filter further by test, URL and apply a date range to narrow the changes down.

Clicking on an image will give you similar, but fewer options, than when you're viewing in the Test Output.

You can see the current or baseline image or a side-by-side comparison. You'll also be able to highlight visual changes or dynamic content by clicking the Highlighting dropdown.

For further review, you can click "Results" in the upper right corner and be taken to the Test Output page.

Recap

Reflect

What did we learn?

  • How Visual Testing works
  • How to create a Visual Test
  • How Visual changes don't fail a test, but just generate insights
  • Where to find results and insights

Connect

How is it related to what you know and are doing?

  • Visual testing is created automatically along with any Test under a Plan, without any additional effort required from the user
  • Visual testing complements work youโ€™re already doing and is an important part of an end-to-end testing solution

Apply

How to use this going forward?

  • Be sure to check on Visual changes to ensure the look of your application is what you expect
  • Leverage Visual testing to even monitor your production environment, as it won't impact performance

Feedback

Was this helpful?

Take this 2 minute survey to let us know how we can make mablU better!