This lesson is made to help you more easily understand test results and isolate issues faster.
Ultimately, this lesson will guide you towards a repeatable process of test review, so that your team can feel confident finding issues and communicating them.
When using mabl to test the frontend of an application:
For things that work: rock on 🤘
For things that are broken: find the root cause, communicate it, and get the problem fixed ASAP 🛠
mabl's debugging tools help you:
With clear, repeatable troubleshooting practices, your team can find and fix bugs faster, which leads to improved overall app quality, and most importantly, happy users 🙂
Network Issue - network issue detected
Auto-heal - element found based on secondary search criteria
There are two different types of pixel-based visual change detection:
Compares each current screenshot to the last executed test.
(Automatic for each test run)
Uses the last 10-15 executions of a test to establish a baseline, and highlight whether areas deemed as 'static' differ from the baseline - the baseline is updated on an ongoing basis.
(Must enable in Plan under 'Advanced Settings', and run test ~10 times)
Learn more about mabl's visual testing here!
The following tips are meant as guidelines for the majority of test failures that you will experience, but they are not exhaustive.
Some of these practices will depend on different types of failures.
As you test more with mabl you will develop a cadence that works for you and your team 😎
ⓘ Test failed and not sure why? Reach out to our support team! Click the button in the lower-left corner of the mabl app (that looks just like the one below) and start chatting with us ANY time! 👍
1. Start with the failure ⛔️
2. Work your way backward ⏪
3. View the last passed test for context ⬅️ ✅
4. Reproduce the failure on your own 🔂
5. Reach out to mabl support for help 🙂
Let’s dig into each one!
Clicking on the steps before the failure, work your way backward to understand what happened leading up to the failure using the Step Analysis Framework for each step.
ⓘ Be sure to look for the yellow circle in the screenshot to see what element mabl interacted with during that step execution.
Using the dropdown under 'Time of run' at the top of the results page to see previous runs of the same test.
Look at screenshots to compare the behavior.
Use Virtual Tabs to go back and forth between tests easily when viewing multiple test output pages.
First, edit the test and try reproducing the failure.
Next, go into your application WITHOUT using the trainer and try reproducing the failure.
Use the chat icon in the lower left-hand corner of the mabl App.
After finding a failure, the approach will differ depending on the nature of the failure.
The following are good next steps to be aware of:
Let’s expand on each one 🤠
- From the dropdown, select the reason that most closely describes why the test failed
If the reason you're looking for isn't there, select 'Other', or submit a feature request here!
ⓘ Did you know?
You can filter on failure reasons from the Results page (left navigation bar). You can also view the percentage of each failure reason in the past 30 days from the Dashboard!
Help give context to your teammates by leaving comments on the test output page.
Shows details of the commenter, and timestamp of comment.
Select the type of information you would like to export by downloading it and sharing it with the necessary people on your team
To export results as a PDF - click the three dots net to Edit
ⓘ Did you know?
If you navigate to the failed step, copy the URL, and then share it with teammates it will bring them directly to the step of the failure!
Since everyone's Jira instance is a bit different, click the links to our help docs below, which have more specific instructions for how to approach that!
Jira Cloud Integration [Doc]
Jira Server Integration [Doc]
What mabl's debugging tools are, and how to best use them for reviewing test results and isolating issues.
- Form a process around reviewing test results that is known and understood by the team
- Find those pesky bugs faster and more effectively than ever before! 👾]