0% Complete

Introduction

Learning Objectives

By the end of this you will:

  • Understand what Workspace and Plan reports are, as well as how to configure and distribute them
  • Understand how to use Webhooks to export data automatically from mabl
  • Understand how to retrieve results using the Execution Result API (this can only be used in conjunction with the Deployment Events API)

The Why

  • Ability to disseminate information to a single user or group of users in the workspace
  • Insights into plan and overall Workspace health and status

Definitions

Workspace reports

  • Workspace reports are a summary of activity in the Workspace that are sent to your email address.  
  • Workspace reports are configured for, and by, an individual user, under their User settings.
  • The summary includes an overview of the Workspace, recent test runs, passes and failures, as well as recent insights.
  • Links to those recent runs, insights and the individual tests themselves are available in the email.

Plan report

  • Plan reports are a plan-level report that can be sent to everyone or a subset of users in your Workspace.  
  • They can be triggered on a schedule and/or based on an event, like a Plan passed or failed.  
  • The triggers are mutually exclusive, meaning that if you have a Scheduled Report and a Triggered Report you will receive both.

Webhook

Webhook JSON payloads tell you the status, cause of that status and whether the Plan/Test(s) passed or failed, along with the following identifiers of the Plan/Test(s):

  • Plan
    • ID, Name, Label(s), href, tags, Execution Status, Execution ID, Execution href
  • Test(s)
    • ID, Name, Label(s), href, tag, Start Time, End Time, Success (true, false)

Execution Event API

The Execution Event API will return a JSON object containing summary information about all Plans and tests that were triggered by the Deployment Event API.

Getting Started

How to use workspace reports

A Workspace Report is configured under User Settings, which can be found by clicking the Workspace name dropdown in the upper right corner of the application.

 

From there you can schedule the Workspace Report that will be delivered only to you - whether you'd like that report delivered Daily, Weekly or on a Custom schedule (specific day(s) and time of day).

Weekly reports will be sent starting on the date that you save the Reporting Settings.

For example: if you pick Weekly and save on a Wednesday, you will receive reports every Wednesday.

How to use plan reports

Changing settings for Plan reports is as easy as Workspace reports.

 

Once you're in the Plan you'd like to update, simply click the envelope icon to open the Reporting Settings modal.

By default, mabl will send reports to all users in the Workspace -- you can choose individual emails and how often you'd like the report sent - daily, weekly. To disable, set the frequency to Never.

Weekly reports follow the same scheduling logic as Workspace Reports.

NOTE: If you have selected Workspace and Plan reports, you will receive 2 separate reports

How to use webhooks

  1. In the lower left corner, click Settings, then navigate to the Webhooks tab
  2. Click “Add Webhook” to configure
  3. Add the name of the webhook
  4. Add the target URL that mabl will call and deliver a JSON payload describing a pending or completed execution
  5. Set pre- and/or post-execution triggers
    1. Set your outcomes to On Failure and/or On Success to fire when those are true
    2. For pre-execution webhooks, you can also check a box to require a success response, which will trigger the endpoint to return a 2xx/3xxresponse for the test to continue. If a response other than 2xx/3xx is received, the test will be failed.
  6. Set strict SSL Validation if required. mabl will always ensure all connections use valid SSL certs
  7. All webhooks for a given Workspace are fired in parallel, which a maximum time of 60 seconds to complete, otherwise it will be considered failed.

Recap

Reflect

What did we learn?

  • How to add Webhooks to pre- and post-execution stages

Connect

How is it related to what you know and are doing?

  • Plan and workspace reports are a way you can keep a pulse on what is happening on the workspace level or plan level.

Apply

How to use this going forward?

  • This will enable you to get data out of mabl automatically and into your CI workflow

Feedback

Was this helpful?

Take this 2 minute survey to let us know how we can make mablU better!