How to Set Up Your Workspace and Support Your Quality Assurance Infrastructure

Whether you’re new to mabl, or a seasoned user, good workspace hygiene is critical to your team’s testing success. In this interactive session, Damien will share best practices for organizing your workspace, creating effective tests, and understanding test results. Sign in to your own mabl workspace and follow along!

Transcription

Damien Bell

Good afternoon, everyone. I hope that you are having a great day so far. Let's talk about setting your workspace up for success. So, some key points that I want you to take away from today are first, understanding how to implement some mabl best practices, supporting a solid foundation for your quality assurance infrastructure. Second, staying organized to scale your company presence as you onboard new products, people, and teams in your mabl workspaces. Third, mabl as a product has a lot of features that people overlook that help accelerate teams as they organize and categorize their work. We're going to be diving into some of those. Lastly, despite there being a huge gap between different organizations in terms of size, product functionality, team member skill level, and testing methodologies. mabl can fit into what you're trying to do, and make it significantly easier.

Before we get too far ahead of ourselves, I'm Damien. I run the support organization over here at mabl. My job is to manage a team of support engineers who help our customers with the development and debugging of tests, environments, networks, honestly, you name it. We're the Swiss Army knives over here at mabl. I try to be a voice for our customers and really help them be heard within mabl. I believe that there's a need for us to learn from what you all are trying to do and to ensure that we're moving forward together.

This presentation is based on how some of our most successful customers have set up mabl to work in their workspaces. So to start off with, let's talk a little bit about the primary components of a workspace within mabl. On the right, you can see a hierarchy of how each part of the workspace interacts with mabl. I'm going to take a minute to walk through how I think of these so that we can all start off on the same page.

Firstly, we have the test all the way down at the bottom. Since many of the following points are going to talk about tests, let's begin by defining that. For me, I think of a test is a series of automated actions called steps, which tell mabl how to interact with and validate a specific workflow or functionality within your application. Some examples of steps include navigating to a URL, finding a button, inserting text in a form, and dozens of other useful page interactions. A step is the smallest building block that can be used inside of a test.

Next, plans. A plan is an ordered group of tests that hold configuration for how mabl carries out your automation. Some factors that are taken into account are the environment, the application, the execution frequency, and which browsers you're targeting.

Next up, and not in the diagram on the right is labels. A label is a word or a phrase that you attribute to tests and plans for the purposes of classification organization and in the case of plans deployments via CI/CD.

Next up is a description, which is a brief summary or anecdote about a test or a plan that tells your team what it tests and what the context is. Lastly, is flow. A flow is a reusable subset of test steps that you've designated and can be inserted into other tests. An analogy of how you might think of this in a really approachable way might be to consider how Google is structured. The workspace overall could represent all of the products and services owned and operated by Google. The environments here could be Dev, QA, prod, and others. Each of these could have its own special hardware and software configuration credentials and data which acts completely independently of the others. The apps, in this case, might be Gmail, Google Search, and Google Calendar and while each of these are owned by Google and might share some interaction, they don't share many features and are likely worked on by separate teams within the organization.

Plans are groups of tests, which seek to fully validate a piece of a component feature or interaction within the application. An individual test could be ensuring that a single page in the application works the way that you intended to. Thinking about Gmail, perhaps you'd want to log in, click the compose button, and then after sending your email, click sent items to ensure that appears there. Lastly, as an example of a flow, which we're not going to talk about much throughout this presentation, are reusable steps that you can use to automate often reused actions and the example above where we talked about logging into Google, the insertion of the username and password could be an example of a flow that you'd like to reuse elsewhere.

Let me show you what some of these look like inside of mabl. When in your workspace, if you click plans on the left hand side of the page, you'll see something like this when the page comes up. There'll be a list of plans that have descriptions, labels, and environment data that describe what they intend to do. Notice the filters up near the top of the page, which allows you to search for tests depending on their name, label, or environment.

Here's an example of the test page. This will list out all of the tests that you have in your workspace, you can see how many plans the test is in by looking at the plan count column. The labels and descriptions are also seen in this view as well. When you click on an individual test in the previous view, you'll be taken to this page which shows you much more detail about a single test. You can see the name of the test up near the top of the page along with the description and the labels from the previous view. On this page, you can also see the quote train steps, which is midway down the page there, which are those building blocks of automation that we mentioned back on the slide with the diagram. These tests allow us to understand what each interaction in the test is trying to accomplish. When considering how you could go about setting up your mabl workspace to work for you, I took into consideration how long it would take to get things working perfectly.

This timeline like many of these slides is going to be subjective, but it does give some good guidelines for you to consider when your organization starts automating with mabl and if you're already a mabl customer, don't fret. A lot of this might be new information to you as well.

The first thing I wanted to talk about today is mabl University. mabl U is one of the best ways that you can quickly come up to speed on how to use mabl. Believe it or not, this is one of the ways that we onboard internal employees who aren't familiar with how to use mabl yet. For learning the basics, as well as a few more advanced use cases, mabl University is going to provide you with a lot more engaging exercises than just working through the help documentation. It is frequently updated. So it always has the latest and greatest stuff that we have to offer here at mabl.

For our second point, a recurring theme that we're going to mention a lot throughout this exercise is that there's a strong need for collaboration to use mabl to its fullest potential. We believe that getting all of the members of your organization together and working to define some standards around what you plan to accomplish using mabl makes a lot of sense. For some organizations who have relied on manual testing, perhaps getting all your members up and running with the mabl desktop application and having them complete the self-driven courses at mabl University would be a good place to start.

Building off what the users learned from mabl U, having your testing organization build out some smoke tests and flows that interact with basic functionality like logins, simple form submissions, and registration workflows is a good place to start. For organizations with a more sophisticated testing infrastructure, there may be a need to import tests or to copy over existing functionality, which may require some strategy and ordering around tests that are the most important from a business perspective.

No matter what your goals might be for this step, getting folks together and agreeing that you're going to set up some common business language will pay serious dividends down the line. Building off of that last point, coming to an agreement on using a standard naming scheme for your tests and plans will allow you to collaborate and onboard new employees into mabl quickly and easily. It also allows you to move quality assurance resources between departments, projects, and components, and keep using a business language that they can understand.

In a lot of ways, this actually helps with test coverage as well. If you have a suite of plans that are named and described, well, you can understand what gaps might not be included by the virtue of naming and tagging alone. Labeling fills in some of the blanks that may not have been covered by naming conventions alone. Applying labels to both plans and tests will allow you to be granular about product features, tests teams, and goals that a specific plan or task may be trying to achieve. Labeling will also allow some cool product features that we're going to loop back on and discuss a little later on in the presentation.

So let's move on to naming. Naming is a simple, yet powerful concept to working with mabl. One of the most important yet difficult things you'll need to do with your team is helping people understand what you've already tested, and what hasn't been covered by your testing. If you look at the image on the right, you can see an example of some plans I made which might be good examples of names schemes. I use the second bullet point under the plans main bullet to inform my naming of these, including team names, the environment in what type of tests we're running, which will help me quickly understand the intention of all the tests underneath the plan is.

For tests, you should name according to what features or sub-features or components are being covered. So that it's easy to quickly find tests and understand what they're trying to do. Using and maintaining robust descriptions and labels will help significantly in this endeavor as well. At first blush, you might think the name doesn't matter that much. But if you really lean into naming things well, a failing test can tell you a lot of information about what users may be experiencing in real time. For instance, if you have a test called checkout/first time user payment flow, and it was passing, but a recent release has caused that to fail, you can quickly understand what group of users are impacted and inform your development team of where the bug is presenting itself, and allow for quick prioritization and fixing.

As I mentioned in earlier slides, much of this is going to be organization-specific. Perhaps you don't even take payments over the internet. So that last example didn't resonate with you. But I'd like to take a few minutes instead to focus on what happens when you don't curate your workspace and name things properly.

So this slide is one of my personal favorites, what not to do. Imagine that you're a new employee to a company and I asked you to update the test or make sure that the payment system is working correctly. This is the list of plans that you're presented with. The names don't describe what part of the product they're working on. Aside from the ambiguously named check API at the top, very few of them even mention if they're going to be testing browser API, specifically. There's varying spacing, naming conventions, and very little meaning conveyed about what the plan is trying to accomplish.

While onboarding new members to your team, confusion about the intention of plans and tests can cause people to replicate work that's already been done and while it's possible that you may not redo work that's already been done, you're extremely likely to spend more time than you need to searching for what to work on next. It's important to get the team to understand the value gained by staying organized, especially as you scale.

I think that's a great way to start talking about labels. Now, I promise this is the only time I'm going to use an image during this presentation. In this case, I wanted to show off some functionality of labels and descriptions. So looking specifically at the labels in the plan, you'll notice that I labeled these as UI tests belonging to the red team, which are being used to test regressions. For someone new or even somebody who hasn't visited this workspace in a while, they should have an immediate understanding of what this suite of tests is trying to accomplish.

The naming scheme also lets us know that this is being used to test our staging environment, which may have unique UI elements which don't appear in other environments that need to be validated against. Using thoughtful conventions allows for us to discern at a glance the intention of what a plan should do. But as I alluded to before, there's other advantages to using labels. The first of which is a cool and often overlooked feature within mabl.

So now that I've talked about where to find and set up labels, let's talk about how you can use them to navigate quickly through the UI. When you click on any label, a search menu will appear on the right-hand side and give you a list of results sorted by relevance to your label. This allows you to easily find tests plans and flows that are related to what you're currently working on. This functionality has saved me many hours of searching through things while in customers' workspaces, is one of the most useful hidden gems that I'm going to tell you about. I should also mention that this is available anywhere within mabl. If you ever click a label, the search pane will pop up.

Next, let's talk about where to find labels in the UI. If you click on any plan, or test for that matter, you'll see an overview like the page that you see on your screen now. If you click the highlighted pencil icon, you'll be taken to the edit page for the plan, which looks like this. This giant red box up at the top that I've highlighted, gou can find the plan name, description, and labels. Despite my using hyphens when representing multiple words for labels, that's just my personal taste. You can use spaces, underscores, slashes, anything you want. Before we move on, I wanted to take a little bit more time to show you what echo steps are. These aren't directly related to organizing your workspace. It's worth mentioning that echo steps can keep your tests more tidy. So while on the trainer, sometimes when you add steps, flows and steps may have a confusing name. But you can always add echo steps without worrying about changing anything in your tests. These work exactly like comments and code for those of you who come from a development background. You can add any string here and it won't impact anything.

Imagine for a moment that you make a test and one of the ways that you ensure that the page is fully loaded correctly, is to check for the phrase "Hello name" where the name is the name of the logged-in user, and then years later, your company expands internationally and now users are expecting "Bonjour" followed by the name. Your test begin to fail based off of this. Echo steps can talk about the intention that the page is fully loaded, and that the greeting is visible, which would allow future testers to ensure the same thing, but with a different level of specificity.

So jumping back out to our aspirational timeline again, now you've made it through your first few months at mabl. Let's talk through where your quality assurance organization is headed in the 90-180 day time frame. There's a lot of value that you can add to your quality assurance team, but even more value and visibility that can be realized by adjacent teams and management about the value that QA is providing. So the first point on the slide seems to make a lot of sense. But working towards full test coverage is a lofty goal within the first 90 days, because there's a lot of work that needs to be done to make this happen. Much of the work here is organizational and cross-cutting, involving multiple teams and stakeholders. Having multiple departments working together in a workspace to test a product requires coordination and collaboration.

Your first 30-60 days are going to be spent learning the product and building out a foundation of basic functionality within mabl. Organizing what needs to be tested and understanding the logic employed within your application will inform you on what needs to be tested next. This is another time I need to remind you that every organization is different. The timelines they're setting out may not mesh with the processes you have in your organization. Timelines may be longer or shorter.

Before we move on, let me give you an example of some of the challenges that you may face while you're trying to maintain coverage. So in a past role, we had a button that appeared in the UI, called Get Started and when you clicked on this button, a large amount of computation would take place behind the scenes and this was done to find a resource that was the "Least busy." So in my time working at this company, this button underwent more than three dozen revisions on the back end of the product over three years and each one of them worked differently and presented a very different results page. Mabl could have clicked on any of these buttons, but the context behind what's important on the next page was always different.

Revisiting this test to maintain coverage would have been necessary. Thinking back to a few minutes ago, Echo steps could have helped inform us as to what was important for the intention of this testing. So let's think about some examples that apply to a lot of businesses, where people might not consider usability and accessibility. Are you ensuring that forms can accept names on your web app that support special characters like accent marks? Also, keep in mind that accent marks are different for different countries. Are you ensuring that bad data like a name that has numbers in it is disallowed? Lastly, does your web application support various screen sizes in mobile devices? As you continue your journey with mabl fully vetting out your tests using a mixture of regression testing and negative testing can allow you to fully vet your application's functionality.

Before we move on, I just want to say that many businesses don't realize the value gained here. Because testing that a field hasn't changed, despite years’ worth of changes being pushed out to production is massive.

So moving along to our second bullet point. Reviewing the test case failures is an important part of creating a mature quality assurance process and making the most out of mabl. Classifying these test run failures will unlock additional context for people wanting to track the quality of your workspace over time. Understanding how these metrics change will allow you to easily report improvements or slippage in product quality.

Lastly, we strongly suggest that you start considering baking mabl into your CI/CD process and considering how that looks. Customers who have implemented mabl as part of their deployments have realized significant value in accelerating their release cycles and reducing the amount of bugs that make it to their production environment. We'll circle back on this topic in more detail when talking about what this may look like in your organization at a later slide.

So once you and your team have worked out the kinks in your first few tests, you're cranking out high-quality automation. Classifying test failures, when they take place will unlock a great piece of functionality for tracking test quality over time. On the dashboard, when you click one of the tiles that shows an environment's name, you can have an overview of test history related to that environment. If your team is diligent about classifying failing tests, you can quickly understand the trajectory of your product quality. This is an often overlooked feature that gives great visibility to both how product management and quality assurance on how your team is utilizing mabl and how product quality is evolving within your organization.

Before I move on from this slide, I think that it's also important to show that over time, more bugs are being caught in QA and fewer are making it into production. For those of you who are wondering where these test failures are found and how to classify them within the UI, if you click on any failed test in your run history, near the top of the page, you'll see a drop-down menu that says add failure reason. As you fill these out, the environments on the dashboard will start to populate with data automatically. Just because you haven't done this in the past doesn't mean that you can't start doing this today. This is great data for showing time and dollar savings for your quality assurance organization.

Moving along, let's give a quick bit of thought to how CI/CD planning is going to work. mabl offers a suite of integrations as well as some generic endpoints which can be triggered via cURL calls to make testing and results available to a suite of excellent tools. With system integrations like Jira, GitHub, Slack, Jenkins, Big Query, and many others, mabl is ready to work with a tech stack that you've already built for your development organization. Before making any changes to your setup, consider for a moment how this will affect how many runs per month you are using. In some companies, a development organization might release 10-20 components a day and if you were to run a full UI regression suite, for each release with 25 tests, this could add up to hundreds or thousands of additional runs each month.

So going back to our aspirational timeline one last time. Let's talk about six months and beyond. Now, you've learned everything that there is to know about mabl, you've leveled up your team and now you're trying to get the most value out of mabl. Let's talk about what this looks like. When you were first getting started a few months ago, you were just learning how to use the trainer. Now you're an expert. At this point, we've built a basic smoke tests or regression suite, and have labels for each type of test within a plan. You're looking to bake mabl in your workflow even more by adding system integrations to Jira, Slack, and GitHub, or nearly a dozen other integration partners which we fully support. You have multiple teams working together and fewer bugs are getting into production and when they do you understand exactly why they happened and creating Jira issues is quick and easy.

Now, no testing effort is fully vetted without end-to-end testing and you're just starting to dip your toes into the water. For those of you who aren't familiar with what end-to-end testing is, end-to-end testing is when you are testing the functionality and performance of an application under circumstances that are similar to real-world usage. If you want to get more information on what this looks like using mabl, we have another session immediately after this one hosted by Jeff Zubka one of our engineering managers talking through how your organization might approach this effort.

As a final point, deployment events are a logical next step to be able to getting the most out of mabl. Deployment events are kicked off using a cURL call and can fire off specific tests within a plan that match a specific label. With a little bit of work, you can kick off all the tests with the "search label" or "navigation label" or any label on deployments you make upstream. This allows you to make part of acceptance testing that mabl tests pass, which is the last piece of added value you can squeeze out of your highly optimized workspace. These tests will kick off in the cloud so that information about whether they pass or fail will be immediately available to your entire team. There's documentation at the link on the slide, which will be a good place to start. But if you need more help, the support team is ready and willing to work for you.

So in summary, we've covered a lot of ground in a short period of time here today. We talked about organization and how that's going to be a coordinated team effort while leaning into how you can use flows, labels, and descriptions to keep your workspace organized and easily understood by both new and existing mablers. We talked a lot about naming and deciding what works and what doesn't for your organization. We also looked at organizing test failures and how that will propagate environment-specific metrics that you look at.

We closed off from the final slide by talking a little bit about system integrations and deployment events to really take enabled to the next level. In short, our aspirational timeline for setting up mabl is really based on how to quickly create, scale, and optimize your testing experience, and to give visibility into how mabl is being used to save time, money, and effort within your quality assurance organization.

Thanks, everyone.

How to Set Up Your Workspace and Support Your Quality Assurance Infrastructure

Transcription

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Gabe Alvarez-Millard

Damien Bell

Quality Engineering Resources

Top 5 Lessons Learned in 2024 State of Testing in DevOps Report

New Features Strengthen Automated API Testing in mabl

Delivering Excellence in Customer Support