Whether you’re new to mabl, or a seasoned user, good workspace hygiene is critical to your team’s testing success. In this interactive session, Damien will share best practices for organizing your workspace, creating effective tests, and understanding test results. Sign in to your own mabl workspace and follow along!
Good afternoon, everyone. I hope that you are having a great day so far. Let's talk about setting your workspace up for success. So, some key points that I want you to take away from today are first, understanding how to implement some mabl best practices, supporting a solid foundation for your quality assurance infrastructure. Second, staying organized to scale your company presence as you onboard new products, people, and teams in your mabl workspaces. Third, mabl as a product has a lot of features that people overlook that help accelerate teams as they organize and categorize their work. We're going to be diving into some of those. Lastly, despite there being a huge gap between different organizations in terms of size, product functionality, team member skill level, and testing methodologies. mabl can fit into what you're trying to do, and make it significantly easier.
Before we get too far ahead of ourselves, I'm Damien. I run the support organization over here at mabl. My job is to manage a team of support engineers who help our customers with the development and debugging of tests, environments, networks, honestly, you name it. We're the Swiss Army knives over here at mabl. I try to be a voice for our customers and really help them be heard within mabl. I believe that there's a need for us to learn from what you all are trying to do and to ensure that we're moving forward together.
This presentation is based on how some of our most successful customers have set up mabl to work in their workspaces. So to start off with, let's talk a little bit about the primary components of a workspace within mabl. On the right, you can see a hierarchy of how each part of the workspace interacts with mabl. I'm going to take a minute to walk through how I think of these so that we can all start off on the same page.
Firstly, we have the test all the way down at the bottom. Since many of the following points are going to talk about tests, let's begin by defining that. For me, I think of a test is a series of automated actions called steps, which tell mabl how to interact with and validate a specific workflow or functionality within your application. Some examples of steps include navigating to a URL, finding a button, inserting text in a form, and dozens of other useful page interactions. A step is the smallest building block that can be used inside of a test.
Next, plans. A plan is an ordered group of tests that hold configuration for how mabl carries out your automation. Some factors that are taken into account are the environment, the application, the execution frequency, and which browsers you're targeting.
Next up, and not in the diagram on the right is labels. A label is a word or a phrase that you attribute to tests and plans for the purposes of classification organization and in the case of plans deployments via CI/CD.
Next up is a description, which is a brief summary or anecdote about a test or a plan that tells your team what it tests and what the context is. Lastly, is flow. A flow is a reusable subset of test steps that you've designated and can be inserted into other tests. An analogy of how you might think of this in a really approachable way might be to consider how Google is structured. The workspace overall could represent all of the products and services owned and operated by Google. The environments here could be Dev, QA, prod, and others. Each of these could have its own special hardware and software configuration credentials and data which acts completely independently of the others. The apps, in this case, might be Gmail, Google Search, and Google Calendar and while each of these are owned by Google and might share some interaction, they don't share many features and are likely worked on by separate teams within the organization.
Plans are groups of tests, which seek to fully validate a piece of a component feature or interaction within the application. An individual test could be ensuring that a single page in the application works the way that you intended to. Thinking about Gmail, perhaps you'd want to log in, click the compose button, and then after sending your email, click sent items to ensure that appears there. Lastly, as an example of a flow, which we're not going to talk about much throughout this presentation, are reusable steps that you can use to automate often reused actions and the example above where we talked about logging into Google, the insertion of the username and password could be an example of a flow that you'd like to reuse elsewhere.
Let me show you what some of these look like inside of mabl. When in your workspace, if you click plans on the left hand side of the page, you'll see something like this when the page comes up. There'll be a list of plans that have descriptions, labels, and environment data that describe what they intend to do. Notice the filters up near the top of the page, which allows you to search for tests depending on their name, label, or environment.
Here's an example of the test page. This will list out all of the tests that you have in your workspace, you can see how many plans the test is in by looking at the plan count column. The labels and descriptions are also seen in this view as well. When you click on an individual test in the previous view, you'll be taken to this page which shows you much more detail about a single test. You can see the name of the test up near the top of the page along with the description and the labels from the previous view. On this page, you can also see the quote train steps, which is midway down the page there, which are those building blocks of automation that we mentioned back on the slide with the diagram. These tests allow us to understand what each interaction in the test is trying to accomplish. When considering how you could go about setting up your mabl workspace to work for you, I took into consideration how long it would take to get things working perfectly.
This timeline like many of these slides is going to be subjective, but it does give some good guidelines for you to consider when your organization starts automating with mabl and if you're already a mabl customer, don't fret. A lot of this might be new information to you as well.
The first thing I wanted to talk about today is mabl University. mabl U is one of the best ways that you can quickly come up to speed on how to use mabl. Believe it or not, this is one of the ways that we onboard internal employees who aren't familiar with how to use mabl yet. For learning the basics, as well as a few more advanced use cases, mabl University is going to provide you with a lot more engaging exercises than just working through the help documentation. It is frequently updated. So it always has the latest and greatest stuff that we have to offer here at mabl.
For our second point, a recurring theme that we're going to mention a lot throughout this exercise is that there's a strong need for collaboration to use mabl to its fullest potential. We believe that getting all of the members of your organization together and working to define some standards around what you plan to accomplish using mabl makes a lot of sense. For some organizations who have relied on manual testing, perhaps getting all your members up and running with the mabl desktop application and having them complete the self-driven courses at mabl University would be a good place to start.
Building off what the users learned from mabl U, having your testing organization build out some smoke tests and flows that interact with basic functionality like logins, simple form submissions, and registration workflows is a good place to start. For organizations with a more sophisticated testing infrastructure, there may be a need to import tests or to copy over existing functionality, which may require some strategy and ordering around tests that are the most important from a business perspective.
No matter what your goals might be for this step, getting folks together and agreeing that you're going to set up some common business language will pay serious dividends down the line. Building off of that last point, coming to an agreement on using a standard naming scheme for your tests and plans will allow you to collaborate and onboard new employees into mabl quickly and easily. It also allows you to move quality assurance resources between departments, projects, and components, and keep using a business language that they can understand.
In a lot of ways, this actually helps with test coverage as well. If you have a suite of plans that are named and described, well, you can understand what gaps might not be included by the virtue of naming and tagging alone. Labeling fills in some of the blanks that may not have been covered by naming conventions alone. Applying labels to both plans and tests will allow you to be granular about product features, tests teams, and goals that a specific plan or task may be trying to achieve. Labeling will also allow some cool product features that we're going to loop back on and discuss a little later on in the presentation.
So let's move on to naming. Naming is a simple, yet powerful concept to working with mabl. One of the most important yet difficult things you'll need to do with your team is helping people understand what you've already tested, and what hasn't been covered by your testing. If you look at the image on the right, you can see an example of some plans I made which might be good examples of names schemes. I use the second bullet point under the plans main bullet to inform my naming of these, including team names, the environment in what type of tests we're running, which will help me quickly understand the intention of all the tests underneath the plan is.
For tests, you should name according to what features or sub-features or components are being covered. So that it's easy to quickly find tests and understand what they're trying to do. Using and maintaining robust descriptions and labels will help significantly in this endeavor as well. At first blush, you might think the name doesn't matter that much. But if you really lean into naming things well, a failing test can tell you a lot of information about what users may be experiencing in real time. For instance, if you have a test called checkout/first time user payment flow, and it was passing, but a recent release has caused that to fail, you can quickly understand what group of users are impacted and inform your development team of where the bug is presenting itself, and allow for quick prioritization and fixing.
As I mentioned in earlier slides, much of this is going to be organization-specific. Perhaps you don't even take payments over the internet. So that last example didn't resonate with you. But I'd like to take a few minutes instead to focus on what happens when you don't curate your workspace and name things properly.
So this slide is one of my personal favorites, what not to do. Imagine that you're a new employee to a company and I asked you to update the test or make sure that the payment system is working correctly. This is the list of plans that you're presented with. The names don't describe what part of the product they're working on. Aside from the ambiguously named check API at the top, very few of them even mention if they're going to be testing browser API, specifically. There's varying spacing, naming conventions, and very little meaning conveyed about what the plan is trying to accomplish.
While onboarding new members to your team, confusion about the intention of plans and tests can cause people to replicate work that's already been done and while it's possible that you may not redo work that's already been done, you're extremely likely to spend more time than you need to searching for what to work on next. It's important to get the team to understand the value gained by staying organized, especially as you scale.
I think that's a great way to start talking about labels. Now, I promise this is the only time I'm going to use an image during this presentation. In this case, I wanted to show off some functionality of labels and descriptions. So looking specifically at the labels in the plan, you'll notice that I labeled these as UI tests belonging to the red team, which are being used to test regressions. For someone new or even somebody who hasn't visited this workspace in a while, they should have an immediate understanding of what this suite of tests is trying to accomplish.
The naming scheme also lets us know that this is being used to test our staging environment, which may have unique UI elements which don't appear in other environments that need to be validated against. Using thoughtful conventions allows for us to discern at a glance the intention of what a plan should do. But as I alluded to before, there's other advantages to using labels. The first of which is a cool and often overlooked feature within mabl.
So now that I've talked about where to find and set up labels, let's talk about how you can use them to navigate quickly through the UI. When you click on any label, a search menu will appear on the right-hand side and give you a list of results sorted by relevance to your label. This allows you to easily find tests plans and flows that are related to what you're currently working on. This functionality has saved me many hours of searching through things while in customers' workspaces, is one of the most useful hidden gems that I'm going to tell you about. I should also mention that this is available anywhere within mabl. If you ever click a label, the search pane will pop up.
Next, let's talk about where to find labels in the UI. If you click on any plan, or test for that matter, you'll see an overview like the page that you see on your screen now. If you click the highlighted pencil icon, you'll be taken to the edit page for the plan, which looks like this. This giant red box up at the top that I've highlighted, gou can find the plan name, description, and labels. Despite my using hyphens when representing multiple words for labels, that's just my personal taste. You can use spaces, underscores, slashes, anything you want. Before we move on, I wanted to take a little bit more time to show you what echo steps are. These aren't directly related to organizing your workspace. It's worth mentioning that echo steps can keep your tests more tidy. So while on the trainer, sometimes when you add steps, flows and steps may have a confusing name. But you can always add echo steps without worrying about changing anything in your tests. These work exactly like comments and code for those of you who come from a development background. You can add any string here and it won't impact anything.
Imagine for a moment that you make a test and one of the ways that you ensure that the page is fully loaded correctly, is to check for the phrase "Hello name" where the name is the name of the logged-in user, and then years later, your company expands internationally and now users are expecting "Bonjour" followed by the name. Your test begin to fail based off of this. Echo steps can talk about the intention that the page is fully loaded, and that the greeting is visible, which would allow future testers to ensure the same thing, but with a different level of specificity.
So jumping back out to our aspirational timeline again, now you've made it through your first few months at mabl. Let's talk through where your quality assurance organization is headed in the 90-180 day time frame. There's a lot of value that you can add to your quality assurance team, but even more value and visibility that can be realized by adjacent teams and management about the value that QA is providing. So the first point on the slide seems to make a lot of sense. But working towards full test coverage is a lofty goal within the first 90 days, because there's a lot of work that needs to be done to make this happen. Much of the work here is organizational and cross-cutting, involving multiple teams and stakeholders. Having multiple departments working together in a workspace to test a product requires coordination and collaboration.
Your first 30-60 days are going to be spent learning the product and building out a foundation of basic functionality within mabl. Organizing what needs to be tested and understanding the logic employed within your application will inform you on what needs to be tested next. This is another time I need to remind you that every organization is different. The timelines they're setting out may not mesh with the processes you have in your organization. Timelines may be longer or shorter.
Before we move on, let me give you an example of some of the challenges that you may face while you're trying to maintain coverage. So in a past role, we had a button that appeared in the UI, called Get Started and when you clicked on this button, a large amount of computation would take place behind the scenes and this was done to find a resource that was the "Least busy." So in my time working at this company, this button underwent more than three dozen revisions on the back end of the product over three years and each one of them worked differently and presented a very different results page. Mabl could have clicked on any of these buttons, but the context behind what's important on the next page was always different.
Revisiting this test to maintain coverage would have been necessary. Thinking back to a few minutes ago, Echo steps could have helped inform us as to what was important for the intention of this testing. So let's think about some examples that apply to a lot of businesses, where people might not consider usability and accessibility. Are you ensuring that forms can accept names on your web app that support special characters like accent marks? Also, keep in mind that accent marks are different for different countries. Are you ensuring that bad data like a name that has numbers in it is disallowed? Lastly, does your web application support various screen sizes in mobile devices? As you continue your journey with mabl fully vetting out your tests using a mixture of regression testing and negative testing can allow you to fully vet your application's functionality.
Before we move on, I just want to say that many businesses don't realize the value gained here. Because testing that a field hasn't changed, despite years’ worth of changes being pushed out to production is massive.
So moving along to our second bullet point. Reviewing the test case failures is an important part of creating a mature quality assurance process and making the most out of mabl. Classifying these test run failures will unlock additional context for people wanting to track the quality of your workspace over time. Understanding how these metrics change will allow you to easily report improvements or slippage in product quality.
Lastly, we strongly suggest that you start considering baking mabl into your CI/CD process and considering how that looks. Customers who have implemented mabl as part of their deployments have realized significant value in accelerating their release cycles and reducing the amount of bugs that make it to their production environment. We'll circle back on this topic in more detail when talking about what this may look like in your organization at a later slide.
So once you and your team have worked out the kinks in your first few tests, you're cranking out high-quality automation. Classifying test failures, when they take place will unlock a great piece of functionality for tracking test quality over time. On the dashboard, when you click one of the tiles that shows an environment's name, you can have an overview of test history related to that environment. If your team is diligent about classifying failing tests, you can quickly understand the trajectory of your product quality. This is an often overlooked feature that gives great visibility to both how product management and quality assurance on how your team is utilizing mabl and how product quality is evolving within your organization.
Before I move on from this slide, I think that it's also important to show that over time, more bugs are being caught in QA and fewer are making it into production. For those of you who are wondering where these test failures are found and how to classify them within the UI, if you click on any failed test in your run history, near the top of the page, you'll see a drop-down menu that says add failure reason. As you fill these out, the environments on the dashboard will start to populate with data automatically. Just because you haven't done this in the past doesn't mean that you can't start doing this today. This is great data for showing time and dollar savings for your quality assurance organization.
Moving along, let's give a quick bit of thought to how CI/CD planning is going to work. mabl offers a suite of integrations as well as some generic endpoints which can be triggered via cURL calls to make testing and results available to a suite of excellent tools. With system integrations like Jira, GitHub, Slack, Jenkins, Big Query, and many others, mabl is ready to work with a tech stack that you've already built for your development organization. Before making any changes to your setup, consider for a moment how this will affect how many runs per month you are using. In some companies, a development organization might release 10-20 components a day and if you were to run a full UI regression suite, for each release with 25 tests, this could add up to hundreds or thousands of additional runs each month.
So going back to our aspirational timeline one last time. Let's talk about six months and beyond. Now, you've learned everything that there is to know about mabl, you've leveled up your team and now you're trying to get the most value out of mabl. Let's talk about what this looks like. When you were first getting started a few months ago, you were just learning how to use the trainer. Now you're an expert. At this point, we've built a basic smoke tests or regression suite, and have labels for each type of test within a plan. You're looking to bake mabl in your workflow even more by adding system integrations to Jira, Slack, and GitHub, or nearly a dozen other integration partners which we fully support. You have multiple teams working together and fewer bugs are getting into production and when they do you understand exactly why they happened and creating Jira issues is quick and easy.
Now, no testing effort is fully vetted without end-to-end testing and you're just starting to dip your toes into the water. For those of you who aren't familiar with what end-to-end testing is, end-to-end testing is when you are testing the functionality and performance of an application under circumstances that are similar to real-world usage. If you want to get more information on what this looks like using mabl, we have another session immediately after this one hosted by Jeff Zubka one of our engineering managers talking through how your organization might approach this effort.
As a final point, deployment events are a logical next step to be able to getting the most out of mabl. Deployment events are kicked off using a cURL call and can fire off specific tests within a plan that match a specific label. With a little bit of work, you can kick off all the tests with the "search label" or "navigation label" or any label on deployments you make upstream. This allows you to make part of acceptance testing that mabl tests pass, which is the last piece of added value you can squeeze out of your highly optimized workspace. These tests will kick off in the cloud so that information about whether they pass or fail will be immediately available to your entire team. There's documentation at the link on the slide, which will be a good place to start. But if you need more help, the support team is ready and willing to work for you.
So in summary, we've covered a lot of ground in a short period of time here today. We talked about organization and how that's going to be a coordinated team effort while leaning into how you can use flows, labels, and descriptions to keep your workspace organized and easily understood by both new and existing mablers. We talked a lot about naming and deciding what works and what doesn't for your organization. We also looked at organizing test failures and how that will propagate environment-specific metrics that you look at.
We closed off from the final slide by talking a little bit about system integrations and deployment events to really take enabled to the next level. In short, our aspirational timeline for setting up mabl is really based on how to quickly create, scale, and optimize your testing experience, and to give visibility into how mabl is being used to save time, money, and effort within your quality assurance organization.
Before we field any questions, I just want to mention we have an absolutely incredible team of folks here at mabl, who work really hard to make sure that everyone who has questions about how to make mabl work for their business. If you can't think of a question right now, when something pops in your head 30 or 45 minutes, feel free to stop and ask. We're honestly happy to help. With that said floor is open to any questions that you folks may have.
Thank you, Damien, you truly covered a lot and I would recommend anyone new to mabl definitely download the PDF of these slides after the event, because there's a lot of great stuff to reference there. What questions does everyone have? Let's get started. All right, Damien, any tips for workspace hygiene? Is there a good way to deprecate tests without creating problems for my teammates?
Yeah. So in general, I think that when doing any type of workspace hygiene, it's important that you're collaborating, we do have the ability to archive or just outright delete tests that no longer suit the direction that your product is moved in. As I've mentioned a number of times, that viewing this as a collaborative effort and getting everybody to agree on this and say, maybe we've done this other functionality over here, and this test no longer is necessary before deprecating, it's a good idea. But otherwise, I think that just overall, involving multiple people, and not just making decisions on your own is the best way to keep things tidy.
Are there any labeling strategies you could suggest to help teams organize in group work?
So there were a few on the slide that I had mentioned earlier, as Gabe so wonderfully mentioned, when he rejoined us, downloading these slides is great. The one that I used a little bit earlier was mentioning the team, as well as the environment that something's working on followed by a name. But really, you can use whatever naming strategies make the most sense to you whether something's a regression test, a smoke test, an end-to-end test or simply naming the component that you're going to be working on, are all really valid strategies as far as naming goes, and then the labels kind of fall into place based off of what you've named your test.
Is there an additional cost to access content in mabl University?
No, everything in mabl University is completely and totally free. So what's interesting about mabl University is there are so many new updates that have come to it just over the past year. When we mentioned it to people in support, a lot of the time people say, oh, I don't want to pay for anything. That sounds like it's going to be expensive and no, it's free and it's a once again, linked in the slides that we have. So absolutely consider checking it out. Even if you are somebody who's been using mabl for a while, as I said, we cover a lot of ground in there and go over some advanced use cases that may not be immediately apparent to people who don't come from a quality assurance automation background.
How can I share my local tests within my workspace?
Yeah. So if you have tests that you're just working on locally, say within the trainer, the moment that you save a test, it goes to the cloud and other members on your team can begin collaborating and working on it and whether or not you run those tests in the Cloud it's how we base our billing. So the moment you save it, it goes to the cloud, and everyone can help you collaborate on it.
If you have multiple live environments, can you direct the test in the plan to specific URLs?
Yes. So if you have multiple live environments, say you have three production servers, which are load-balanced, but you've used a strategy, where you can say, prod-1.google.com, that will still load. So you can direct where you're going to kind of override your load balancing strategy. You can bake that into the environment variable that you set up. So when you're creating an environment within mabl, you're given this opportunity to define a base URL that you're going to be taking actions against and so when you first load up the trainer, it will pull up that URL and so you have the ability to navigate to any URL of your choosing after that. So that's a really long way of saying yes.
We have one more question. First time using mabl University. Is there help support if I have questions or get stuck?
Absolutely, and one of the really important things that I like to talk about as being the person who runs the support organization, is that the first and most important rule is to be helpful. Any question you have about mabl, please feel free to bring it to us. If you can't understand how to make a specific type of assertion, or how to get something to work the way you want, or something has changed in your tests and they went from passing to failing and you can't figure out why and you just want some expert help, we're here for it. So, please don't hesitate to reach out.
Awesome. All right and with that, we have run out of questions. Damien, thank you so much for your wisdom today. If you do have questions that come up later after the event, we'll connect you with our speakers after the conference. So thanks so much for joining and see you at our next session at Experience.