Cross-browser testing is the process of testing whether web-based applications will work properly in a variety of different web browsers. It is designed to ensure that applications being built in one browser are compatible with others. Cross-browser testing will identify if there are problems rendering content across different browsers - for example, a feature available in the most modern browsers may not render in older browsers.
In short, it answers the question: “Do users see your web site or app the way that you want them to see it?”
Why is Cross Browser Testing Important?
HTML and CSS are standards which are generally designed to work in all web browsers. These languages are simple markup rules to tell a browser how to behave when it receives a set of instructions.
However, even while the latest browser versions are becoming increasingly more in parity as to how they render code, there are still many opportunities for incompatible HTML and CSS rules and frameworks to ruin your user experience. Discrepancies in the way that code is interpreted is particularly true of older web browsers. New developments in the code come out all the time, particularly with HTML5 and CSS3, however not all browser vendors have made the decision, or have had the opportunity, to adopt the latest web standards.
What this means is that exciting new effect that you created that works fine in Chrome may render slightly differently in Firefox, or Safari, or not at all in Internet Explorer. Yes, it is true that IE has been deprecated in favor of Edge (which has its own issues), but this does not mean that all users have stopped using it. In fact, if someone is using an older computer, there’s a high likelihood that they may not see all of your fancy new designs, or they may render in a way that looks ugly, or worse, is completely unusable.
It’s never a good idea to blame the end-user. You are providing the product, and there’s a chance that your users may not be able to use the latest and greatest version, regardless of how good it is. We need to meet the users where they live, and not expect them to jump through hoops (such as installing a specific piece of software to meet your needs). The truth is that if we expect them to do this, in many cases they simply won’t.
We do cross-browser testing to:
Identify broken functionality (the site is completely unusable)
Recognize usability issues caused by an incompatible browser or browser version.
Improve the way the web app or site renders the page to ensure a positive user experience.
The goals of cross-browser testing are twofold:
Determine if the appearance of a web page varies from one browser to another
Verify that the app under test actually functions in all browsers being tested
Note: Some applications are specifically designed to work in one browser or other. Many of Google’s applications are designed to run best in Chrome because they own it. Similar things may be true for Edge or IE and Microsoft.
This is not a recommended approach; however, each has its own business reasons for adopting this method. It’s inadvisable for most companies to force users to use one or other browser.
What causes cross browser issues:
Some browsers have bugs.
No matter how well a piece of software is designed to be built, it is still built by people who, by nature, are flawed. It’s always possible for browsers, and sometimes even newer releases, to not have had all the problems weeded out during development, and bugs can make their way into production. As web developers, we do not have much control over the bugs within the browsers themselves; they are client-side by nature, and we cannot control what exists on a user’s computer. However, we do need to ensure that we have identified if there are any issues within each browser, we test to ensure that our content is rendered in an acceptable fashion.
Different levels of support for technology
Browsers are not all made by the same companies, and for whatever reasons, they choose to follow or not follow official W3C guidelines for rendering HTML and CSS. Older browsers, for instance, may not be able to render some of the more advanced features available within HTML5 and CSS3.
Some browsers are designed to have less functionality
In some cases, it doesn’t make sense to have browsers be able to handle each aspect of code quite simply due to limitations in the devices for which they are being designed. Mobile devices tend to have a little less processing power than a desktop computer. As a result, the browsers designed for smartphones may provide slimmed down interpretations of HTML. They may also may not provide functionality to handle some of the more advanced animations, which may not work particularly well in a hand-held device.
Cross Browser Testing Best Practices
Before beginning any cross-browser testing, it makes sense to have a plan in place regarding the content of the site
It’s important to determine exactly what content and functionality a site should have. This should be broken up into categories including
What are the ideal features that should be rendered?
Which features fall into the “nice to have” category?
What is the minimal acceptable functionality and display the site should render, in the advent that someone is using an out of date browser?
Every web browser runs off of a browser engine. These engines are designed specifically to transform HTML into web page resources that can be viewed by the browser. They are not standalone pieces of software but are absolute necessities in how web browsers work.
There are a number of different engines on the market; and each one translates HTML somewhat differently. It’s not entirely necessary to know the details of each engine, except to understand that they are what drive how different browsers work.
There are many but the main ones are:
Blink: Chrome, Opera, Microsoft Edge
EdgeHTML: formerly used in Microsoft Edge; used in Universal Windows Platform Apps.
Webkit: Safari, and all iOS browsers
Gecko: Firefox (and Thunderbird email client)
Trident: Internet Explorer (and Outlook email client)
It’s important to note that not every browser engine will run in every operating system. For instance, EdgeHTML will only run in the Windows OS, and Webkit will run on everything except Windows.
Identify which browsers to test. There are many. In an ideal world, we would test all of them, but in the real world, it's best to narrow it down to a few, depending on the browsers and devices most of your users are on. There are plenty of production analytics tools out there (such as from Google Analytics or Matomo) that give you visibility into what browsers your users are using. You may assume to test on both Chrome and Firefox, but there may be a substantial number of users that are using Safari, Edge, and IE. You may also see some people using Opera.
Not only should you test multiple browsers, but you should test multiple versions. For instance, mabl always runs tests on the latest version and version before that, by default. While it’s considered best practice for people to upgrade their browsers regularly, many simply do not. Some companies even limit access to specific browsers, or have a policy that dictates when their software can be updated and restricts their employees to upgrade their software on their own. Bottom line is, not using a compatible version or browser cannot be seen as the fault of the user.
It’s best to divide the browsers you wish to test into three grades. These are below:
A grade: These consist of the top-notch, most common modern browsers. These are known to be fully capable. For testing purposes, everything should work perfectly in these devices.
B grade: These are older and less capable browsers. They are known to have compatibility issues. For this type of browser, it’s important to test to ensure that you are giving at least a satisfactory access to your content and that the main functionality continues to work. Can your users still access and complete the CTA on each page?
C grade: These consist of a set of browsers which are rare and/or unknown. You cannot test these, but you can include fallbacks to handle situations where they won’t render your content, much like in the B-grade browsers. For example, checks can be added to render a simpler variation of the site or redirect the user if the user's browser doesn't meet the most minimum requirements.
To translate this into real world terms, as for a bare minimum, it’s wise to test on at least the last few versions of the following A-grade browsers
Beyond this, you will want to ensure you can at least provide a minimal acceptable experience on B-grade browsers such as IE 8,9
Finally, you will also absolutely want to make sure your site is accessible with WCAG AA compliance.
If you are planning on working with some cutting-edge tools, it makes sense to be forward thinking, and you may wish to consider testing on several pre-release (i.e. still in beta testing) browsers. Here’s a listing of some of these, and access to their developer tools.
- Firefox Developer Edition
- Edge Insider Preview
- Safari Technology Preview
- Chrome Canary
- Opera Developer
You will also wish to do testing on multiple different devices, as different browsers render differently depending on the machine on which it is being displayed. These should include mobile devices, tablets, laptops, desktops, and possibly some television browsers.
You will also wish to test different viewport sizes. For instance a 1366 x 768 resolution, common among some laptops will look different than 1024 x 768, or 800x600 which was common on old small monitors. You can consider testing a set of breakpoints (e.g. tablets, phones) to test how responsive your app is, on different devices, and provide mobile and desktop presentation modes for your app if the way it's rendering can't be resolved.
When to start cross-browser testing?
As with all testing techniques, particularly if you are taking a shift-left philosophy to application testing, you will want to start your cross-browser testing as soon as possible. This means during development, when the application is complete, and even after it has been released.
It’s wise to get all functionality working as best as possible on all browsers during the development process in order to save in tech debt costs. mabl makes this easy with a scriptless test creation process and a built-in testing cloud that provides a no-ops approach to cross-browser testing.
When you get into serious testing phases, consider the following approach:
Test in a few stable browsers, in three main OS's (Windows, Apple, Linux) if possible
Test using common phone and tablet browsers
Test in any other browsers you have on your list
Run some lo-fi accessibility testing to ensure that differently abled individuals can access your content effectively. Try navigating with only a keyboard, or with a screen reader.
What should you test?
The end-to-end user experience - Test all the interconnected pieces of your website - from server-side functionality, as well as client-side code which must run in the browser, and even external services like email and file downloads
GUI – make sure that the interface renders cleanly and clearly in all browsers tested
Response to input – Make sure that inputs effectively respond the same in all browsers, and if not, create an alternate method for access to important pieces of functionality
Performance – you will want to make sure that the site will load in a reasonable amount of time in all devices
Note that not all things that are problematic in different browsers are necessarily “errors” but are just display issues. For this reason you will wish to set clear benchmarks and goals for each of your test plans.
Manual Cross Browser Testing
Manual testing is what it sounds like. For manual testing, the business should identify which browsers the application should support. Testers will then need to manually run through each of the same test cases in each environment and make notes on how the application responds in each browser and report any bugs that come up along the way. Much of this testing can be done either by oneself, or by user groups, possibly in conjunction with usability testing.
The flaws in manual testing are self-evident; it’s not possible to cover each and every browser this way, upon every code change, and it might be difficult to gain access to devices for each test case. Cell phones themselves have such a wide variability that it is difficult to ensure that content will render properly on every devices. To summarize, manual testing is also both costly and time consuming, but many teams face the challenge of standing up an automated testing process that is reliable, easy to maintain, and fast to execute.
Automating Cross Browser Testing
Using automated techniques is far more ideal. It typically involves using software such as mabl, which contains the test infrastructure (cross-browser) to run an unlimited number of tests in parallel (it's fast), and it easily plugs into both developer workflow and QA workflows (easy to use). mabl also provides screenshots from your app at every step of every test so that it's easy to verify what the app looks like on any viewport and browser you are testing on.
While not every aspect of cross-browser testing makes sense to be handled with automation, particularly for some of the more obscure browsers and devices, a great deal of time can be saved by using mabl to run through some of the most popular browsers.
mabl is an automated testing solution that makes it easy to integrate cross-browser testing into your CI/CD pipeline; you can treat cross-browser testing as part of your normal regression testing processes, and eventually every other aspect of testing you perform, ranging from system, end-to-end, visual testing and more.
Sign up for a free trial of mabl for reliable and effortless cross-browser testing.