End-to-End Testing

End-to-end testing is a software testing method which tests the flow of an application from start to finish. It is a complete test to ensure that a piece of software has accomplished all of its functional needs.

While most other forms of testing test individual components of an application, end-to-end testing can be seen as a sort of combination of these: the test of an application as a whole. 

End-to-end testing is very extensive, and as a result considered to be a very expensive kind of testing; it covers each aspect of how an application runs, in a real-world, production-like scenario.  

The amount of time constructing and running end-to-end tests mean that these are generally not run regularly throughout the testing process but are reserved for final runs. While it would be ideal to be able to run this sort of test more often, it is typically only run a few times during a testing process due to its expense; typically after most, if not all, of the application has been built.

End-to-end testing can be performed at any stage of development where there is at least a theoretically working product, but it is typically only required to be done near the end of the process and usually after functional and system testing.

Why is End-to-End Testing Important?

For all websites and software, it is important to ensure that they work properly as they are designed. Most sites are comprised of complex systems with many interconnected subsystems. If any subsystem fails, the entire application could fail. For this reason, we generally test each aspect of our applications as individual components. We also, however, need to make sure that each of these systems work together, so we engage in integration testing to ensure that one subsystem will communicate effectively and accurately with all related subsystems. 

Beyond looking at individual and related components in this way, we need to be certain that the entire application operates properly as a whole.

End-to-end testing is designed to ensure that all dependencies of an application are functioning correctly and delivering accurate results both between various subsystems and also with the end-user. End-to-end testing makes sure the entire system flows the way that it was designed to. In some ways it is like system testing, but it tests beyond system interactions and includes real-world behavior with human interaction.

We need to know that our software will work for an end user in precisely the way that it was designed to, with a series of normal inputs and expected responses.

Let’s break this down a little to look at individual methods of testing that happen throughout the life cycle of application development.

Throughout the testing process, we generally test individual components at a micro level. This can include unit testing, which is a test to check if individual components work specifically within a closed independent environment. For example, a unit test may determine whether a form takes input data and submits it correctly to a database. At a slightly higher level we have integration testing, which tests how individual components interact with each other. For instance, if the above form submits data correctly, does this trigger a process to enable some other activity as designed?

We also engage in other forms of testing throughout development, such as smoke tests (to make sure a system does not “catch fire”) and sanity tests (individual tests to ensure that things are working at a nominally basic level if a change has been made). Sanity tests are related to regression tests, which are briefly discussed below.

System tests are tests of an entire application, and are most similar to end-to-end testing, but they have some major differences, which will also be discussed below.

End-to-end testing tests the following criteria:

  • It ensures that the system works as designed from beginning to end.
  • It creates a much more extensive form of testing that covers all subsystems in the architecture and tests how they work as a whole.
  • It helps to identify any issues that may come up with each individual system and ensures that the entire system flows more efficiently and effectively as a whole, including interaction with external inputs.

End-to-end Testing Process

To create a solid test framework, it’s important to break the application down into its individual parts. This involves defining user functions, defining a set of conditions for each user function, and the creation of several test cases.

While this may seem like a large amount of work, good end-to-end tests can make use of modules that have been created for other forms of testing, such as unit tests, integration tests, system tests, and more.

User functions

Being able to effectively test the application in any sort of reusable way requires constructing a framework, which is a series of functions or methods designed to test each aspect of the software.

The first part of framework development involves listing all the features that go into the application. This also should include a specification of which components interact or interconnect with each other. 

Next, define what input data is to be used, and what output data is to be expected for each feature within the application.  

From here, you will need to define the relationship between the functions and determine whether any specific function can be reused, or whether these need to be written independently for each aspect of the application.

Conditions

Next you will need to identify a set of conditions for each function. In other words, the circumstances in which a specific response will be required. This should include timing, sequencing and any conditions for specific data that could affect the working of any other aspect of the application. 

Test Cases

Create a series of individualized scenarios based on an expected user workflow.

For each scenario, create a separate test case which will test the functionality of each function and subsystem.     

For every separate condition, you will want to create a separate test case.

The process involves stages:

  • Requirements – documentation.
  • Design of individual components.
  • Component design and testing.
  • Development – coding and design. 
  • Testing – set up environment for end-to-end setup, test design and execution.

End-to-End Testing vs Regression Testing

Regression testing is designed to determine whether changes to a system have broken other parts of a system. End-to-end testing will incorporate regression testing as part of its overall whole; a good end-to-end test may pick up some regression problems that may not have been identified in previous tests.

End-to-End Testing vs Integration Testing

Integration testing tests functionality between different parts of an application. End-to-end testing includes this but can be seen as a larger approach to the whole; it’s more about the integration of all of the parts of an application, rather than integrations between different modules. In an end-to-end test you should be able to identify whether the application works as designed from start to finish.

End-to-End Testing vs System Testing

End-to-end testing is in some ways similar to system testing, but in a more extensive capacity.

Let’s look at the basic aspects of system testing so that we can understand the differences.

System testing involves:

  • Full internal tests of integrations of a system.
  • Identifying the components interactions with each other inside the system.
  • Ensuring an accurate output based on the criteria provided through inputs.

System testing involves looking at how the application works as a system and a series of subsystems. It involves integration testing and ensures that each aspect of the system runs well with each other. However, unlike end-to-end testing, it does not look outside the system itself. End-to-end testing validates to make sure that all internal systems are not only running but also connecting with any and all external user interfaces.

System testing looks at features as subsystems to the whole. End-to-end testing looks at the user and graphical user interfaces as well as the back end to ensure that all functionality that is experienced by an end user works well within the back end.

This also includes interactions with other systems, which may be run by external organizations, and connected via API or some other method.

System testing is generally done after integration testing; end-to-end testing is typically performed after system testing has been completed.

System testing is typically a little easier to automate than end-to-end testing, and as a result, many people choose to do end-to-end testing manually. However, it is still possible to automate many features of end-to-end testing, although it usually requires some user input

Conclusion

End-to-end testing is an essential form of testing that should occur with every software development project. While it is somewhat expensive, it is something that should be performed at least once or twice prior to an application being promoted into production. While we can gain a lot of knowledge from individualized methods of testing such as unit and integration testing, we can not be sure that we have a fully developed application until we’ve run through it completely with a proper end-to-end test.

Luckily, if we have invoked a solid shift-left philosophy in our testing processes, end-to-end testing will be far more likely to be successful, and will enable us to release bug-free applications successfully with a minimal level of risk.