End-to-end (e2e further in the text) testing is immensely important since there are issues that can only be discovered while testing the system as a whole, testing it from one end to another.
That also adds a greater level of complexity to e2e tests, making them longer, slower to execute, and with more points of potential failure, compared to lower-level tests with a smaller scope, especially when compared to tiny and blazing fast unit tests. Since e2e tests bring a lot of value, we don’t want to neglect them. We also want to make our lives easier when doing e2e testing, which we can do with the right tools. Apart from automating our e2e tests, we can speed up our e2e tests if when they are not automated. Let us dig deeper into what exactly we can do to make e2e testing easier and faster to execute.
How to Make e2e Testing Easier and Faster to Execute
Analyze the value e2e tests will bring
While having the previously mentioned in mind (e2e are expensive and slower than other tests) we should take heed to prioritize which part of our application under test needs to be covered with e2e tests, and which parts can be covered at lower test levels. When balancing the costs and the potential return on the investment, we should have the customer in mind – doing extensive e2e testing of the FAQ section will not be as valuable as covering the whole checkout, or the entire registration journey with e2e tests. Having the most important customer journeys monitored and covered with e2e tests will add a significant level of security – an alarm that will go off when something in this journey fails. This is why these kinds of tests need to be run on a regular basis, such as a part of a nightly build, for example. Also, when deciding which journeys to automate, you should talk to the stakeholder and your team members, especially with the subject matter experts (or the domain experts) and with technical team members, such as other automation engineers and the developers, to reach informed decisions regarding the e2e coverage.
Avoid duplication – test whatever is possible at lower levels
You don’t really need two e2e tests checking parts of the same flow – this should be avoided, if it’s at all possible. It is much cheaper to check all the business logic at the unit level and to cover more complex business rules with API automated checks – which are way faster than checking the same functionality using the UI. Keep a low number of e2e UI tests, check only what gives a lot of value to the customer.
Utilize cloud infrastructure to reduce costs and increase speed
The pros of using cloud infrastructure, over on-prem, are the ease of configuration and the availability of resources ( almost infinite). Additionally, the prices of cloud services have been dropping, as there are more competing players in the cloud arena. For this, you can utilize a cloud-based test orchestration platform like LambdaTest’s HyperExecute. You can also use cloud containers to manage and organize your tests better and make executing times faster as well! There is also an on-prem enterprise option for this feature, as some organizations cannot use the cloud due to legal restrictions.
Use tools like LambdaTest to cover multiple browsers
The need to cover multiple browsers or mobile platforms can make your testing efforts very time-consuming. Just imagine, you have a fairly complex web app that needs to be tested on all the modern browsers to make sure all the important features work. Such sort of testing should be made faster with the use of proper tools. These tools can be particularly helpful if you need to support multiple versions of different browsers, and operating systems and to check against multiple different screen resolution sizes for this u need to perform online browser testing.
It can be very beneficial to be able to test on real physical hardware, without having to buy a whole bunch of dedicated testing devices – you would practically just rent the usage of these for a certain period of time. Another important feature to look for is if the cross-browser testing tools have native integrations with the developer tools, in the supported browsers, so you can debug and test more thoroughly. Other uses for such tools are the ability to test on the local host, if you have an app in local development, or to test geolocations with a different IP address. Also, make use of browser extensions for cross-browser testing tools.
Taking screenshots automatically, in different browsers you test on, is yet another handy feature that comes with these kinds of tools. LambdaTest comes with a browser of its own – LT Browser which comes pre-packaged with advanced features like creating custom resolution sizes of your own for testing the responsiveness of your web apps.
Leverage existing tools for visual testing
Visual regression testing can catch a lot of issues that are either very hard to detect using tools like Selenium, or not possible with regular UI functional testing tools. Plenty of customer-impacting issues can be prevented if caught by doing visual testing. This can be done in the cloud as well with LambdaTest’s Smart UI tool.
Issues such as wrong element colors, various size-checks, padding validation, layout issues (such as responsiveness), text-related issues, and element positioning, can be good candidates to check using Pixel-by-Pixel comparison, which can be run across well over 3000 desktop and mobile devices. Automated visual checks should be run in parallel, to speed up your release cycles and to make the test execution a lot faster while cutting down the feedback loops considerably.
Visual regression tests are also great candidates to be included in the CI/CD pipeline. Another useful practice would be to leverage extensions and integrate them into the other tools you’re using.
Include e2e tests into your build pipeline early
As soon as you have at least one working e2e test you should include it into your CI/CD pipeline, or make sure to include the test execution as part of your build process if you have scheduled releases.
Integrate reporting tools with your e2e tests
Given the costly nature of the e2e tests, having good and detailed reporting in place is a great way to get additional value from your e2e tests. There are many libraries that allow you to access their reporting capabilities (usually via an API) from your automation framework. Examples would be ReportNG, Allure reports, Report Portal, etc. These libraries can generate a lot of visually appealing reports, like pie charts and status bars for your testing, so you can visualize how many of your tests are passing, failing, being skipped, etc.
For failing tests you can usually drill into the details. But, for the non-testing folks, this info is usually not very interesting. Do remember, it can be very valuable for the testers, maintaining (so they can debug the failing checks and fix them or report bugs) the automation solution, and for the developers as well – if you have them involved in the automation process, which is almost always a good idea to do!
Most modern test management tools also have extensive reporting capabilities for your test execution.
Another great practice is to have the reporting for your automation connected to your most common communication channels, such as having a Slack notification, a Teams message, or similar. It’ll help you be on top of things like— when tests in the build pipeline fail, what is the status of the test execution, etc. In this sense, your e2e tests (especially if you have them configured to be, for example, executed repeatedly – like as a part of a nightly build) can act as an alarm – warning you in time when an important feature stops working or misbehaves. It’s always better to find an issue internally and fix it than to have the customer report it to your support department because it could mean losing that customer, and many others who had the same issue. It is a known fact that due to many competitors in the market today, customers have a lot of options to choose from, meaning that if they have an issue with using your app, a lot of them won’t even tell you about it, they will just move to the competitor. So, always have the customer’s satisfaction as your main priority!
Common approaches to e2e testing
Here are some of the most important things to consider when doing e2e testing.
- Make sure that your app is deployed properly and is testable This will speed up testing and prevent some issues from happening, like problems with the test environment. For these things, you can use cloud services to make your life a bit easier.
- Make sure that other testing activities are completed— all the features have been already tested in isolation, integration points have been tested and all unit tests are passing. This is done to make sure that the system is working well and all the unique pieces are connected together and interacting as they should. Do e2e testing after all the other stages of testing have already been completed. That means you should wait until all the individual components have been tested.
- Keep the end-users experience in mind. Talk to the customer, if possible, and connect with the product owners and technical support people to learn more about how the real users use your app. With such knowledge in mind, your e2e tests will be more valuable and successful.
- When it is time for e2e testing, make sure you have a process in place that is accepted by everyone. Avoid significant deviations from the testing process, as these can lead to slower releases, and can cost additional time and money.
Conclusion
In order to test end to end, it is crucial to follow the agreed-upon set of practices – to make sure that testing is doable and cost-effective.
What is the goal behind an e2e test? Try to think like the user, imagine that you are using the app for the very first time, and ask some questions. Is the app easy to navigate and are all the important user options easily available and accessible? Is the journey evident to the user – the features are pretty self-explanatory to use. Can the user get from point a to point b – like completing the registration or the checkout process? Have the acceptance side of testing in mind as well and create your tests according to that.
Try to cover with e2e tests the parts of the app where a lot of issues are expected, and the most important parts for the customer. Try to use the risk-based approach here to determine what to cover. There are decision tables you can use to visually determine how risky a certain feature is.
Do not test for exceptions when doing e2e testing, this can be covered at lower test levels, to save resources and test faster.
Maintain your e2e test set regularly – since e2e tests cover big parts of the app, they are usually longer and more complex than other tests. If one e2e test can be logically divided into two or more tests, then you should do that, since with every new big step in the test (like an entire screen or page in the app) the complexity of the test will increase significantly, and it will be harder to debug it as it will have multiple points of failure – which is one of the biggest pain points of having e2e tests.
Make sure your tests are well organized and use consistent naming – it’s best to make it obvious just by reading the title of the test to understand what it does.
Have pre-defined test setup and teardown – e2e tests will require test data and will usually write to the database as well., Do make sure that you clean up as you might risk having duplicate data, which can result in bugs that could have been avoided. Eg like primary key violations if you try to, say, register a test user with the same email twice.
Plan your e2e tests well, and as early as possible. The e2e tests which you have automated should also be checked manually by a real human, as there are a lot of issues the automation will ignore – it will only check what you tell it.
Also, use your e2e tests to speed up your exploratory testing – if you have an e2e test that fills out a lengthy form you should use it to save time.
Depending on the nature of your project, you might want to treat end-to-end testing as an essential part of the build pipeline. Prepare e2e tests as early as possible, invest time in them, and make sure you have the resources needed to have the ability to e2e test even before the feature goes live in production. Good luck!
Published on Java Code Geeks with permission by Mirza Sisictester, partner at our JCG program. See the original article here: Make e2e Testing Easier With the Right Tools Opinions expressed by Java Code Geeks contributors are their own. |
