Why is test automation so important for ecommerce websites?

Jun 16 2023

Software projects are not exactly like cars, where you can easily separate areas. If you change a tire on your car, you will not feel the urge to start the engine to see if it still starts up. In software projects however, something like this may happen.

Why do we need to automate tests?

Things in software are very much intertwined, and a small change somewhere may cause something completely different to break. In systems, with such a high interdependence, extensive testing is needed. Every time you change something, the whole project must be tested; a process called regression testing.

As you can imagine, regression testing takes a lot of time and resources. And not only that. If you have been working on a software project as a tester, filing out the same form for the hundredth time will eventually get boring and make you sloppy. You will start to not see this and not see that. This is where test automation comes into play.

In this article we explain how test automation works, look at the challenges involved and explain how ecommerce merchants can do it well.

USE AND FUNCTION

What is test automation and how does it work?

Test automation is the creation of programs that simulate user interaction with an application. This makes them perfect for regression testing. You can just run these programs and they will do all the tests and, in the end, tell you where they had problems with their predefined flows.

I remember working on a project when developers forgot to format the dates in a table. So instead of the date, the table showed the number of seconds that had passed since 1970. None of us saw it. After a while your eye sees numbers and are satisfied. But the automated tests did see the error, and they even checked that each date was correct.

When setting up test automation, you will create tests that cover almost all of the functions visitors can use on your website. Then, ideally, then you will set up a test for each one of these functions. But of course, most of the time you just let the automated tests perform as an end to end flow, where they for example select a product, put it in your shopping cart, enter personal data, select a delivery and payment method, and check if the order has gone through.

Challenges

The challenges of automated testing

However, this comes at a cost. Automated tests must be created step by step. While a manual tester, who does everything on their own, can just click themselves through an application, an automated test has to be carefully constructed.

As a manual tester or user, you may just click away an upcoming popup window, but an automated test has to be prepared for that. It must know when this window will pop up and can be closed, otherwise it will get stuck.

Thus, it takes some time to create these tests. And since they can usually only be created after a rather stable version of the application is available, test automators often live in this no-man’s-land between the creation and the release of new features.

Once the development has been done, projects managers want to deliver quickly since everything is ready and oftentimes there is no time left for test automation. This is another reason why test automation is often neglected.

Why ecommerce merchants are opting for test automation.

But nowadays more and more companies are relying on test automation. And in my opinion, as an ecommerce merchant you would not want to work with software companies that don’t use test automation. Neglecting test automation is the first step to neglecting quality.

Of course, you can always try to click through everything. It is certainly enough to see if something big is broken. But it’s a lot like having a car mechanic take a quick look under your engine hood to see if there is an engine at all and if it looks alright. You might not want to drive with that car. And you certainly would not want this to have happened when you take a plane. Of course, web applications are much less dangerous. But think about it: a customer who crashes a couple of times might not try his luck again and go looking for other service providers.

Why is test automation often getting lost in maintenance?

Test automators tend to get lost in maintenance. Automated tests are not a dream tool, that you can run and then it tells you exactly how many errors you have in your application and where they are. Yes, it will find errors, but it will have a lot more fails that are not due to errors.

Automated tests are very fragile creatures. They must interact with all the elements the user interacts with, but if any one of these elements is altered, they will not find them anymore. How can this be? Let us look at a simple example. If we have a button to continue after having selected a payment method, it might have the following html code associated:

Since there might be many buttons on the page, the automated tests will be programed to look for a button with the class “continue”. But if now this class changes, we have this:

The automated test will not find the button and report that there was a button missing. The result will be a failed test. This of course will be a false result from a functional point of view since the button is there. The test then needs maintenance to adapt it to the altered button.

Oftentimes the increased need of maintenance is due to a lack of synchronization between automators and developers. Automators may not be informed about the areas of the application that have changed. And developers of course have no idea on which elements test automators rely on to connect their tests to the website. In particular, when there is a lot of change in design, automated tests are bound to suffer from this, because they often come along with a lot of such minor changes.

Automated tests love long boring forms that they can fill in so rapidly that your eyes cannot follow. If you do not alter the fields, the tests will run forever. Furthermore, one additional problem for automated tests are jungles of popup windows, because they cannot interact with elements that are behind these windows.

As a user we can just click them away when they popup. But an automated test has to wait for the window to popup, even if it is not yet there, then close it and only then it can start interacting with the elements on the page. You can imagine, that this is especially difficult when these windows are not mandatory and may or may not open up.

The solution to make it all work

But there is of course a solution for these maintenance problems, and it comes with a magical surplus. If developers knew on which elements automators relied, everything would be fine. And this can be done at a low cost. Developers can just add such elements. Let’s take as an example our button from above. A special attribute for automation could be added to it:

Now the tests could refer to this special attribute and they would be resistant to changes in other attributes or even in the Element itself, which for a design reason might be changed from a button to a link. If, on a webpage, every element that can be interacted with also has one of these attributes the test automation can refer to, tests would become very stable. But this of course is something extra to do for developers and it hardly will be done if automation is not seen as an important part of the whole development process.

There is even one aspect where it becomes obvious that automation is not taken into account in the development process. When automators learn their trade, they often learn to rely on id attributes of web elements which look like this:

These id attributes are meant to be stable and unique on a webpage. And many automation tools come along with built-in references to these ids. For instance in Selenium, a standard tool for test automation, you can just tell the computer to get you an element with its id. If you want to use the class, like in the example above, it’s a tiny bit more complicated. However, a lot of development frameworks generate these ids automatically and they do not generate them in a consistent way at all. So, they became utterly worthless for automation. Both worlds are not very much aligned concerning this issue.

A magical fit between development and automation

Now what about this magical surplus I mentioned above? Imagine we have a webpage, where each functional element has a fixed attribute test automation could refer to, like we have seen in the example above:

Test automation as such is quite a simple thing. It just translates functional test step descriptions into automated test step descriptions. Click on this, then fill in this field and click on that. The hardest thing about telling a computer program to do this is explaining which elements are meant to go with this and that, which button or field we want it to click on or fill in. But now imagine, these elements had a fixed type of reference like in the example above. Then the part of figuring out the reference would become something trivial, something that could be done by a computer program itself. And that means we could automate the referencing. And thus, we could automate test automation all together. We would just have to write the testcases in a rather formal language and the automated testcases could be generated in an automatic way from there. We would have automated automation.

One of my former bosses who looked and thought a lot like Leonardo da Vinci used to say that software production is still in a state very close to neolithic stone tool production, where each stone is produced individually and is different from all others. He always painted a picture, where in a future and more industrialized form of software production, human beings would write no code at all, but everything would be generated from models. And user interfaces as well as automated tests could then be generated from the same models and come with a perfect fit. But we all know that neolithic stone tools used to be high tech for millions of years, only to be replaced by metalwork a couple of thousand years ago. So, we might still be far from this ideal of full automation. But it is not a bad thing to have something like this in mind when thinking of possible improvements.