services
A holistic approach that accelerates your current vision while also making you future-proof. We help you face the future fluidically.
Digital Engineering

Value-driven and technology savvy. We future-proof your business.

Intelligent Enterprise
Helping you master your critical business applications, empowering your business to thrive.
Experience and Design
Harness the power of design to drive a whole new level of success.
Events and Webinars
Our Event Series
Featured Event
26 - 27 Nov
Booth F99 | The European Retail Exhibition, Paris
Our Latest Talk
By Kanchan Ray, Dr. Sudipta Seal
video icon 60 mins
About
nagarro
Discover more about us,
an outstanding digital
solutions developer and a
great place to work in.
Investor
relations
Financial information,
governance, reports,
announcements, and
investor events.
News &
press releases
Catch up to what we are
doing, and what people
are talking about.
Caring &
sustainability
We care for our world.
Learn about our
initiatives.

Fluidic
Enterprise

Beyond agility, the convergence of technology and human ingenuity.
talk to us
Welcome to digital product engineering
Thanks for your interest. How can we help?
 
 
Author
Vivek Shringi
Vivek Shringi
connect

Research proves that our brains are much more receptive to watching visuals as compared to reading text. Ever since modern web development has adopted this concept, UI designs are no longer restricted to static web pages or stationary layouts. Every day, animations and graphics are increasingly being used to greatly impact users. Many different types of media files, like high-definition images and videos, are constantly being integrated into web design. As a consequence, these changes have added another layer of new challenges in testing such web applications.

Why you need visual automated testing

Visual testing is a testing type where actual web applications are tested with expected user designs. You can think of it as just like your typical spot-the-difference game, comparing two almost-identical pictures. Only that, in this case, if the tester needs to play this game many times a day, it would become very time-consuming.

Imagine if a web application is also delivering many animations like scrolling behaviors, auto sliders, and events which are also connected with some audio. Here are some examples:

Scrolling animations

  • A car is approaching you when you scroll the page from top to down.
  • A text detaches from old text and attaches with new text.

Auto slider

Automatic slider with fixed time which shows the new picture.

Events synchronized with audio

As the graphics play out, they should be well synchronized with the corresponding audio. For example, having the right sounds to be played when a user wins playing a game on a web application. It is critical that the winning animation works exactly how its producer wants and should be well synchronized with its audio.

Sometimes, one complete feature delivers not just animation but also the accompanying functionalities. An appropriate test strategy is necessary to deliver excellent product quality in optimized effort.

Without good test planning, it can be a huge challenge to deliver quality software. Testers should use the right mix of testing types and testing levels.

This is where visual testing can play a crucial role in reducing the testing effort as it verifies not only the visual part but also the software’s usability and functionality. It can be executed either manually or automatically, all thanks to modern browser automation tools, which can easily verify the UI layouts on websites.

Automation tools support visual testing, comparing the current visual with expected visual piece-by-piece and highlighting the difference. The test fails if the visuals do not match. A threshold value is possible to provide if the test needs to be passed or failed in case of difference.

Visual testing v/s functional testing

The role of visual testing starts when functional testing cannot verify enough and gets bound by its limitations. Functional testing can only verify one property, but this verification cannot guarantee that UI layout has no defects. Let us look at some use cases where automation can help and provide visual feedback in no time. As we all know, the position of different UI components is as important as their functionality. UI and functionalities fulfill the complete user experience to deliver services with confidence.

Here are some instances where visual testing can be more effective than functional testing:
UI layouts with different screen sizes

In this era of fast-paced modern web development where products are frequently released in the market, it can be quite time-consuming to test UI layouts on every supported screen size manually. This is where you can say hello to automation.

Automation can test visuals on different screen sizes in no time, that too with higher efficiency.  

Nagarro's Ginger AI platform

 
UI layouts on different browsers

It is equally critical to test UI layouts on supported browsers like Chrome, Safari, and Firefox. Testing the visuals on every browser can be a time-taking and tiresome monotonous task. We can use automation to help us here as well, to not only be more productive but also avoid human errors.

Nagarro's Ginger AI platformAs you can see in the above example, there are some minor differences across different browsers (primarily, a different font is used). Visual testing also ensures that the UI layout is unique, even if the user uses the same application in different browsers.

 

Drag and Drop

At times, simply verifying CSS properties might not be enough and the QA could verify if the UI also changes as intended by different user actions like a commonly used action such as drag-and-drop.

Here is an example: A user can view different slides by dragging the image from right to left just like turning a page in the book. Visual automation test takes the snapshot after performing a drag-drop event and comparing the screen with the expected screenshot.

drag and Drop in Nagarro's website

 
Animation

Modern web development also delivers amazing animations and sometimes, image comparisons are not enough to see the impact of animations. When a user scrolls from top to bottom, some websites deliver amazing animations. These visual effects cannot be verified only by comparing screenshots. To verify the complete effect, human intelligence is required. The automation tool can record the video, and the tester can manually verify if the desired effect is working as expected.

 

The above video shows how a slider changes the slides after a certain amount of time. While functional tests can verify the behavior, they are not as impactful as visual testing.

 

The above example shows the scrolling behavior from the start right till the end. An automation tool not only verifies the screens but also records the video of the complete scrolling journey. What’s more, it can also be tested manually.

 

Cookie position

While it is easy to check if a cookie pop-up is present in the browser, it is equally important to verify its relative position. A visual automation test can easily detect if the cookie’s position has changed.

cookiePosition02
Hover

Today, as web applications use multiple hover effects, it is important to verify their impact. While functional tests can verify the CSS properties, this would not be enough. Only visual testing can identify the differences easily.

Why do we say so? Here are some examples:

Nagarro website's Hover on the button

In this example above, when you hover on the button, an arrow animation is played.

Nagarro website's image zoom featureThis example is now a common practice, where an image gets zoomed on hovering over it.

It is also quite ubiquitous to expose sub menu items on highlighting the main menu item. While this scenario can also be checked functionally by verifying the element visibility, visual testing can test much more effectively.

Nagarro website's Main MenuIn the above example, on hovering over the main menu item, all submenu items are opened.

Floating menu

A floating menu is another common feature in modern web applications. It gives the user a possibility to access the main menu, irrespective of its page location. The user can access floating menu on the top without scrolling again to the top.

Visual testing can make sure that the main menu remains visible with all the required elements even after scrolling to the middle of the page or moving down or upwards.

Chatbots

Chatbots are a great way to connect to users when they visit a website. Chatbots should be activated once the reader has stayed on the website’s page for at least a couple of seconds. For the development team, it is equally important to verify if the bot opened correctly.

Visual automated testing can check if the chatbots have been triggered correctly.

Visual automated testing can check if the chatbots have been triggered correctly.

Pop-ups

Verifying a pop-up and its position can also be critical to a website’s functionality. The pop-up presence can be tested by verifying programmatically, but verifying with visual testing may be a better option as it can also test its relative position and other visual aspects.

And guess what, this is by no means an exhaustive list. These are just a few examples that show where visual testing has an upper hand over functional or any other type of testing.

Strategy for visual automation testing

It’s important to have the right test strategy in place before implementing visual automated testing. Here are some best practices that should always be followed:

  • Only a few scenarios should be visually tested and not all test cases should be automated. Since even a single line of code can introduce a lot of maintenance, it may increase the automation effort.
  • It is always good to start with manual testing. After that, once the UI is finalized, visual automated tests should be introduced for regression testing.
  • Components with frequent UI changes should not be targeted for visual automated testing.
  • Verifying the entire screen in one iteration can be risky in visual testing. This is because the chances of test failure are higher, and it may create a lot of noise if visual regression testing is part of the CI pipeline.
  • Complex animations in web applications should be tested manually. You can refer to various tools for videos of the required user actions. The testers can watch the video to evaluate if the animation is working as intended.
  • Do not mix functional test and visual test in one test case. The ground rule of any automation is to perform only one verification in one test.
  • Set enough tolerance so that slight visual changes can be ignored by the tool (e.g., 5%, more than 200 pixels).  
  • The visual test should be small, and one image is enough to verify in one test.
  • Frequent changing elements may be masked to avoid unwanted errors.

Tool selection criteria

There are plenty of visual automated testing tools available in the market. Some of them are open source and some of them are enterprise tools. Before deciding on any tool in the market. The team must discuss the following questions:

  • Why do we need a visual testing tool?
  • Do team members have enough skills to write automation scripts?
  • How much time and effort can be saved by introducing visual automation testing?
  • Are there any use cases that can be selected for visual testing?
  • What features of an automation tool can be used to test our web application?

What’s cool about the tool?

Here are some key features to evaluate the tool:

Core features

  • Tool should compare full screen or part of the screens and identify the difference.
  • Tool should have threshold value to fail or pass the difference (it can be in pixels or percentage of the changes)
  • Tool should be capable of ignoring the background.
  • Tool should be able to mask a sub element of target element to ignore the changes.
  • Tool should identify a change of an element instead of full screen.

Others features

  • Reporting
  • Integration with CI/CD tools
  • Integration with communication channel like Slack or Teams
  • Usability – how easy it is to integrate the tool and write the test scripts.

Conclusion

Visual automated testing can not only help finding bugs early in software development but also save significant regression testing effort. It empowers the Frontend developers to give automatic feedback in no time if visual testing is integrated with continuous integration.

Nagarro has already delivered many successful projects where introducing visual testing played a crucial role in saving regression testing effort and finding defects early. Thanks to Nagarro's expertise in visual testing, we not only help development teams find the best visual testing tool but also ensure we implement the right use cases.