1. Introduction
2. Benefits of Visual Testing
3. How Visual Testing in Appium Elevates Our Testing
4. EPractical Example: Ensuring Our App’s UI with Appium Visual Testing
5. Conclusion
In the world of software testing, ensuring that an application works correctly doesn’t always mean it displays as expected. Automated tests allow us to verify if our application behaves as intended, but what happens when elements are not in the right location, lack the expected color, or overlap unexpectedly? This is where Visual Testing becomes an essential ally.
Benefits of Visual Testing
Visual Testing ensures that your application’s user interface looks and behaves consistently across all platforms and devices, providing an optimal experience for the end user. Here are some key benefits:
- Comprehensive Visual Validation: While automated tests can confirm that a button is “clickable,” Visual Testing verifies if that button is visible, positioned correctly, has the proper color, and is not hidden by other elements. This ensures the application looks professional and the interface is user-friendly.
- Detection of Layout Issues: Design changes, screen sizes, and content updates can cause visual problems that go unnoticed in automated tests. Visual Testing can detect these issues and ensure UI elements are not overlapping or misplaced.
- Consistency in User Experience: Visual Testing ensures the visual appearance of your application is consistent across all devices and platforms. This is crucial for maintaining a uniform brand experience and improving user satisfaction.
- Perfect Complement to Automated Testing: In an automated test, you might verify that a text appears when an action is completed, but without visual tests, you couldn’t ensure the text appears in the correct place, with the right size, and without overlapping. Visual Testing complements these tests by ensuring functionality is backed by proper visual presentation.
How Visual Testing in Appium Elevates Our Testing
Appium, combined with its image plugin, takes Visual Testing to another level. Using image comparison, Appium not only verifies the presence of elements but also validates their appearance and position on the screen. Here are some ways the image plugin enhances Visual Testing:
- Detection of Unwanted Visual Changes: If you make a change to the application, you can compare it with a reference image to ensure there are no unwanted alterations in the UI. For example, if the “Book” button changes color or shifts position, the plugin will detect it, allowing you to fix the issue before it reaches the end user.
- Testing Across Devices and Resolutions: The way an application appears can vary depending on the device and resolution. Appium’s image plugin allows you to compare how the application looks in different configurations, ensuring it maintains a consistent appearance.
Practical Example: Ensuring Our App’s UI with Appium Visual Testing
For our practical example, we’ll use Appium 2.0 with the images-plugin and Pytest as the test framework. The reports generated by Allure Report will help us visualize our test results effectively.
To perform visual tests, it’s essential to have a reference image representing the expected state of the user interface on the device. This reference image serves as a “model” to compare the application’s current appearance during testing.
It’s crucial for the reference image to always have the same dimensions as the application’s current capture to ensure an accurate comparison. Since screen dimensions vary depending on the device or resolution, specific reference images are required for each device you want to test.
The Process Is Simple:
- Capture the Reference Image: First, you need to take a screenshot of the application when it’s in the desired state. Ensure that all elements are correctly positioned, with the expected colors, sizes, and styles. This image will be used as a comparison point in future tests.
- Store the Reference Image: Save this image in a specific directory within your test project. It’s important to maintain an organized record of these images, as you may need multiple reference images for different screens, resolutions, or application states.
- Compare with the Current Capture: During the execution of visual tests, Appium will take a new screenshot of the application’s current state and compare it with the reference image. Appium’s image plugin will evaluate whether there are significant differences between the two images and alert you if inconsistencies are found.
Enough Talking—Show Me How It’s Done! 😅
In our case, we want to ensure that the Welcome screen displays correctly at all times. As mentioned, we first need a reference image.
Test
Reference Capture.
import base64from typing import Tuple
from typing import Optional
from PIL import Image
def image_comparison(self, image_template_path: str, actual_state_screen_path: str) -> Tuple[float, Optional[str]]:
# Open the expected image template in binary mode and read its contents
with open(image_template_path, ‘rb’) as img:
image_template = img.read()
# Open the actual state screenshot in binary mode and read its contents
with open(actual_state_screen_path, ‘rb’) as img:
actual_state_screen = img.read()
# Encode the expected image template to base64 format for comparison
base64_image_template = base64.b64encode(image_template).decode(‘utf-8’)
# Encode the actual state screenshot to base64 format for comparison
base64_actual_state_screen = base64.b64encode(actual_state_screen).decode(‘utf-8’)
# Use the driver to compare the two images for similarity and get the results
get_images_result = self.driver.get_images_similarity(
base64_image1=base64_image_template,
base64_image2=base64_actual_state_screen,
visualize=True # Visualize the comparison result
)
# Return the similarity score and the visualization of the comparison
return get_images_result[‘score’], get_images_result[‘visualization’]
We have defined a method called image_comparison that visually compares a reference image with a screenshot of the application’s current state and returns the comparison result.
The method returns two values: the similarity score (score) and the visual representation of the differences (visualization). The score indicates how similar the two images are (1.0 means they are identical), and visualizationcontains the image highlighting any differences, if present.
The actual test will perform the verification as follows:
import allure
@allure.step(“User verifica que UI contiene los elementos esperados”)
def user_verify_ui(self):
# Define the directory where screenshots will be saved
screenshots_directory = f”../screenshots/”
# Generate a unique name for the screenshot using the device name and current timestamp
unique_screenshot_name = f”{get_custom_device_name(self.driver)}_{GenerateData.CURRENTIME_NANO_SECS.value}”
# Create the full path for the screenshot
screenshot_full_path = f”{screenshots_directory}{unique_screenshot_name}”
# Capture the current state of the UI and save it to the defined path
self.screenshot(screenshot_full_path)
# Retrieve the expected screenshot for verification from the designated path
expected_screenshot_path = retrieve_screenshot_saved_for_ui_verification(
self,
path=f”../pages/images/{get_custom_device_name(self.driver)}/splash_page”,
screen_name=”splash”
)
# Retrieve the actual screenshot taken for UI testing
actual_screenshot_path = retrieve_path_screenshot_taken_for_ui_testing(self, screenshot_full_path)
# Compare the expected screenshot with the actual screenshot and get the comparison results
image_comparison_result = self.image_comparison(expected_screenshot_path, actual_screenshot_path)
# Save the visual comparison result as a PNG file
self.save_base64_to_png(image_comparison_result[1], “../screenshots/splash_page”)
with open(“../screenshots/splash_page”, “rb”) as image_file:
allure.attach(image_file.read(), name=”UI Verification Screenshot”,
attachment_type=allure.attachment_type.PNG)
# Assert that the similarity score is above the threshold of 0.97
assert image_comparison_result[
0] >= 0.995, f”UI verification failed. Similarity score: {image_comparison_result[0]}”
The user_verify_ui function is designed to verify that an application’s user interface (UI) contains the expected elements.
For this case, the score verification ensures that the similarity score is at least 0.995 (Yes 😎, we’re very meticulous), indicating that the images are sufficiently similar.
Positive Result
In the image, the visual differences between the two screenshots are limited to the device’s menu bar. These changes typically include elements such as battery status, time, signal strength, and other notifications, which are dynamic and variable aspects of the device’s interface.
Report shows no difference between the expected and current state.
Negative Result
“Alright, but show me that it detects an unexpected change 😤”
In this case, the customer’s name has disappeared. While it’s not an error that prevents booking your vacation, we all prefer a flawless experience.
In this scenario, the missing text is identified:
Report Showing Differences Between Expected and Current States
Conclusion
Visual Testing offers significant benefits by ensuring that the user interface looks and functions as expected. It enables the detection of visual errors, unintended changes, and ensures consistency in the user experience across different devices and browsers. This reduces the risk of issues that could harm the brand image and enhances the quality of the final product, saving time and costs by catching problems early.
In summary, Visual Testing is an investment that ensures flawless interfaces and a superior user experience.
