Testing continues to be a significant obstacle in organizations striving for faster software delivery. Despite increased awareness and investments in testing tools, the real challenge lies in the difficulty of testing, especially as organizations scale automation. Recent data reveals that only 24.37% of organizations pass 90% of desktop tests, and 25.45% pass 90% of mobile tests, underscoring the struggle with test quality.
The Challenge of Test Quality
Aggressive investments in testing tools and technologies have kept the difficulty of testing manageable, especially as organizations push for automation at scale. A recent study highlights that only 24.37% of organizations pass 90% of their desktop tests, and merely 25.45% pass 90% of mobile tests. This underscores the challenge organizations face in maintaining test quality, hindering their ability to accelerate application delivery and shorten release cycles.
Doing It Right: Four Key Steps
Organizations must focus on improving test quality and speed to overcome these challenges. One major factor contributing to test failures is the time it takes to run them. According to research, tests completed in two minutes or less are twice as likely to pass as those lasting longer than two minutes. Let’s explore four crucial steps to reduce test run times effectively.
1. Script Atomic Tests
Creating a better and faster automation experience starts with scripting atomic tests. Instead of bundling multiple functionalities into one test, focus on individual functionalities. For instance, in an online retail scenario, script separate tests for user login, viewing gift card balance, adding items to the cart, checkout, and transaction processing. Running these atomic tests in parallel significantly reduces overall run time.
2. Run Tests in Parallel
Running atomic tests sequentially can negate the benefits of their speed. Therefore, executing tests in parallel reduces overall test run times. For example, a suite with 100 two-minute atomic tests would take more than three hours sequentially but just two minutes in parallel. Quantity doesn’t necessarily impact run time; even with a higher test count, parallel execution proves more time-efficient.
3. Reduce Selenium Commands
Minimizing the number of Selenium commands in a test script is crucial for maintaining atomic testing principles. Excessive commands increase execution time and introduce potential points of failure. Streamlining commands shrinks the overall run time, creating a more stable testing environment.
4. Use Explicit Waits
Optimizing test run time involves choosing explicit waits over implicit waits. While implicit waits introduce delays between script steps, explicit waits allow the script to proceed immediately after completing the preceding step. Despite the added complexity, relying solely on explicit waits positively impacts test run time.
Putting It Together: A 66% Improvement
Applying these tactics to an unoptimized test suite with 311 commands reduced the total to 127.
The optimized suite executed against the same device in 430 seconds, marking a 66% improvement from the initial 1,265 seconds (over 21 minutes). Achieving shorter, more stable tests may seem daunting, but organizations can reach this goal and deliver better software faster by prioritizing key tactics and best practices.