Speed is everything for developers. As teams push to build, test, and release software more quickly, the demand for high-quality code continues to grow; however, thorough quality assurance (QA) can sometimes slow down the process.
Traditional testing processes often create bottlenecks that stall productivity and delay feedback loops. Long test queues, slow build times, and overwhelming cloud resource costs are just a few of the hurdles standing between developers and efficient workflows.
These challenges are especially pronounced when scaling projects, adding new features, or managing increasingly complex test suites.
Many organizations are turning to AI-augmented QA to streamline the testing process and ease the manual work involved in getting code from commit to production. By automating key aspects of quality assurance, developers can ship releases faster without compromising quality.
While AI-powered tools like GitHub Copilot have already gained popularity, there's still a huge opportunity for specialized AI tools to elevate engineering workflows, from DevSecOps to QA, according to ICONIQ’s 2024 State of AI Report.
With the power of AI, CloudBees Smart Tests helps QA teams and developers use AI-augmented QA to identify which tests are most likely to fail and prioritize them. AI is like a smart filter, sorting through the noise of test failures and allowing only the most important issues to rise to the surface, making test management more efficient.
By running smaller, more focused subsets of tests, developers can reduce test execution times, optimize cloud resources, and enhance developer feedback loops.
Here’s how three CloudBees Smart Tests customers – GoCardless, a global automotive manufacturer, and a leading DevOps test data management provider – used AI-augmented QA to
Accelerate release cycles
Cut costs
Boost developer productivity
1. GoCardless: Cutting Cloud Costs and Reducing Test Times By 50%
GoCardless is a global leader in direct bank payments, processing over $35 billion annually across more than 30 countries. With a large volume of transactions and a growing number of tests in their pipeline, the GoCardless team faced a common challenge in the world of development: slow CI times that were limiting productivity. Their Developer Enablement team required an AI-augmented QA solution that would help reduce testing times while simultaneously reducing cloud resource costs.
The Challenge
GoCardless had a unit test suite with around 60,000 test cases. Running this suite on every push function across multiple branches created significant strain on cloud resources, and the team could not afford to scale up further. They needed to reduce testing time by at least 40% to meet their Service-Level Objectives (SLO).
The Solution
By implementing CloudBees Smart Tests, GoCardless was able to intelligently subset tests, focusing on those most likely to fail and running them first. This not only improved developer productivity by drastically reducing test times but also led to significant cost savings in cloud resources.
The Results
With an AI-augmented QA solution, GoCardless cut machine hours by 50% per test run, saving 8,500 hours in their first full month of implementation. Testing times dropped from over 300 minutes to just 48 minutes per run, dramatically improving the CI pipeline's speed and test efficiency.
“Thanks to CloudBees Smart Tests, we’ve managed to increase the speed of our pipeline for our engineers, reducing waiting time and costs and increasing productivity and satisfaction all around. The product is thorough and easy to integrate, which allowed us to get it on production in just one month.”
Bastian Zamorano
Product Manager for Developer Enablement at GoCardless
2. Global Automotive Manufacturer: Optimizing Hardware Resources and Speeding Up Feedback Loops with AI-Augmented QA
The DevSecOps team at a global automotive manufacturer faced a unique challenge. Testing against physical hardware, such as head units (HUs), was slowing down their builds and increasing queue times. As more tests were added, hardware constraints grew, and resource costs began to spiral out of control. With the pressure to provide fast feedback to developers, the team turned to Smart Tests to address their hardware usage issues while maintaining test quality.
The Challenge
The global auto manufacturer’s test suite was designed to run on real physical hardware, but the limited number of testing stations meant developers had to wait long periods to see if their code changes had passed. As the team continued to scale, the issue worsened, resulting in increased queue times and higher resource usage. AI-Augmented QA played a crucial role in addressing these scaling issues by helping the team prioritize and streamline testing processes.
The Solution
CloudBees Smart Tests’ PTS enabled the team to focus on the tests most likely to fail, thereby reducing the time and hardware resources required for each run. By utilizing its split subset feature, they were able to evenly distribute tests across available hardware resources, optimizing test execution.
The Results
The result was a significant reduction in hardware usage, accompanied by more efficient parallel testing. Along with more efficient parallel testing. By prioritizing tests based on their likelihood of failure, the team saved significant time and reduced the costs associated with running tests on physical hardware. This approach seamlessly integrated the power of AI and ML into their workflow with minimal effort.
3. Leading DevOps Test Data Platform Cuts Costs and Saves 40,000 Hours with AI-Augmented QA
A leading provider of DevOps test data management solutions set out to accelerate product release cycles while maintaining the highest quality standards. Their platform streamlines secure test data delivery for development, testing, and analytics, helping teams move faster without compromising compliance. To improve efficiency, the team focused on reducing the duration of long regression tests and shortening the feedback loops.
The Challenge
The company’s broad integration with cloud services, databases, and enterprise applications created a heavy testing burden, requiring hundreds of test cases to ensure compatibility. This led to multi-day test suite execution times, slowing development cycles, driving up cloud costs, and forcing developers to spend hours manually analyzing test failures. The team needed a way to streamline release testing without sacrificing quality, ideally while also reducing cloud spend.
The Solution
To address the challenges, the team implemented CloudBees Smart Tests, which feature predictive test selection and AI triage capabilities, thereby reducing the time spent on bug detection and improving test efficiency. CloudBees Smart Tests identifies likely failing tests based on code changes and historical data, while AI-driven recommendations help prioritize tests based on business impact. This approach allowed the team to focus on relevant tests earlier in the development cycle, providing faster feedback and optimizing resource usage.
The Results
The company reduced pre-commit testing time by 66%, decreasing the average run time from 6 hours to 2 hours, and cut regression testing time by 80%. As a result, nightly suites now complete in just a couple of hours, rather than days. This translates to thousands of test execution hours saved annually, resulting in significant cloud cost reductions. Developers now commit changes with greater confidence and accelerate release cycles without sacrificing quality.
How AI-Powered QA Solves Common Challenges to Boost Productivity Across Industries
Although these customers come from different industries, they faced similar challenges: slow testing cycles, resource inefficiencies, and the need for faster developer feedback. CloudBees Smart Tests helped each team address these obstacles directly, improving both speed and efficiency across their development workflows.
By implementing AI-powered Predictive Test Selection, all three teams were able to:
Reduce Testing Times: Whether it’s cutting machine hours or reducing the number of tests executed, each team saw faster feedback cycles.
Optimize Hardware/Cloud Costs: By running only the necessary tests, each team saved valuable resources and reduced infrastructure costs.
Improve Developer Productivity: With reduced waiting times, developers could focus on coding instead of waiting for tests to complete, resulting in quicker deployments and improved job satisfaction.
Boost Customer Satisfaction and Revenue: Faster, reliable releases enhance customer experience, drive loyalty, and optimize revenue by ensuring high-quality products reach the market more efficiently.
As software complexity increases and release cycles accelerate, AI-powered testing becomes key for delivering reliable code.
CloudBees Smart Tests combines predictive test selection, AI-driven triage, and actionable insights into one flexible platform, helping DevOps teams eliminate bottlenecks, optimize test suites, and accelerate releases. With its AI co-pilot for test suite optimization, CloudBees Smart Tests enables faster test runs, smarter failure diagnosis, and more comprehensive test insights.
See how CloudBees Smart Tests can drive real ROI and transform your DevOps process. Book a demo today to experience AI-augmented QA in action.