Quality assurance (QA) testing has long been the backbone of software development, but traditional QA approaches have not kept pace with modern development cycles and complex UIs. Most organizations still rely on a hybrid approach combining manual testing with script-based automation frameworks like Selenium, Cypress, and Playwright – yet teams spend a significant amount of their time maintaining existing test automation rather than creating new tests. The problem is that traditional automation is weak. Test scripts break with UI changes, require specialized programming knowledge, and often provide incomplete coverage across browsers and devices. While many organizations are actively exploring AI-powered testing workflows, current approaches are inadequate.
In this post, we’ll explore how agentic QA automation addresses these challenges and walk through a practical example using the Amazon Bedrock AgentCore browser. Amazon Nova Act To automate testing for a sample retail app.
Benefits of Agentic QA Testing
Agent AI moves QA testing from rule-based automation to intelligent, autonomous testing systems. Unlike traditional automation, which follows preprogrammed scripts, agentic AI can observe, learn, adapt, and make decisions in real time. Key benefits include autonomous test generation through UI observability and dynamic adaptation with changes of UI elements – reducing maintenance overhead that consumes QA teams’ time. These systems mimic human interaction patterns, ensuring that testing occurs from a real user perspective rather than along rigid, scripted pathways.
AgentCore browser for large-scale agentive QA testing
To realize the potential of agentic AI testing at enterprise scale, organizations need robust infrastructure that can support intelligent, autonomous testing agents. AgentCore Browser, a built-in tool of Amazon Bedrock AgentCore, addresses this need by providing a secure, cloud-based browser environment designed specifically for AI agents to interact with websites and applications.
The AgentCore browser includes essential enterprise security features such as session isolation, built-in observability through live view, AWS CloudTrail logging, and session replay capabilities. Operating in a containerized ephemeral environment, each browser instance can be shut down after use, providing clean test conditions and optimal resource management. For large-scale QA operations, the AgentCore browser can run multiple browser sessions simultaneously, so organizations can simultaneously parallelize testing across different scenarios, environments, and user journeys.
Agent QA with Amazon Nova Act SDK
The infrastructure capabilities of the AgentCore browser become truly powerful when combined with an Agent SDK like Amazon Nova Act. Amazon Nova Act is an AWS service that helps developers build, deploy, and manage fleets of trusted AI agents to automate production UI workflows. With this SDK, developers can break down complex testing workflows into small, reliable commands while maintaining the ability to call APIs and perform direct browser manipulation as needed. This approach provides seamless integration of Python code throughout the testing process. Developers can interleave tests, breakpoints, and assertions directly within the agentive workflow, providing unprecedented control and debugging capabilities. This combination of the AgentCore browser cloud infrastructure with the Amazon Nova Act Agent SDK creates a comprehensive testing ecosystem that transforms the way organizations approach quality assurance.
Practical Implementation: Retail Application Testing
To illustrate this change in behavior, let’s consider developing a new application for a retail company. We created a simulated retail web application to demonstrate the agentic QA process, assuming that the application is hosted on AWS infrastructure within a private enterprise network during the development and testing phases.
To streamline the test creation process, we use KiroAn AI-powered coding assistant to automatically generate UI test cases by analyzing our application code base. Kiro examines the application architecture, reviews existing test patterns, and creates comprehensive test cases following the JSON Schema format required by the Amazon Nova Act. By understanding the application’s characteristics – including navigation, search, filtering, and form submission – Kiro generates detailed test steps with actions and expected results that are instantly executable through the AgentCore browser. This AI-supported approach dramatically accelerates test creation while providing comprehensive coverage. The following demonstration shows Kiro generating 15 ready-to-use test cases for our QA testing demo application.
After the test cases are prepared, they are placed in test data directory Where? pytest Automatically finds and executes them. Each JSON test file becomes an independent test that Pytest can run in parallel. uses outline pytest-xdist Distributing tests across multiple worker processes, automatically utilizing available system resources for optimal performance.
During execution, each test gets its own separate AgentCore browser session through the Amazon Nova Act SDK. The Amazon Nova Act Agent reads the test steps from the JSON file and executes them – performing actions like clicking a button or filling out a form, then verifies that the expected results occur. This data-driven approach means teams can create comprehensive test suites by simply writing JSON files, without the need to write Python code for each test scenario. The parallel execution architecture significantly reduces testing time. Tests that would normally run sequentially can now execute simultaneously in multiple browser sessions, with pytest managing the distribution and aggregation of results. An HTML report is automatically generated using the pytest-html and pytest-html-nova-act plugins, providing test results, screenshots, and execution logs for full visibility into the testing process.
One of the most powerful capabilities of the AgentCore browser is the ability to run multiple browser sessions simultaneously, enabling truly parallel test execution at large scale. When PyTest distributes tests across worker processes, each test spawns its own isolated browser session in the cloud. This means that your entire test suite can execute simultaneously instead of waiting for each test to complete sequentially.
The AWS Management Console provides full visibility into these parallel sessions. As shown in the following video, you can view active browser sessions running concurrently, monitor their status, and track resource usage in real time. This observation is important for understanding test execution patterns and optimizing your testing infrastructure.
Beyond simply monitoring session state, the AgentCore browser provides live view and session replay features to see what Amazon Nova Act is doing during and after test execution. For an active browser session, you can open Live View and see the agent interacting with your application in real time – clicking buttons, filling out forms, navigating pages, and validating results. When you enable session replay, you can view recorded events by replaying the recorded session. This allows you to validate test results even after test execution is complete. These capabilities are invaluable for debugging test failures, understanding agent behavior, and gaining confidence in your automated testing process.
For complete deployment instructions and access to sample retail application code, AWS CloudFormation templates, and the PyTest testing framework, check out GitHub repositoryThe repository contains the components necessary to deploy and test the application in your own AWS environment,
conclusion
In this post, we explained how AgentCore can help parallelize Agentic QA testing for browser web applications. An agent like Amazon Nova Act can perform automated agentic QA testing with high reliability.
About the authors
Kosti Vasilkakis is a Principal PM at AWS in the Agentic AI team, where he led the design and development of several Bedrock AgentCore services, including Runtime, Browser, Code Interpreter, and Identity. He previously worked on Amazon SageMaker from its early days, launching AI/ML capabilities that are now used by thousands of companies worldwide. Early in his career, Costi was a data scientist. Outside of work, he builds personal productivity automation, plays tennis, and enjoys life with his wife and children.
Ved Raman Senior Solutions Architect for Generative AI at Amazon Nova and Agentic AI at AWS. She helps customers design and build agentic AI solutions using the Amazon Nova Model and Bedrock AgentCore. He previously worked with customers building ML solutions using Amazon SageMaker and also worked as a Serverless Solutions Architect at AWS.
Omkar Nyalpelli is a Cloud Infrastructure Architect in AWS Professional Services with deep expertise in AWS Landing Zones and DevOps methodologies. His current focus is on the intersection of cloud infrastructure and AI technologies – specifically leveraging generative AI and agentic AI systems to create autonomous, self-managed cloud environments. Through its work with enterprise customers, Omkar explores innovative approaches to reducing operational overhead while increasing system reliability. Apart from his technical activities, he likes to play cricket, baseball and do creative photography. He holds an MS in Networking and Telecommunications from Southern Methodist University.
Ryan Canty is a Solutions Architect at Amazon AGI Labs with over 10 years of software engineering experience, specializing in designing and scaling enterprise software systems across multiple technology stacks. He works with customers to leverage Amazon Nova Act, an AWS service to build and deploy highly reliable AI agents that automate UI-based workflows at scale, bridging the gap between cutting-edge AI capabilities and practical business applications.
