From Test Automation to Agentic Quality Engineering: MCP, Playwright and RAG

Дата публикации: 01-10-2026 11:56:41

From Test Automation to Agentic Quality Engineering: A Practical Journey with Playwright, MCP and RAG IntroductionSoftware testing is going through another major transformation.Traditional automation has already moved testing from manual execution to repeatable, CI/CD-driven automation. The next evolution is AI-driven and agentic testing, where AI is not limited to generating code but can also understand a test objective, interact with an application, generate automation, execute tests and analyze failures.This approach is increasingly referred to as Agentic Quality Engineering.I explored this approach through a practical proof of concept using Playwright, MCP agents and Retrieval-Augmented Generation (RAG). The objective was to understand how AI could be integrated into an existing automation framework and eventually executed through a CI/CD pipeline.The implementation covered the complete flow—from use-case understanding and application exploration to test generation, execution, failure analysis and headless CI/CD execution.What Is Agentic Testing?Traditional automation generally follows:Requirement
↓
Test Case
↓
Automation Script
↓
Execution
↓
Pass / Fail
↓
Script MaintenanceIn an agentic approach, the workflow becomes:Requirement
↓
AI Agent
↓
Retrieve Relevant Context
↓
Explore Application
↓
Generate Test Assets
↓
Execute
↓
Analyze Result
↓
Recover / Recommend / EscalateThe major difference is that the AI agent participates in the reasoning and decision-making process, rather than only producing automation code.Our Practical ImplementationThe most interesting part of this approach was implementing it against a real automation workflow.The POC used Playwright as the browser automation layer, an MCP-based AI agent for tool interaction, and RAG for project-specific knowledge.The overall flow was:Test Use Case
↓
Markdown File
↓
AI Agent
↓
Application / DOM Exploration
↓
Locator Identification
↓
Method Generation
↓
Test Case Generation
↓
Playwright Execution
↓
Failure Analysis / Recovery
↓
CI/CD Headless ExecutionThe goal was to move beyond a simple AI coding assistant and build an AI-assisted testing workflow that could work with the existing automation framework.1. Start With a Business Use CaseInstead of directly writing automation code, the functional requirement was maintained as a structured Markdown use case.For example:Booking Use Case
1. Login to the application.
2. Navigate to the booking module.
3. Search using a registration number.
4. Select the required service.
5. Enter mandatory information.
6. Submit the booking.
7. Validate booking confirmation.The use case becomes the starting point for the AI agent.This creates a separation between:What needs to be testedandHow it should be automated.2. Let the Agent Understand the ApplicationThe next step was connecting the AI agent to the application through Playwright MCP.The agent could interact with the application and inspect the DOM/page structure to identify the elements required for the requested workflow.Conceptually:Use Case
↓
Open Application
↓
Inspect DOM / Page
↓
Identify Elements
↓
Map Elements to ActionsFor a booking workflow, this could include identifying registration fields, service selection, customer information, submit actions and confirmation messages.This reduced the amount of manual effort required during initial automation development.3. Generate Locators, Methods and TestsOnce the relevant elements were identified, the next step was converting that understanding into automation assets.The workflow was:DOM Understanding
↓
Locator Identification
↓
Page Method Generation
↓
Test Case GenerationThe objective was not to generate isolated scripts.The generated automation was designed to fit into the existing Playwright framework structure and reusable automation approach.The agent could also derive positive, negative and exception scenarios from the use case.For example:Positive: valid registration, valid details and successful booking.Negative: invalid registration, missing mandatory information.Exception: duplicate booking, expired registration, session timeout or backend failure.4. RAG as the Knowledge LayerBrowser interaction alone is not enough for enterprise testing.A project also contains:Existing test casesBusiness rulesAPI specificationsDomain knowledgeTest data requirementsKnown defectsHistorical failuresFramework conventionsThis is where RAG becomes important.In the POC, RAG was explored as a way of providing the AI agent with relevant project-specific context. AI Agent
│
▼
RAG
│
┌────────────┼────────────┐
↓ ↓ ↓
Test Cases Documentation Domain Rules
│ │ │
└────────────┼────────────┘
▼
Context
↓
AI + PlaywrightA useful way to think about the architecture is:RAG → What does the agent need to know?
MCP → What can the agent do?
AI Agent → How should it reason and act?This effectively gives the automation framework a project knowledge or memory layer.5. Intelligent Failure AnalysisA major benefit of the agentic approach is the ability to investigate failures instead of simply reporting them.Traditional automation may report:FAILED
Element not foundAn agentic workflow can examine additional evidence:Test Failure
↓
DOM Inspection
↓
Locator Analysis
↓
Application State
↓
Network / API Information
↓
Historical Context
↓
Failure ClassificationThe issue could be:Locator changesTest-data problemsEnvironment instabilitySynchronization issuesBackend failuresWorkflow changesActual application defectsThis enables a broader form of self-healing, where the system attempts to understand the reason for failure before recommending or applying a correction.That distinction is important because automatically changing a locator could otherwise hide a genuine application defect.6. CI/CD Integration and Headless ExecutionA major part of the implementation was taking the automation beyond local execution and making it suitable for CI/CD execution in a headless environment.To support this, the framework was packaged with the required execution components, including:test.bat file for standardized test executionTest sequence files to control execution order and test flowDocker files to containerize the execution environmentRequired Playwright/browser dependenciesHeadless execution configurationCI/CD pipeline integrationThe overall execution flow became:Git Repository
↓
CI/CD Pipeline
↓
Docker Environment
↓
test.bat / Test Sequence
↓
Playwright
↓
Headless Browser
↓
Test Execution
↓
Results / LogsThis was an important step because the POC was no longer dependent on a developer's local machine.The same automation could be packaged into a controlled environment and executed through the pipeline using headless browsers.Docker also helped provide consistency between local and CI environments by packaging the required execution dependencies.7. Integrating AI With the Existing FrameworkThe objective was not to replace the existing automation framework.The framework continued to provide components such as:Page ObjectsTest casesUtilitiesTest dataReportingVersion controlCI/CDAI was introduced as an intelligence layer around these existing components.Existing Playwright Framework
│
├── Page Objects
├── Test Cases
├── Utilities
├── Test Data
└── CI/CD
│
▼
AI Layer
┌────┴────┐
▼ ▼
RAG MCP
│ │
Knowledge ToolsThis makes the approach more practical for enterprise adoption.8. From AI Automation to Intelligent CI/CDWith the framework integrated into CI/CD, the next evolution is making the pipeline more intelligent.A future workflow could be:Code Change
↓
AI Impact Analysis
↓
Identify Affected Areas
↓
Select Relevant Regression Tests
↓
Dockerized Execution
↓
Playwright Headless Execution
↓
AI Failure Analysis
↓
Evidence + Recommendation
↓
Quality GateInstead of simply running automation, the goal is to make the pipeline more context-aware and intelligent.The long-term vision is not just faster execution, but smarter decisions about what to test, what failed and why.9. Human-in-the-Loop and SecurityAgentic testing should not mean unrestricted automation.An agent can be allowed to:Read requirementsExplore test environmentsGenerate testsExecute approved testsCollect evidenceAnalyze failuresHigher-risk actions should remain controlled.Enterprise implementations therefore need:restricted permissions, approved environments, tool controls, auditability and human approval where required.10. Testing the AI TesterAnother important question is:How do we know the AI agent itself is producing good results?The agent should be evaluated using measures such as:Area Example Requirement understanding Correct interpretation Test generation Valid scenarios Coverage Critical scenario coverage Locator generation Reliability Failure analysis Root-cause accuracy Self-healing Recovery success Tool usage Correct tool selection Stability RepeatabilityThis creates two quality layers:Application Quality
+
Agent Quality
↓
Overall QualityKey Learnings From the Practical ImplementationThe project highlighted several important lessons.Context is critical.AI can generate technically valid automation but still misunderstand business behavior without project-specific context.RAG and MCP complement each other.RAG provides knowledge, MCP provides tools, and the AI agent provides reasoning and orchestration.Existing automation architecture remains important.AI should enhance established practices such as Page Objects, reusable utilities, CI/CD and version control.Self-healing needs diagnosis.A failure should be understood before automation is changed.CI/CD is essential for real-world adoption.Packaging the solution with test sequence files, test.bat, Docker and headless execution helped move the POC from a local experiment toward a repeatable engineering workflow.Human expertise remains important.AI can accelerate testing, but quality engineering still requires domain knowledge, risk assessment and engineering judgment.What This Means for QA EngineersThe role of the QA engineer is evolving from simply writing and maintaining scripts toward designing intelligent quality systems.The skillset is expanding from:Testing + Automationto:Testing + Automation + AI + RAG + MCP + CI/CD + Agent EvaluationThis does not mean replacing QA engineers.It means reducing repetitive work and allowing engineers to focus more on test strategy, product risk, complex failures and quality decisions.ConclusionThe combination of Playwright, MCP and RAG creates a new direction for software testing.In our practical implementation, the journey started with a Markdown-based use case, followed by AI-driven application exploration and DOM understanding, locator and method generation, test creation, Playwright execution, and failure analysis.RAG added project-specific knowledge, while MCP provided the agent with the ability to interact with testing tools.The solution was then taken beyond local execution by integrating it with CI/CD, creating test sequence files, a test.bat execution layer and Docker-based execution, and running the Playwright tests in headless mode.This demonstrates an important shift:AI can become an intelligence layer around an existing automation framework—not just a tool for generating test scripts.The future of QA may therefore not be about writing more automation.It may be about building intelligent quality systems that can understand requirements, interact with software, generate and execute tests, analyze failures and continuously assist engineers throughout the software delivery lifecycle.

Схожие новости

#	Наименование новости	Тональность	Информативность	Дата публикации
1	Building an AI Coding Agent: Automating Code Writing and Testing	0	4.6	25-07-2026
2	ИИ уже меняет не только разработку, но и тестирование, аналитику ...	0	7.15	28-09-2026
3	AI Transformations And Agile Transformations Rhyme	0	7.34	06-09-2026
4	Building a Customer Support AI Agent with RAG: A Step-by-Step Guide	0	10.29	22-07-2026
5	[Перевод] От тестирования релиза с высоким уровнем риска к новому ИИ-инструменту для QA	0	11.89	26-09-2026
6	Can Your Team Name the Work It Already Runs With AI?	0	6.08	14-09-2026
7	Does Your AI Know the Scrum Guide? Twelve Questions to Find Out	0	9.71	21-09-2026
8	Особенности агентской разработки ПО, обеспечивающей соответствие замыслу и требованиям	0	7.39	26-09-2026
9	Understanding Agentic AI: Deep Dive into Autonomous AI Agents	0	5.73	12-09-2026
10	Professional skepticism is a dev’s best skill	0	11.76	25-09-2026

Классификация: . Схожих патентов: 0. Схожих новостей: 10. Тональность: 0. Информативность: 5.98. Источник: community.hpe.com.