6 min read
AI agents in the QA loop
This post shows what it looks like when AI agents are integrated into the development cycle for test generation, self-healing tests and PR fixes. In most 'AI for QA' tools, the AI is bolted on.
I’m tired of toy “AI for QA” demos that amount to a chatbot writing a brittle Cypress test from a prompt.
Across six apps for a global pet care company, I’ve been shipping AI as a participant in the quality loop. The agents work as collaborators alongside the human team, watch the pipeline and take on work.
What “AI in the QA loop” looks like
There are four integrations that change how a team operates.
1. Test plan generation from UI exploration
An agent opens the app, navigates it, builds a site map, identifies user flows, and produces a test plan before a human has touched it. Coverage gaps become visible before anyone writes a line of Playwright.
2. Test authoring from work items
When a ticket moves to “Ready for QA,” an agent reads the acceptance criteria, reviews the related code changes, and proposes a Playwright test that validates the new behavior. A human reviews and edits the test before merging it.
3. Self-healing on selector drift
Selectors break all the time. When CI goes red because of one, an agent analyzes the DOM change, identifies the new locator, and patches the test. It opens a PR with the fix and a diff showing what moved, and a human approves it.
4. Inline vulnerability + performance flagging
Agents crawl PRs for OWASP top-10 red flags, performance regressions, and obvious security mistakes. They leave inline comments, always escalate what they find and sometimes fix it themselves.
What this does to the team
This changed two things on the team, and both surprised me.
First, manual QA effort drops sharply. Humans are still needed for judgment, exploratory testing, accessibility and UX, but the mechanical layer (test authoring, maintenance, regression triage) goes from 70% of the team’s time to 20%.
Second, the remaining QA work gets harder in a good way. Agents handle the click(button).expect(modal) loops, and the people on the team design test strategy, critique agent output and catch the weird stuff AI misses.
What it does not do
It does not replace senior QA judgment or humans in user testing and accessibility work, and it isn’t good at catching novel, spec-ambiguous defects. Anything sold as “fully autonomous QA” will let regressions through sooner or later.
What you need to make this work
- A strong deterministic test framework underneath. Mine is Playwright with strict TypeScript. You cannot build reliable AI-driven QA on a brittle foundation.
- Proper CI with observable runs. Agents need to read CI output to learn.
- A human review gate on every agent-authored change.
- Version-pinned model choices. Model upgrades will regress your prompts, so test them like any other dependency.
Where this goes
The top-of-funnel test authoring question is mostly solved within 18 months. The interesting work then moves up the stack, to AI agents that reason about test strategy, prioritize what to cover and propose architectural changes to make code more testable.
By 2027, I expect writing Playwright tests by hand to be the slow option.
If you’re leading a QA org and want to pressure-test where AI fits, book a call. I’ve done this end-to-end and can shortcut a lot of the learning.