This is Part 2 of a 2-part series. Read Part 1 here.
Stanford’s 6x performance boost came from wrapping a single model in a verification harness. But one brilliant testing agent does not provide an enterprise release gate. You need to run thousands of them concurrently across chaotic environments.
The agents do the testing. The platform orchestrates the execution, stores the evidence, and connects the verdicts to the business.
End to end test execution gets complicated when your volume doubles. Running agents at enterprise scale demands isolated cloud execution separate from the continuous integration runners building the application. It requires massive parallelism to fit a full regression suite into a standard cycle without bottlenecking the whole engineering organization.
The platform needs constantly updated browser coverage, secure tunnels to reach private staging environments, and the ability to manage pristine test data so parallel runs do not corrupt each other.
Testing agents capture exactly what the application did at every single step. The testing platform has to retain what the application did, stored securely for as long as an auditor requires.
You need affordable storage for thousands of runs a week, and teams need search functionality that can pull up a specific run from eight months ago in seconds, providing tamper-proof evidence so no one can quietly edit history.
Agents write tests faster than humans can review them. Without a system of record, your suite becomes an unmaintainable mess. Teams need traceability linking requirements to runs, two-way sync with test management systems, and versioning to roll back bad changes. Deduplication ensures ten different agents do not independently write ten variations of the exact same login test.
Distinguishing actual bugs from flaky tests across thousands of runs requires storing historical data per test, per step, and per environment. The platform must roll up root causes so that a crashed database shows up as one alert, not four hundred isolated test failures.
A release gate only works if the verdict reaches the right people. The platform must deliver a definitive pass, fail, or lack of signal directly to CI/CD, issue trackers, and test management systems.
Failed runs must route back to the developer's coding agent, packaged with the DOM trace, screenshots, network logs, and failure classification. Engineers need to access these verdicts via tools like Atlassian Rovo without logging into a separate dashboard.
Testing agents need their own guardrails. Engineering leaders require unified health views showing pass rates, flakiness, and time-since-last-change across all environments.
The system needs an immutable ledger of every decision an agent made, including self-healing actions. Strict role-based access control dictates who can edit tests. Agents need rules too; a coding agent should be able to read a test result without having the ability to edit the test suite.
If you are building this capability, prioritize signal over scale. Five tests you trust completely are more valuable than five hundred tests you ignore.
Separation of duties, immutable evidence, and infrastructure maintenance are complex systems problems. They will compound as models get faster at writing code.
We have spent the last three years building mabl’s testing harness and the platform that runs it. If your team decides to build this in-house with a coding agent and Playwright, understand what you are signing up for: a massive, secondary software system that requires its own dedicated infrastructure, maintenance roadmap, and on-call rotation right alongside your coding harness.
This is Part 2 of a 2-part series. Read part 1 here.