Autonomous test-healing for Drupal teams.

When a Playwright test breaks overnight, a Claude agent diagnoses the failure, classifies it as test-issue or website-issue, and — if it's a fix the agent is allowed to make — opens a pull request before anyone gets to their desk. The workflow is in our public repo.

Pilot autonomous healing on your suite
Book a testing review

Four things that happen when a test fails.

The agent doesn't “fix your tests.” It reads the failure, decides what kind of failure it is, and takes the action its operating mode allows. Sometimes that's a PR. Sometimes it's an issue with a diagnosis. Sometimes it's nothing.

A contrib module renames a class.

A module update ships a CSS-class rename. Your ATK smoke test fails overnight on a .field-name-field-foo selector that's now .field-foo. The agent reads the stack trace, checks the rendered DOM, classifies as test-fix, opens a PR updating the selector, labels it test-fix. Green by morning.

A Views block label changes.

Someone renames “Latest articles” to “Recent articles” in the Views UI and commits the config export. Your Playwright assertion on getByRole('heading', { name: 'Latest articles' }) fails. The agent reads the config diff, sees the coordinated rename, updates the assertion, opens a PR. Not a site bug — a config update the test hadn't caught up to.

The site is actually broken.

A recent commit to a custom module nulls out a form field on edit. The test correctly fails. The agent diagnoses it as a website-side issue, files an issue with the stack trace + hypothesis + suggested fix, and tags the last commit's author. No PR. No auto-merge. A human decides.

It's just flake.

One nightly run, a test fails on a transient network blip against Pantheon. The agent retries, the test passes, the agent classifies as transient, closes the investigation. No PR. No issue. No noise. One line in the run log.

The guardrails are in the repo.

Every night, Playwright tests run against this site in CI. When one fails, a GitHub Action invokes a Claude agent with the failure artifacts and the constraints it's allowed to operate under. Here's the top of heal-tests-claude.yml — the file that decides what the AI is permitted to touch:

# Only fix test code in CI. Site-code changes require a human.
operating_mode: ci
permissions:
  modify_test_code: true       # selectors, assertions, waits
  modify_site_code: false      # PHP, Twig, JS, CSS - never in CI
  open_pull_request: true
  auto_merge: false            # a human reviews every PR
classification:
  test_issue:   [selector_outdated, assertion_outdated, timing, url_change]
  website_issue:[element_missing, server_error, broken_form, empty_content]

Locally, the same agent runs with operating_mode: local — the engineer at the terminal can let it fix site code too, because changes can be verified against a running DDEV instance before commit. The mode is explicit. The permission flags are explicit. Nothing is implicit.

Read the full workflow on GitHub

The toolkit we run AI on top of.

The agent isn't a testing framework. It's a layer that sits on top of the frameworks your team already runs — or should be running.

Playwright.

The primary framework we write and maintain tests in. Every healing vignette above plays out against a Playwright suite. If you're already on Playwright, the agent wires in on day one.

Cypress.

Legacy-supported. If your suite is Cypress, the agent reads failure output and opens PRs the same way — fewer success cases than Playwright today, but the classification logic is framework-agnostic.

ATK — Automated Testing Kit.

Our open-source Drupal-specific test toolkit (Playwright-first, Cypress-supported). The selector patterns in the vignettes above come from ATK. It's on the site's /open-source-projects page and in the public repo.

Testor.

The Drupal module that wires test-run reporting into the site's admin UI — because “the tests are green” is more useful when a content editor can see it without opening GitHub Actions.

This site is the first dogfood.

Every night, Playwright tests run against the very site you're reading. When a test breaks, the workflow above fires. For the full narrative — architecture choices, rasterization strategy, mobile-spacing reconciliation, and every other decision that shaped this site — read how we built this site.

-  await expect(page.getByRole('heading', { name: 'Latest articles' })).toBeVisible();
+  await expect(page.getByRole('heading', { name: 'Recent articles' })).toBeVisible();

One of the commits the agent opened on this repo. A config-rename ripple — Views UI changed the block label; the Playwright assertion had to catch up. The agent read the failure output, read the views.view.*.yml diff, decided it was a coordinated rename, and shipped the PR. Closed the loop in about eleven minutes, start to merge.

Over the last 90 days, [N] tests have been auto-healed on this site without human intervention.

Read the full story

Ready to let your tests heal themselves?

Start a healing pilot