Reviewing Automation PRs with Claude and Cursor

AI assistants can produce a convincing Playwright test in seconds. That is exactly why automation pull requests need a stricter review model.

The most expensive defects in test code are not syntax errors. They are false positives: tests that pass while proving less than reviewers think. An assistant can select the first matching element, wait for an arbitrary timeout, mock away the behavior under test, or copy a brittle selector pattern across twenty files. The suite stays green and confidence quietly falls.

I use Claude or Cursor as a second reviewer, never the approval authority. The model finds patterns and asks questions; deterministic checks and accountable humans decide whether the test belongs in the suite.

Claude Code supports repository instructions such as CLAUDE.md and scoped rules, while Cursor provides persistent rules and review features. Cursor’s Agent Review and Bugbot can read repository guidance, including BUGBOT.md. Consult the current official documentation for Claude Code project instructions, Cursor rules, and Cursor Agent Review.

Give the reviewer a narrow job

Do not ask, “Is this PR good?” Give the model the diff, relevant test policy, changed production behavior, and a prioritized review contract.

Here is the prompt I use as a starting point:

You are reviewing a test-automation pull request as a skeptical senior SDET.

Goal: identify defects that could create false confidence, flakiness, unnecessary maintenance, security exposure, or excessive CI cost.

Review only the supplied diff and referenced interfaces. Do not edit files. Do not assume a test is correct because it passes.

For every finding provide: 1. Severity: blocker, high, medium, or low 2. File and changed line 3. Failure mechanism 4. Evidence from the diff 5. Smallest safe remediation 6. A validation that would prove the remediation

Check specifically for: - assertions that do not prove the stated business outcome - selectors that ignore role, label, text, or test-id contracts - arbitrary sleeps, inflated timeouts, retries, first/nth masking ambiguity - shared mutable state, order dependence, and non-unique test data - mocks that bypass the integration the test claims to cover - missing negative, permission, error, idempotency, or recovery cases - leaked secrets or personal data in fixtures, traces, logs, or snapshots - duplicated helpers and violations of repository test boundaries - expensive setup, unnecessary browser work, and unsafe parallelism

If evidence is insufficient, say so. Do not invent repository behavior. Return findings first, ordered by severity. End with residual risks and recommended deterministic checks. Do not approve or merge the PR.

The phrasing matters. “Failure mechanism” forces the review beyond style. “Evidence from the diff” reduces unsupported speculation. “Smallest safe remediation” discourages gratuitous rewrites. The final prohibition keeps the tool in an advisory role.

The human review checklist

I run the following checklist before merging any automation code substantially written or changed by an assistant.

1. Test intent and risk

• —Does the test name state an observable behavior?
• —Is the behavior important enough for this test level?
• —Which product risk does the test mitigate?
• —Would a unit, component, API, or contract test prove it faster?
• —Are the important negative and recovery paths represented somewhere?

If nobody can state the risk, the test is probably accidental coverage.

2. Assertion strength

• —Does the final assertion prove the user or business outcome?
• —Could the test pass if the backend operation failed?
• —Is it asserting transient UI state rather than a durable outcome?
• —Are multiple outcomes collapsed into one vague visibility check?
• —Were assertions deleted, softened, or moved inside helpers?

Prefer Playwright’s web-first assertions, which retry until their condition is met, over manual polling or immediate state reads. See Playwright assertions.

3. Locator quality

• —Does the locator reflect how a user or accessibility tree identifies the element?
• —Is a test ID an intentional contract rather than a DOM accident?
• —Could localization or duplicate content make it ambiguous?
• —Do .first(), .last(), or .nth() hide a strictness problem?
• —Is CSS tied to layout or styling classes?

The assistant should not “repair” an ambiguous locator by selecting the first result. Ambiguity may be the defect.

4. Synchronization and flake resistance

• —Are there hard waits or arbitrary sleeps?
• —Were timeouts increased instead of diagnosing the condition?
• —Does the test wait for the outcome that matters?
• —Can animations, background requests, or eventual consistency change timing?
• —Does it rely on retries to pass?

Playwright performs actionability checks before actions and supports auto-retrying assertions. Use those mechanisms deliberately; do not stack custom waits around them without a specific reason. Review the official auto-waiting behavior.

5. State and isolation

• —Does each test create unique, owned data?
• —Can it run alone, in parallel, and in any order?
• —Does cleanup hide the evidence needed after failure?
• —Are accounts, feature flags, clocks, queues, and caches controlled?
• —Could one test mutate another test’s tenant or environment?

Run the changed test repeatedly, then run it with randomized order or maximum parallelism where practical. A single green execution proves very little about isolation.

6. Abstraction and ownership

• —Is the helper domain-owned and appropriately scoped?
• —Does it hide assertions or conditional paths?
• —Is a new abstraction justified by repeated stable behavior?
• —Does the PR duplicate an existing component, fixture, or builder?
• —Will failure output still identify the business step?

AI tends to over-abstract copied patterns. Three explicit lines in one test are often cheaper than a generic helper with five options.

7. Security, privacy, and tool authority

• —Does test data contain real personal information or secrets?
• —Can traces, screenshots, videos, or model prompts expose tokens?
• —Did the assistant receive broader repository or CI permissions than necessary?
• —Are generated commands and patches reviewed before execution?
• —Are model output and external content treated as untrusted input?

Use read-only review tokens whenever possible. A reviewer does not need merge, release, or secret-management authority.

Back the review with deterministic checks

An AI review complements, but never replaces, compilation, lint rules, dependency-boundary checks, selector policies, secret scanning, changed-test execution, and repeated-run or quarantine analysis. Store the review prompt and repository rules in version control. Pin the workflow version, log which model produced each finding, and test prompt changes against historical PRs.

Measure precision rather than comment volume: accepted high-severity findings, false-positive rate, repeated issue categories, review latency, and defects that escaped despite AI review. Disable or revise rules that create noise; reviewers learn to ignore a bot faster than teams expect.

Claude and Cursor can make automation reviews faster and more consistent. Their best contribution is not approval—it is disciplined skepticism at scale. Keep the model read-only, require evidence, and make every suggested change earn its place through deterministic execution and human judgment.