Run mutation testing on every pull request #5

Merged

cyplo merged 2 commits from mutation-testing into main 2026-11-08 22:24:40 +00:00

Conversation 0 Commits 2 Files changed 9 +114 -9

cyplo commented 2026-11-08 22:16:25 +00:00

Owner

• Copy link
• Copy Markdown

Mutation testing has only been run by hand so far, which is how mutants.out/ ended up committed in #1. This makes it routine.

cargo-mutants is in the dev shell and configured so a plain cargo mutants runs exactly as CI does. CI mutates the lines each pull request changes and fails on any mutant the tests miss. On main, and on any branch through the new manual trigger, it mutates all the code and uploads the report. The full run reports without gating for now, because main has untested code today; the next PR closes those gaps and turns gating on.

The first manual run also exposed a CI flaw: the runner's shared cargo target directory let one run test another run's build. A manual run reused the pull request run's binary of the same commit and failed reading the other run's deleted checkout. Each cargo job now builds in its own checkout.

Infrastructure only: no Rust code changes.

Mutation testing has only been run by hand so far, which is how `mutants.out/` ended up committed in #1. This makes it routine. cargo-mutants is in the dev shell and configured so a plain `cargo mutants` runs exactly as CI does. CI mutates the lines each pull request changes and fails on any mutant the tests miss. On main, and on any branch through the new manual trigger, it mutates all the code and uploads the report. The full run reports without gating for now, because main has untested code today; the next PR closes those gaps and turns gating on. The first manual run also exposed a CI flaw: the runner's shared cargo target directory let one run test another run's build. A manual run reused the pull request run's binary of the same commit and failed reading the other run's deleted checkout. Each cargo job now builds in its own checkout. Infrastructure only: no Rust code changes.

cyplo added 1 commit 2026-11-08 22:16:25 +00:00

Set up mutation testing with cargo-mutants ... fb37a9c182

cargo-mutants is in the dev shell and configured in .cargo/mutants.toml
to run under nextest with a debug-info-free profile, so a plain cargo
mutants locally matches CI. CI mutates the lines each pull request
changes and fails on any mutant the tests miss; on main, and on any
branch through workflow_dispatch, it mutates all the code and uploads
the report, without gating until the existing gaps are closed.
CLAUDE.md tells agents to mutate what they touched before finishing and
to run full mutation on CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ai-agent added 1 commit 2026-11-08 22:18:56 +00:00

Build each CI job in its own checkout ... e761c74395

The runner shares one CARGO_TARGET_DIR between checkouts, and cargo
treats another checkout's build there as up to date. A manual run on the
mutation-testing branch reused the pull request run's build of the same
commit, so its fixture tests read the other run's deleted checkout and
17 of 44 failed. With different commits it would have reported on code
it never checked out. Every cargo step now unsets CARGO_TARGET_DIR, as
the mutation job already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cyplo merged commit e761c74395 into main 2026-11-08 22:24:40 +00:00

cyplo deleted branch mutation-testing 2026-11-08 22:24:40 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

cyplo/deescalate!5

No description provided.