Concepts

How test scopes run

What a repo's targeted test command is handed, how scopes that share a concurrency group queue on a machine, and how test-selection state carries from one worktree to the next.

A repo's test_scopes say which commands verify a change. This page is what Kraft does when it runs them. The accepted forms and refusals of each key are in repos.yaml.

Targeted commands

A scope's targeted_command runs only the tests a change affects.

  • What it receives. The command gets KRAFT_TEST_BASE, the item's whole-branch base commit, and KRAFT_CHANGED_FILES, the path of a file listing the changed paths, one per line. Environment variables describes both. The list is the diff this round's scopes were selected from: the whole branch in round 0 and after a failed round, and only what changed since after a clean one. Both are set for this command alone, and a sandbox gets the list read-only.
  • An empty list is an answer. A round that changed nothing since a clean one still runs it, with an empty list. With no list at all, Kraft runs command instead: the item has no base commit, or git couldn't diff it. A workspace member has no list either. The event says why (full_reason).
  • What your command must do. For a change it can't map to tests, it runs more tests, never fewer: a tool that selects tests from imports has to run everything for a YAML file it knows nothing about. Kraft only hands it the change.
  • The full suite is the chain's. A chain whose merge request runs CI leaves the full suite to it. A chain without CI adds a node with suite: full: see Run the full suite as its own node.

kraft repo connect --verify runs it too, against HEAD^1 and the last commit's paths, and skips it on a repo with one commit.

Scopes that share a group

Scopes that name the same group run one at a time on the local worker, and on each remote machine. While one runs, another scope of that group waits for it to finish, whichever work item or repository it belongs to, because the limit is the machine's CPUs, not the number of items. A scope without a group never waits for one and never makes another wait.

  • When to set it. On a scope whose command fills the machine by itself: a test runner that starts a worker per CPU, a browser suite, a full build, a command that starts servers and waits for them. Leave lint, type checks and single test files out. The sign that a scope needs one is a failure that appears only when two items verify at once and passes on a retry with no change.
  • One name for everything heavy. Two scopes that would overload the machine together share a group, across repositories too. Give two scopes different groups only when the machine can run one of each side by side.
  • What the board says. It names the work item that holds the group: Waiting for a machine lists the wording. The wait doesn't count against the step's time cap. A restart keeps the group held on the local worker by a session still running, as long as repos.yaml still reads and still names that session's command.
  • Where it applies. The local worker and every remote machine, each holding its own slot per group. Two machines of one runner group can each run one scope of the group at once; two scopes of it never run together on one machine. A scope on a group of machines goes to a member where its group is free, and when none is, the board says the group is held on that many machines.
  • Held only on the local worker. A scope held back by the local worker alone shows the wait for a machine in the group.
  • One holder. The board names the work item and the machine when a single one holds it.
  • After a restart. A remote machine's group stays held by a step still running there. Steps started before the upgrade don't hold it.

State that carries between worktrees

The carry key lists the files a test-selection tool keeps its state in.

  • Seeding. Kraft copies the whole set from one source: the repo's newest saved copy when it holds any listed file, else the repo's checkout, never a mix. The checkout is only read, and the copy is discarded when a file changes while it is copied, so the tool starts cold. With neither, it starts cold.
  • Saving. After a verification round of suite: targeted in which every scope passed on this machine, and a listed file changed, Kraft saves the worktree's set as the newest copy. A sandboxed item, a remote machine, an in-place item and a workspace member never save.
  • Older trees don't replace newer ones. Each saved copy records the commit its branch forks from, so "newest" means newest by that commit, not by time. An item cut from an older commit than the saved copy's doesn't replace it, because its record lacks every change since, and the next item to start from it would re-run every test those changes touch. A copy with no recorded commit doesn't stop a save, and neither do two commits git can't order.
  • What it is for. Only state that is safe when stale: the tool checks file content, runs more when unsure, and updates on every run. A tool that trusts its cache by timestamp alone doesn't belong here.
  • Throwing one away. run/carried/<repo>-<hash>/current points at the newest copy, under $KRAFT_RUN_DIR. Delete that repo's folder and the next item seeds from the checkout, or starts cold. kraft view storage counts the copies under "other", outside the quota.
  • Sandboxed repos. A repo whose items always run sandboxed never saves, so it stays as warm as its checkout's own record.

kraft repo connect --verify seeds the same set and lists where each file came from.

Copyright © 2026