Evals · self-governing multi-agent fleet

We didn't build an autonomous fleet. We measured how autonomous it actually is.

Six machines, each driven by its own Claude, negotiate every change to a shared system through a consensus ledger and escalate risky actions to a human. Below are the numbers from 15 days of production logs — not a benchmark, not a demo. Every figure traces to a log file; where a metric isn't measurable yet, we say so instead of guessing.

Window 2026-06-29 → 07-14 Unit proposal, not event Events 744 deduped Proposals 71 Source consensus ledger (JSONL)

61%

of proposals reached a decision with no human in the loop

43 / 71 · ledger lifecycle

24%

involved the human at all (Anton approved)

17 / 71 · HUMAN_APPROVED

sync conflicts in the ledger (vault-wide: 1,031)

find *.sync-conflict

13.4×

the naive escalation count over-reports (why we count proposals, not events)

403 events → 30 proposals

The claim the whole design rests on — now checked against the log

The safety invariant held. Perfectly.

0 / 17

Zero Tier-2 proposals — money, irreversible deletion, secrets, mass-send — auto-committed without human approval, across 15 days and 71 proposals. 12 of the 17 went to the human; the other 5 were machine-rejected or are still open. None slipped through. The system is hands-off where it's safe and conservative exactly where it isn't.

Risk tier	Proposals	Auto-committed w/o human	Went to human
Tier-0 · trivial	10	6	0
Tier-1 · reversible	44	29	5
Tier-2 · money / irreversible / secrets	17	0	12

Crosstab of risk tier against terminal disposition, per proposal. Reproduce with analyze_tiers_invariant.py.

Where 71 proposals ended up

Lifecycle disposition

Terminal state	Count	Share	Meaning
Auto-committed (no human)	35	49%	machines agreed and shipped it
Human-approved → committed	16	23%	escalated, approved, then shipped
Rejected	8	11%	machines declined on their own
Escalated, still open	6	8%	tripped timeout / Tier-2, awaiting
Accepted, in-flight	5	7%	not yet committed
Approved, commit pending	1	1%	in-flight

The auto-committed path is propose → accept → commit with ≥2 independent VERIFY events (121 across the window) — propagate-and-verify, logged rather than asserted.

Published because omitting them would make the evals dishonest

What we cannot measure yet honest gaps

NOT INSTRUMENTED

Token cost per consensus round

Ledger events carry no token or cost field. We refuse to eyeball a per-round cost. Fix: stamp tokens_in/out on every PROPOSE / RESPOND / VERIFY from the agent's own usage meter.

TOO FEW SAMPLES

Leader-down recovery time (MTTR)

Failover is designed and armed (one clean recovery logged 07-14 07:27Z), but the ledger doesn't timestamp each down→up episode with a duration. Fix: emit an explicit LEADER_DOWN / LEADER_RECOVERED pair so MTTR is a subtraction, not an inference.

NODE-LOCAL ONLY

Fleet-wide uptime history

Each node reports only itself, and only the current heartbeat snapshot is retained per peer. This Mac's background robot is currently down (~46h) — surfaced, not hidden. Fix: centralize heartbeat history for a fleet uptime series.

Reproducibility. Raw ledger snapshot, engine config, and two analysis scripts are frozen alongside this report. Run verify_claims.py against the snapshot and all ten headline numbers regenerate — 744 events, 71 proposals, 0/17 invariant, and the rest — or it prints DRIFT. Methodology: group by proposal_id, dedup by event_id, exclude smoke-test peers.

Draft · numbers verified 2026-07-14 · publication pending approval