Target and scope
Bind every result to one named target, immutable version, workflow purpose and synthetic resource boundary.
Experimental · synthetic only · open evidence
Synthetic Evaluation Starter Kit v1
Declare one agent workflow before testing it: target and version, simulated tools, resource scope, human approvals, invariants, stop conditions, fault scenarios and evidence capture. Then anchor the result to deterministic, replayable artefacts.
6 scenariosRunnable reference fault classes
12 tracesControl and unsafe pairs
SHA-256 lockCanonical asset integrity
Fail closedMissing declarations are refused
What you declare
Make the action boundary explicit first.
The template turns an informal demo into a reviewable evidence contract without requiring a production connection.
Tools and authority
Declare effects, call limits and approval points. Agent self-approval is prohibited.
Safety and stops
State invariants, stop conditions and expected safe behaviour before running a scenario.
Evidence
Capture messages, tool inputs/results, state transitions, approvals and a hash-linked trace.
Downloadable governed assets
Start from the canonical pack.
| Kit manifest | Open JSON |
|---|---|
| Manifest schema | Open Draft 2020-12 schema |
| Custom-flow template | Download synthetic template |
| Integrity lock | Open SHA-256 asset inventory |
| Reference assessment | Open machine-readable result · Read Markdown result |
| Reference traces | Inspect all paired evidence |
Provenance: every custom flow must replace the unresolved author placeholder with its declared synthetic author. The site build compares each public kit file byte-for-byte with its governed source and fails on drift.
Non-negotiable boundary
Useful because it is constrained.
The kit accepts
- Fictional scenarios authored for evaluation
- Simulated tools with no external effects
- Declared targets, versions and human authority
- Reproducible controls, failures and retests
The kit refuses
- Real-person, pseudonymised or claimed-redacted patient data
- Production systems, external actions or production credentials
- Undeclared tools, scope, versions or authority
- Agent self-approval of consequential actions
This kit is not clinical care or a clinical protocol. It is not certification, conformity assessment, DCB0129 or DCB0160 sign-off, regulatory approval, and it is not deployment authority. It does not establish that a product is safe or effective outside the declared synthetic test.