Evidence Studio guide

Agents v0.10 · Early access · Updated 23 October 2026

Evidence mode searches local source text and quotes excerpts. It does not call a language model, generate embeddings, parse arbitrary documents or crawl URLs.

Prepare sources

Write one clear procedure per source and publish it in Knowledge. Draft and archived sources are excluded from new snapshots. Categories match exact names and control retrieval scope, not document authorization. URLs are metadata only. Confirm the source is suitable for every operator with workspace access.

Tune a profile

Defaults: chunk size 500 characters, overlap 60, topK three, minimum score 0.1, title weight 2 and phrase weight 3. A blank category list includes all eligible sources. Synonyms use term = alternative | alternative, one rule per line. Synonym triggers and query policy phrases use literal normalized substring rules; broad triggers may match unintended questions.

Inspect the debugger

Search displays expanded terms, excerpts, score components, chunk offsets and hashes. Scores rank lexical matches and are not confidence or correctness probabilities. Offsets use UTF-16 indices in snapshot text. Inventory reports source/chunk sizes and duplicate normalized chunks. Read for omitted conditions, stale advice and incorrect product scope.

Build a saved benchmark

Add up to 20 named cases with question, expected IDs, forbidden IDs and abstention. Positive cases need expected IDs; abstention cases cannot have expected IDs. Choose minimum MRR from zero to one. Save all edits saves settings and cases atomically when both panels changed.

Recall is the fraction of expected IDs retrieved. MRR measures the rank of the first expected source among distinct IDs. A case passes when abstention matches, no forbidden source appears and at least one expected source appears for a positive case. Publication requires a positive case, all cases passing and the MRR threshold. Complete recall of all expected sources is not required; inspect it explicitly when needed. These checks do not grade semantic correctness.

Publish and test runtime

1. Run the saved suite and inspect per-case results.
2. Open Releases and request publication preview.
3. Review the exact revision and run, then confirm before expiry.
4. Start a new session from the published profile.
5. Test a supported answer, unsupported query and optional operator queuing.

Changes to configuration, corpus or suite invalidate old passing evidence. Previews are actor-bound and expire after ten minutes. Published versions and active sessions retain their fixed snapshot. Restoring an older version stages settings as a draft that must be retested against current knowledge.

Abstention and limits

Abstention reasons include empty corpus, no searchable terms, query policy and below threshold. Optional queuing sets waiting; it does not join a human audio room. Limits: ten profiles, ten versions each, 100 sources or 250,000 source-content characters per corpus, 2,000 chunks, five returned chunks, 2,000 question characters and 100 searchable input terms. Recent run retention is bounded.

Retained copies

Archiving an original source does not revoke older session/publication copies. Export includes source text and requires confidential handling. Import preview validates settings only, not automatic corpus or case import. Runtime reports omit question/answer text; full transcripts remain elsewhere in the workspace.