--- title: "Forge Runner — Declarative Clinical-Report Agent Platform" --- Forge Runner — Declarative Clinical-Report Agent Platform ## Overview ### What is Forge Runner? Forge Runner is an agno-powered agent pipeline that turns a workflow name into a finished clinical HTML report**. One container image ships every report type; the report you get is selected entirely by the `AGENT` environment variable. There is no per-report service, no per-report image, and — increasingly — no per-report Python. ```bash The whole interface AGENT=health-summary docker run forge-runner ``` At `docker run`, `bin/run.py` reads the workflow's manifest, assembles an isolated runtime workspace from shared resource pools, runs a framework-owned **prep cascade** that fetches and cleans patient data into a fixed on-disk contract, then dispatches to a single execution path that builds the right kind of agno agent and consumes its event stream. A deterministic, framework-owned **post-hook** renders, verifies, and publishes the report. The reasoning agent itself only reads patient data and writes its report artifact — everything dangerous (rendering, publishing, shell access) is taken away from it by construction. ### The product context Forge Runner is the report-generation engine of **N1** — a HIPAA-compliant AI clinical-intelligence platform for licensed clinicians. Its outputs are clinical decision-support documents read by treating prescribers, not consumer wellness summaries. That framing drives every design decision in this document: **patient data (PHI) is the input the AI reasons over**, while the dangerous capabilities (publishing, mutation, network egress, autonomous prescription language) are removed at structural boundaries rather than asked-of-the-model in a prompt. ### The N-of-1, whole-patient philosophy N1 practises functional / integrative medicine: a patient is analysed as **one integrated system**, not organ-by-organ. Six data streams — biomarkers, genetics, diagnoses, procedures, medications, supplements — converge on shared mechanistic nodes. Reports are organised around *mechanisms and cross-system patterns*, never around data-modality silos. This is why the framework's job is to assemble a coherent whole-patient picture and hand it to a reasoning agent, rather than to template-fill per-panel sections. ## The Declarative Vision The north star: **to author a new clinical report, all anyone should need to do is drop in an `AGENTS.md` (one, or several)**. The framework auto-enriches that markdown with everything a report needs — the MCP toolbox, the pre-fetched patient context, the report build blocks (render / annotate / publish), and the guardrails. The author writes *reasoning and report structure in markdown*; they do **not** write Pydantic schemas, Jinja templates, or a bespoke renderer. **Everything generic, parameterised by content.** Every mechanism — the agent-as-judge, the revise loop, the cast assembler, the terminal safety gate, the markdown build block, the guardrails — is built as a *generic framework primitive parameterised by report-supplied content* (rubric markdowns, member `AGENTS.md` files, the manifest). The first report on this path, [Precision Therapeutics](#ptr), ships **only content** — a cast of `AGENTS.md` files, rubric `refs/`, a manifest, and a mechanical `validate.py` — and zero bespoke mechanism Python. ## Architecture The same codebase plays two roles. Holding both views in mind at once is the key to not getting confused. ### View 1 — Source-code layering (where Python lives, how imports flow) - run.py — the single dispatch path - prep/ — standard_prep + the cascade ops - validate_workflows.py — CI manifest check - data/ — queries, models, paths, views - mcps/ — patient-data + 5 others - runtime/ — workspace, post-pipeline, settings - render/, html/, charts/, pdf/, observability/ - runner.py — build_agent() - cast.py — declarative cast assembler - agents_md.py, events.py, mcp.py, models.py - guardrails.py, tool_hooks.py, continuation.py - {name}.yaml — the manifest - AGENTS.md — the orchestrator definition - agents/, refs/, skills/, scripts/ **Imports flow one direction only: `bin/` → `infrastructure/` ← `workflows/`.** This is Rule 9. The only carve-out is a `runtime: langgraph` workflow, which may ship a Python package (a `StateGraph` cannot be expressed in markdown) that imports stdlib + declared deps + `infrastructure/` only. ### View 2 — Runtime execution flow (what happens at docker run) ### Technology stack ## The Five Stages Every run, regardless of report shape, passes through the same five stages. The first three are pure framework; the agent owns the middle of stage 4; stage 5 is framework again. 2. Workflow bundle root agents/{workflow}/... 3. Workflow bundle subdir agents/{workflow}/agents/... 4. Workflow-local workflows/{workflow}/agents/, etc. # Names containing "/" are explicit repo-root paths and bypass the cascade. ## Two Report Shapes The entry `AGENTS.md` selects the shape via `output_format`. Both share prep, assembly, the patient-data surface, and publishing; they diverge only in what the agent emits and which post-hook runs. ## The Declarative Cast The headline new capability. A multi-agent report is declared as an `orchestration:` block in the manifest and compiled by `forge_runner/cast.py:build_cast_workflow()` into a live agno `Workflow` — **with no per-report factory Python**. ### The orchestration block ```json workflows/{name}/{name}.yaml — declarative cast orchestration: ``` kind: workflow steps: - agent: comprehend ```text writes: comprehend.md # handoff file the member must leave in WORK_DIR ``` - agent: map-nodes ```text writes: mechanisms.md ``` - loop: # author ⇄ judge revise loop ```text author: design writes: report.md judge: judge approve_when: APPROVED # judge emits this token (standalone, non-negated) to exit max_iterations: 2 ``` - parallel: [verify-a, verify-b] # optional fan-out of member steps - agent: source-verify - agent: safety-judge ```text writes: safety-verdict.md ``` ### How it compiles — live objects, not from_dict The cast assembler builds each member through the **existing `build_agent()`** (one `AGENTS.md` → one fully-enriched agno Agent: tools, hooks, MCP, model, guardrails), then composes the live objects into agno `Workflow` / `Loop` / `Parallel` via a thin manifest-to-constructor translator. It deliberately does *not* use `Team.from_dict`: that requires a DB to resolve members and silently drops them otherwise, and cannot carry the guardrail/MCP enrichment. Live-object composition needs no DB, no registry, and keeps every hook. ### Token-match semantics for the approval gate ```python # forge_runner/cast.py — approval end-condition # Whole-word match, not preceded by a letter: (?.md + metadata.yaml 7. apply_derived_markers derived_markers(.longitudinal).json — HOMA-IR, eGFR, … 8. apply_chart_generation chart_configs.json + chart_metadata.json (Chart.js) 9. apply_clinical_findings clinical_findings.json — system map + outliers + borderlines ```text │ ▼ ``` apply_patient_briefing size-adaptive whole-patient briefing (see next section) finalize_workspace data/ tree complete — the completion contract **Order matters:** filters before short_ids (so the projection only annotates the filtered scope), short_ids before rollups (so derived files carry both the short id and the canonical UUID), rollups before clinical_summary, and so on. ### The `work_dir/data/` contract ```text WORK_DIR layout — resolve every path via infrastructure/data/paths.py WORK_DIR/ ``` ├── .checkpoints/ langgraph runtime state ├── .progress_* progress reporter state ├── output/ rendered report HTML ├── patient_briefing.md ← apply_patient_briefing │ └── data/ ```json ├── profile.md ← apply_agent_briefings ├── biomarker/ { biomarker.md, metadata.yaml } ├── diagnosis/ { diagnosis.md, metadata.yaml } ├── medication/ { medication.md, metadata.yaml } ├── procedure/ { procedure.md, metadata.yaml } ├── derived/ clinical_summary / clinical_findings / derived_markers(.longitudinal) / biomarker_timeline ├── charts/ chart_configs / chart_metadata / biomarker_index └── json/ patient_info, biomarkers(+readings), diagnoses, ``` **Why one framework-owned cascade.** A single cascade is where safety, telemetry, retries, validation, and best practices live. Each capability is implemented once, audited once, and benefits every workflow. The same logic spread across N workflow prep scripts means N inconsistent implementations and slow rollout of any fix. New steps go in *unconditionally* for every workflow — there is no per-workflow opt-in. ## Patient Briefing — size-adaptive context engineering The final prep step, `apply_patient_briefing`, assembles `patient_briefing.md` — the curated whole-patient picture injected into the agent prompt as `$PATIENT_BRIEFING`. It must stay within a token budget no matter how large the patient's dataset is. This is **context engineering, not RAG**: a model judges clinical salience from a cheap census, then a deterministic assembler fetches to budget. ### The decision 1. **Census.** `data_census(work_dir)` returns total/abnormal biomarker counts, per-system breakdown, and date range from an index aggregate — *zero model calls*. 1. **Budget check.** A cheap estimator (~120 chars per finding ÷ 4 chars/token) compares the projected size against `FORGE_BRIEFING_BUDGET_TOKENS` (default **45 000**). Under budget — the common case — **no model call is made** and the briefing includes everything. 1. **Scout (only on overflow).** A fast model (`gemini-3.5-flash`, temperature 0.3, 120 s timeout) returns a structured `RetrievalPlan`: which systems to lead with, per-system cap, outlier cap, trajectory cap, whether to include the borderline tail, and a one-line rationale. Any failure returns `None` and the assembler falls back to deterministic default caps. 1. **Assemble.** `build_patient_briefing(work_dir, log, plan)` reads existing prep output (no new computation) and emits the briefing: profile, derived markers, severity outliers, abnormals-by-system, trajectories, the full current regimen, all diagnoses, all procedures, genetics — each finding line carrying its `cid` for drill-down. A footer points at the efficient tool paths for what the briefing deliberately leaves out. ## Patient Data & MCP Servers ### The read-only patient-data MCP Reasoning agents read patient data exclusively through the **read-only `patient-data` MCP** — a warm FastMCP process serving ~25 biomarker / chart / findings / diagnosis / medication queries against the run's `work_dir/data/`. Tools surface to the LLM as `mcp__patient-data__`. It is read-only by construction: **no mutating tools, no HTTP egress** — an invariant locked by `tests/test_patient_data_mcp.py`. The server is a thin shim over the importable query library `infrastructure/data/queries/{biomarker,chart,findings,patient}.py`, which prep, post-hooks, and tests also import directly — **one logic implementation behind both the MCP and in-process callers**. ### The MCP registry MCP servers are declared per-workflow in the manifest's `mcps:` block and registered in `MCP_REGISTRY` (`infrastructure/runtime/workspace.py`). agno's `MCPTools` is the single transport: `forge_runner.mcp.build_mcp_tools_from_registry_entry` turns each registry entry (stdio / http / sse) into one `MCPTools` instance entered as an async context manager, so the subprocess lifetime is tied to the run. ## Reasoning-Only Agents The agno report agents are **reasoning-only**. They query patient data and write their artifact(s) to `WORK_DIR` — that's it. There is no shell and no whole-pod file access. Render, CID-mapping, sidepanel annotation, verification, and publishing are deterministic and framework-owned (the post-hook). ### An agent's three tool sources 1. **`AGENTS.md tools:`** — module:symbol refs. The only local tool reasoning agents get is `forge_runner.tools_loader:make_workspace_file_tools`, which returns an agno `FileTools` scoped to `WORK_DIR` (`save_file` / `read_file` / `list_dir` only — content/file *search* is off, so agents can't grep the multi-MB raw data tree). 1. **`AGENTS.md native_tools:`** — raw provider-tool dicts passed through verbatim (e.g. agno-managed web search). 1. **manifest `mcps:`** — each `MCP_REGISTRY` entry becomes an `MCPTools` instance. ### The workspace guard + two reliability hooks `build_agent` installs two agent-wide `tool_hooks` on *every* agent — outermost first — and they propagate to every Function agno manages (FileTools, MCP tools, raw `@tool` callables): ### Three guards against wedged loops 1. **Friendly tool errors (prevention, every workflow)** — the hook above. 1. **Session summaries (prevention, opt-in)** — for long-context workflows (90+ tool calls), `enable_session_summaries: true` in frontmatter (requires `memory` or `checkpoint`). 1. **Loop detection (safety net, every workflow)** — `consume_events` tracks consecutive identical `ToolCallErrorEvent` signatures; after `FORGE_TOOL_LOOP_THRESHOLD` (default 5) it sets the cancel signal and returns `RunOutcome.ERROR` — exits non-zero so K8s retries, and suppresses auto-publish so no stub HTML reaches clinicians. A separate runaway guard trips on identical *successful* calls (`FORGE_TOOL_CALL_LOOP_THRESHOLD`, default 150). ### System prompt assembly ```json 1. AGENTS.md body workflows/{name}/AGENTS.md (frontmatter stripped) ``` 2. Skills agno Skills — progressive disclosure: the model loads a skill's 3. Report CSS / guidance appended by workflow content as needed 4. $PATIENT_BRIEFING etc. user_prompt template substituted at run time ### Model resolution (first hit wins) ```json 1. {AGENT_NAME}_MODEL env per-agent override (e.g. DESIGN_MODEL); "-" → "_", uppercased ``` 2. AGENTS.md model: frontmatter 3. LLM_MODEL / ANTHROPIC_MODEL env global override 4. claude-4.6-sonnet fallback ## Post-Hook & Gates Both halves of a run single-source their paths through `infrastructure/runtime/run_paths.py` (`work_dir()`, `output_dir()`, `report_filename()`, `report_path()`) so the agent and the post-hook agree on where the artifact and the published report live. ### Typed post-hook Composed from `infrastructure/runtime/post_pipeline.py`: ```text run_render(...) Jinja2 ClinicalReport → HTML (sections in render/sections/) ``` run_biomarker_mapping(...) resolve series to Biomarker IDs and citations to exact BiomarkerReading IDs sidepanel-annotate embed per-entity data + interactive sidepanel run_verification(...) HTML exists, no unresolved placeholders, required sections present publish_report(...) copy rendered HTML to run_paths.report_path() `forge_runner/continuation.py` adds in-session recovery for **single agents only** (`runtime: agno`): a nudge if the agent announced but didn't write its artifact, and a content nudge driven by a workflow's optional `scripts/validate.py:check(work_dir) → list[str]` hook — up to two repair rounds when the JSON is present but has blocking content problems. ### Markdown post-hook — the fail-secure gates `infrastructure/runtime/markdown_post_hook.py` runs the same render / cid / sidepanel / verify mechanics, *plus* two fail-secure gates and a resilience backstop. Gate order is data-fidelity first, then clinical safety. The verdict token is read robustly: the first six non-empty lines are scanned, markdown wrapping (`#`, `**`, backticks, `>`) is stripped, and the cleaned uppercased token must stand alone on a line — not be buried in prose. ## Precision Therapeutics — the first cast Precision Therapeutics (PTR, slug `precision-therapeutics`) is the first declarative-markdown multi-agent cast and the report that proves the platform. It produces an integrated **medication + supplement optimisation** report — decision support for treating prescribers — and ships *only content*: a cast of `AGENTS.md` members, rubric `refs/`, a manifest, and a mechanical `validate.py`. ### The seven-member cast ```text precision-therapeutics.yaml — the orchestration block output_format: markdown ``` expected_agent_outputs: [report.md, report.evidence.json] safety_gate: safety-verdict.md fidelity_gate: fidelity-verdict.md mcps: [patient-data, pubmed] orchestration: kind: workflow steps: - agent: comprehend ```text writes: comprehend.md ``` - agent: map-nodes ```text writes: mechanisms.md ``` - loop: ```text author: design writes: report.md judge: judge approve_when: APPROVED max_iterations: 2 ``` - agent: source-verify - agent: fact-check ```text writes: fidelity-verdict.md ``` - agent: safety-judge ```text writes: safety-verdict.md ``` ### Two quality gates (loop) + two safety gates (terminal) The **design ⇄ judge loop** checks the report against two rubrics *before* downstream work: a *completeness* rubric (every medication carries its four mandatory fields — mechanism · this-patient applicability · citation · prescriber disposition; tiers risk-ordered; meds + supps interleaved; schedule; monitoring; all S0–S10 sections; the S9 audit covers every current item) and an *evidence* rubric (no dangling/orphan citations; every claim traces to a citation; med recs anchor to a guideline/CPIC/PMID; frontier items graded honestly). The judge is cheap (Haiku, no patient-data MCP) so revision rounds are fast. The two **terminal gates** are fail-secure. `fact-check` confirms every value, unit, date, and absence-claim is faithful to the patient's source — it does *not* judge clinical appropriateness (the report is allowed to override naive reference ranges with N-of-1 reasoning). `safety-judge` applies the hard-contraindication rubric: autonomous-prescription language, missing prescriber disposition, iron on masked ferritin, DOAC on a mechanical valve, RASi/finerenone without a K⁺ check, abrupt stop of taper-mandatory drugs, supplements against a hard constraint, or any recommendation with no this-patient trigger → `BLOCK`. ## The 11 Framework Rules Non-negotiable rules govern the framework / workflow / infrastructure boundary. The principle behind every one: **shared infrastructure beats isolated implementations**. A fix applied once in the framework propagates to every workflow; the same fix made inside a workflow benefits only that workflow and tends to drift. Source of truth: `docs/reference/rules.md`. ## Reliability & Resilience ### Fail-secure publishing chain The two markdown gates plus quarantine-on-verification-failure mean a report only reaches a clinician if it cleared data-fidelity, cleared clinical-safety, rendered, resolved its citations, and passed structural verification. Any break in that chain leaves the report in `WORK_DIR` (quarantined) and exits non-zero. **Default to closed, never open.** ## Observability Two complementary signals: structured logs scraped into SigNoz, and OpenTelemetry metrics. ### Structured logging `infrastructure/observability/telemetry.py` emits JSON log records. `consume_events` writes one `run_event` record per agno `RunEvent` — the SigNoz log scraper picks them up — and a `run_metrics` line at the end carrying token counts, cost, duration, TTFT, and outcome. Tool failures and the workspace-guard rejections log as security/reliability events. ### OTel metrics `forge_runner/metrics.py` turns `RunOutput.metrics` into custom OTel metrics (tokens, cost, duration, outcome) exported over OTLP to SigNoz when `OTEL_ENABLED=true`. `OTEL_EXPORT_FILE` writes a local JSONL for debugging without a SigNoz endpoint. ## Environment The runtime is configured entirely via environment variables; `bin/run.py` reads them at startup and runners + MCP servers inherit them. Full reference: `docs/reference/environment.md`. ## Workflow Catalogue ## Adding a Workflow #### The declarative path (markdown report) You write *content*, never mechanism Python. The framework supplies the engine. AGENT=my-report MOCK_DATA=1 OUTPUT_DIR=./output \ OPENAI_API_KEY=sk-... uv run python bin/run.py ### The "what goes where" reflex ## Summary ### Forge Runner at a glance One agno-powered container image turns a workflow name into a finished, fail-secure clinical report. The framework owns everything dangerous and everything shared — fetching, cleaning, the patient-data surface, rendering, gating, publishing — and exposes a markdown declaration surface so a new report is authored as content, not code. The reasoning agent only reads patient data and writes its report; the framework does the rest, the same way for every workflow. ### Key strengths ### Repository pointers - `docs/architecture.md` — the narrative mental model. - `docs/reference/` — manifest schema, env vars, prep cascade, MCP registry, patient-data tools, the 11 rules. - `docs/guides/` — task recipes (add a workflow / tool / MCP, run locally, run in Docker, debug with telemetry). - `docs/workflows/` — per-workflow detail pages. - `CLAUDE.md` — the rules + "what goes where" table for AI coding assistants. Forge Runner — internal architecture reference · N1 Healthcare (confidential) · generated from the live `forge-runner` repo (`docs/`, `CLAUDE.md`, and source).