Forge Runner — Declarative Clinical-Report Agent Platform

Forge Runner — Declarative Clinical-Report Agent Platform

Overview

What is Forge Runner?

Forge Runner is an agno-powered agent pipeline that turns a workflow name into a finished clinical HTML report**. One container image ships every report type; the report you get is selected entirely by the AGENT environment variable. There is no per-report service, no per-report image, and — increasingly — no per-report Python.

The whole interface
AGENT=health-summary docker run forge-runner

At docker run, bin/run.py reads the workflow’s manifest, assembles an isolated runtime workspace from shared resource pools, runs a framework-owned prep cascade that fetches and cleans patient data into a fixed on-disk contract, then dispatches to a single execution path that builds the right kind of agno agent and consumes its event stream. A deterministic, framework-owned post-hook renders, verifies, and publishes the report. The reasoning agent itself only reads patient data and writes its report artifact — everything dangerous (rendering, publishing, shell access) is taken away from it by construction.

The product context

Forge Runner is the report-generation engine of N1 — a HIPAA-compliant AI clinical-intelligence platform for licensed clinicians. Its outputs are clinical decision-support documents read by treating prescribers, not consumer wellness summaries. That framing drives every design decision in this document: patient data (PHI) is the input the AI reasons over, while the dangerous capabilities (publishing, mutation, network egress, autonomous prescription language) are removed at structural boundaries rather than asked-of-the-model in a prompt.

The N-of-1, whole-patient philosophy

N1 practises functional / integrative medicine: a patient is analysed as one integrated system, not organ-by-organ. Six data streams — biomarkers, genetics, diagnoses, procedures, medications, supplements — converge on shared mechanistic nodes. Reports are organised around mechanisms and cross-system patterns, never around data-modality silos. This is why the framework’s job is to assemble a coherent whole-patient picture and hand it to a reasoning agent, rather than to template-fill per-panel sections.

The Declarative Vision

The north star: to author a new clinical report, all anyone should need to do is drop in an AGENTS.md (one, or several). The framework auto-enriches that markdown with everything a report needs — the MCP toolbox, the pre-fetched patient context, the report build blocks (render / annotate / publish), and the guardrails. The author writes reasoning and report structure in markdown; they do not write Pydantic schemas, Jinja templates, or a bespoke renderer.

Everything generic, parameterised by content. Every mechanism — the agent-as-judge, the revise loop, the cast assembler, the terminal safety gate, the markdown build block, the guardrails — is built as a generic framework primitive parameterised by report-supplied content (rubric markdowns, member AGENTS.md files, the manifest). The first report on this path, Precision Therapeutics, ships only content — a cast of AGENTS.md files, rubric refs/, a manifest, and a mechanical validate.py — and zero bespoke mechanism Python.

Architecture

The same codebase plays two roles. Holding both views in mind at once is the key to not getting confused.

View 1 — Source-code layering (where Python lives, how imports flow)

• run.py — the single dispatch path

• prep/ — standard_prep + the cascade ops

• validate_workflows.py — CI manifest check

• data/ — queries, models, paths, views

• mcps/ — patient-data + 5 others

• runtime/ — workspace, post-pipeline, settings

• render/, html/, charts/, pdf/, observability/

• runner.py — build_agent()

• cast.py — declarative cast assembler

• agents_md.py, events.py, mcp.py, models.py

• guardrails.py, tool_hooks.py, continuation.py

• {name}.yaml — the manifest

• AGENTS.md — the orchestrator definition

• agents/, refs/, skills/, scripts/

Imports flow one direction only: bin/ → infrastructure/ ← workflows/. This is Rule 9. The only carve-out is a runtime: langgraph workflow, which may ship a Python package (a StateGraph cannot be expressed in markdown) that imports stdlib + declared deps + infrastructure/ only.

View 2 — Runtime execution flow (what happens at docker run)

Technology stack

The Five Stages

Every run, regardless of report shape, passes through the same five stages. The first three are pure framework; the agent owns the middle of stage 4; stage 5 is framework again.

2. Workflow bundle root agents/{workflow}/…
3. Workflow bundle subdir agents/{workflow}/agents/…
4. Workflow-local workflows/{workflow}/agents/, etc.

Names containing “/” are explicit repo-root paths and bypass the cascade.

Two Report Shapes

The entry AGENTS.md selects the shape via output_format. Both share prep, assembly, the patient-data surface, and publishing; they diverge only in what the agent emits and which post-hook runs.

The Declarative Cast

The headline new capability. A multi-agent report is declared as an orchestration: block in the manifest and compiled by forge_runner/cast.py:build_cast_workflow() into a live agno Workflow — with no per-report factory Python.

The orchestration block

workflows/{name}/{name}.yaml — declarative cast
orchestration:

kind: workflow steps: - agent: comprehend

writes: comprehend.md          # handoff file the member must leave in WORK_DIR

- agent: map-nodes

writes: mechanisms.md

- loop:                          # author ⇄ judge revise loop

author: design
writes: report.md
judge: judge
approve_when: APPROVED        # judge emits this token (standalone, non-negated) to exit
max_iterations: 2

- parallel: [verify-a, verify-b] # optional fan-out of member steps
- agent: source-verify
- agent: safety-judge

writes: safety-verdict.md

How it compiles — live objects, not from_dict

The cast assembler builds each member through the existing build_agent() (one AGENTS.md → one fully-enriched agno Agent: tools, hooks, MCP, model, guardrails), then composes the live objects into agno Workflow / Loop / Parallel via a thin manifest-to-constructor translator. It deliberately does not use Team.from_dict: that requires a DB to resolve members and silently drops them otherwise, and cannot carry the guardrail/MCP enrichment. Live-object composition needs no DB, no registry, and keeps every hook.

Token-match semantics for the approval gate

# forge_runner/cast.py — approval end-condition
# Whole-word match, not preceded by a letter:   (?<![A-Z])APPROVED\b
#   "APPROVED"   ✓ matches
#   "UNAPPROVED" ✗ no match (preceded by a letter)
# Negation guard:   \b(?:NOT|UN)[\s-]*APPROVED\b   → fails approval if negated
# Both the approval token and the judge step name come from the manifest;
# the callable is framework code, so the manifest stays purely declarative.

Per-member tool scope

A member declares which MCP servers it gets in its frontmatter mcps: and may narrow to a subset with mcp_tools: {server: [tool, …]}. The assembler resolves only the named tools (keyed mcp__{server}__{tool}) and hands the member a lean slice instead of a fat server’s whole surface. Omitting mcp_tools for a server gives the member all of that server’s tools.

Prep Cascade

The framework owns the entire data-prep lifecycle (Rule 2). Every workflow that enters prep receives the same operations in the same order; workflows cannot toggle steps on or off (Rule 11). The contract between framework and workflow is the set of files that land under work_dir/data/.

The cascade

bin/run.py → _run_prep
standard_prep  (bin/prep/standard.py)

├─ setup_work_env create WORK_DIR / OUTPUT_DIR, seed env ├─ load_patient_data fetch from N1 API via n1_api_client (skipped if SKIP_FETCH) ├─ validate_data_completeness fail fast on missing/malformed datasets ├─ augment_patient_age compute integer age from dob when missing └─ write_doctor_inputs surface DOCTOR_*_REQUEST env vars as files

│
▼  nine framework-owned ops, fixed order (bin/prep/common.py)

1. apply_data_filters honour RECORD_IDS / DATA_START_DATE / DATA_END_DATE
2. apply_short_ids add b…/d…/p…/m… + cb… short ids (UUID preserved)
3. apply_biomarker_rollups biomarker_current / biomarker_timeline / biomarker_index
4. apply_compute_clinical_summary clinical_summary.json (bucketed by short biomarker ID)
5. apply_categorize_diagnoses_by_system LLM diagnoses_by_system (soft-fail w/o API key)
6. apply_agent_briefings profile.md + per-entity .md + metadata.yaml
7. apply_derived_markers derived_markers(.longitudinal).json — HOMA-IR, eGFR, …
8. apply_chart_generation chart_configs.json + chart_metadata.json (Chart.js)
9. apply_clinical_findings clinical_findings.json — system map + outliers + borderlines

│
▼

apply_patient_briefing size-adaptive whole-patient briefing (see next section) finalize_workspace data/ tree complete — the completion contract

Order matters: filters before short_ids (so the projection only annotates the filtered scope), short_ids before rollups (so derived files carry both the short id and the canonical UUID), rollups before clinical_summary, and so on.

The work_dir/data/ contract

WORK_DIR layout — resolve every path via infrastructure/data/paths.py
WORK_DIR/

├── .checkpoints/ langgraph runtime state ├── .progress_* progress reporter state ├── output/ rendered report HTML ├── patient_briefing.md ← apply_patient_briefing │ └── data/

├── profile.md                 ← apply_agent_briefings
├── biomarker/  { biomarker.md, metadata.yaml }
├── diagnosis/  { diagnosis.md, metadata.yaml }
├── medication/ { medication.md, metadata.yaml }
├── procedure/  { procedure.md, metadata.yaml }
├── derived/    clinical_summary / clinical_findings / derived_markers(.longitudinal) / biomarker_timeline
├── charts/     chart_configs / chart_metadata / biomarker_index
└── json/       patient_info, biomarkers(+readings), diagnoses,

Why one framework-owned cascade. A single cascade is where safety, telemetry, retries, validation, and best practices live. Each capability is implemented once, audited once, and benefits every workflow. The same logic spread across N workflow prep scripts means N inconsistent implementations and slow rollout of any fix. New steps go in unconditionally for every workflow — there is no per-workflow opt-in.

Patient Briefing — size-adaptive context engineering

The final prep step, apply_patient_briefing, assembles patient_briefing.md — the curated whole-patient picture injected into the agent prompt as $PATIENT_BRIEFING. It must stay within a token budget no matter how large the patient’s dataset is. This is context engineering, not RAG: a model judges clinical salience from a cheap census, then a deterministic assembler fetches to budget.

The decision

1. Census. data_census(work_dir) returns total/abnormal biomarker counts, per-system breakdown, and date range from an index aggregate — zero model calls.
2. Budget check. A cheap estimator (~120 chars per finding ÷ 4 chars/token) compares the projected size against FORGE_BRIEFING_BUDGET_TOKENS (default 45 000). Under budget — the common case — no model call is made and the briefing includes everything.
3. Scout (only on overflow). A fast model (gemini-3.5-flash, temperature 0.3, 120 s timeout) returns a structured RetrievalPlan: which systems to lead with, per-system cap, outlier cap, trajectory cap, whether to include the borderline tail, and a one-line rationale. Any failure returns None and the assembler falls back to deterministic default caps.
4. Assemble. build_patient_briefing(work_dir, log, plan) reads existing prep output (no new computation) and emits the briefing: profile, derived markers, severity outliers, abnormals-by-system, trajectories, the full current regimen, all diagnoses, all procedures, genetics — each finding line carrying its cid for drill-down. A footer points at the efficient tool paths for what the briefing deliberately leaves out.

Patient Data & MCP Servers

The read-only patient-data MCP

Reasoning agents read patient data exclusively through the read-only patient-data MCP — a warm FastMCP process serving ~25 biomarker / chart / findings / diagnosis / medication queries against the run’s work_dir/data/. Tools surface to the LLM as mcp__patient-data__<tool>. It is read-only by construction: no mutating tools, no HTTP egress — an invariant locked by tests/test_patient_data_mcp.py.

The server is a thin shim over the importable query library infrastructure/data/queries/{biomarker,chart,findings,patient}.py, which prep, post-hooks, and tests also import directly — one logic implementation behind both the MCP and in-process callers.

The MCP registry

MCP servers are declared per-workflow in the manifest’s mcps: block and registered in MCP_REGISTRY (infrastructure/runtime/workspace.py). agno’s MCPTools is the single transport: forge_runner.mcp.build_mcp_tools_from_registry_entry turns each registry entry (stdio / http / sse) into one MCPTools instance entered as an async context manager, so the subprocess lifetime is tied to the run.

Reasoning-Only Agents

The agno report agents are reasoning-only. They query patient data and write their artifact(s) to WORK_DIR — that’s it. There is no shell and no whole-pod file access. Render, CID-mapping, sidepanel annotation, verification, and publishing are deterministic and framework-owned (the post-hook).

An agent’s three tool sources

1. AGENTS.md tools: — module:symbol refs. The only local tool reasoning agents get is forge_runner.tools_loader:make_workspace_file_tools, which returns an agno FileTools scoped to WORK_DIR (save_file / read_file / list_dir only — content/file search is off, so agents can’t grep the multi-MB raw data tree).
2. AGENTS.md native_tools: — raw provider-tool dicts passed through verbatim (e.g. agno-managed web search).
3. manifest mcps: — each MCP_REGISTRY entry becomes an MCPTools instance.

The workspace guard + two reliability hooks

build_agent installs two agent-wide tool_hooks on every agent — outermost first — and they propagate to every Function agno manages (FileTools, MCP tools, raw @tool callables):

Three guards against wedged loops

1. Friendly tool errors (prevention, every workflow) — the hook above.
2. Session summaries (prevention, opt-in) — for long-context workflows (90+ tool calls), enable_session_summaries: true in frontmatter (requires memory or checkpoint).
3. Loop detection (safety net, every workflow) — consume_events tracks consecutive identical ToolCallErrorEvent signatures; after FORGE_TOOL_LOOP_THRESHOLD (default 5) it sets the cancel signal and returns RunOutcome.ERROR — exits non-zero so K8s retries, and suppresses auto-publish so no stub HTML reaches clinicians. A separate runaway guard trips on identical successful calls (FORGE_TOOL_CALL_LOOP_THRESHOLD, default 150).

System prompt assembly

1. AGENTS.md body          workflows/{name}/AGENTS.md (frontmatter stripped)

2. Skills agno Skills — progressive disclosure: the model loads a skill’s

3. Report CSS / guidance appended by workflow content as needed

4. $PATIENT_BRIEFING etc. user_prompt template substituted at run time

Model resolution (first hit wins)

1. {AGENT_NAME}_MODEL env   per-agent override (e.g. DESIGN_MODEL); "-" → "_", uppercased

2. AGENTS.md model: frontmatter
3. LLM_MODEL / ANTHROPIC_MODEL env global override
4. claude-4.6-sonnet fallback

Post-Hook & Gates

Both halves of a run single-source their paths through infrastructure/runtime/run_paths.py (work_dir(), output_dir(), report_filename(), report_path()) so the agent and the post-hook agree on where the artifact and the published report live.

Typed post-hook

Composed from infrastructure/runtime/post_pipeline.py:

run_render(...)        Jinja2 ClinicalReport → HTML (sections in render/sections/)

run_biomarker_mapping(…) resolve series to Biomarker IDs and citations to exact BiomarkerReading IDs sidepanel-annotate embed per-entity data + interactive sidepanel run_verification(…) HTML exists, no unresolved placeholders, required sections present publish_report(…) copy rendered HTML to run_paths.report_path()

forge_runner/continuation.py adds in-session recovery for single agents only (runtime: agno): a nudge if the agent announced but didn’t write its artifact, and a content nudge driven by a workflow’s optional scripts/validate.py:check(work_dir) → list[str] hook — up to two repair rounds when the JSON is present but has blocking content problems.

Markdown post-hook — the fail-secure gates

infrastructure/runtime/markdown_post_hook.py runs the same render / cid / sidepanel / verify mechanics, plus two fail-secure gates and a resilience backstop. Gate order is data-fidelity first, then clinical safety.

The verdict token is read robustly: the first six non-empty lines are scanned, markdown wrapping (#, **, backticks, >) is stripped, and the cleaned uppercased token must stand alone on a line — not be buried in prose.

Precision Therapeutics — the first cast

Precision Therapeutics (PTR, slug precision-therapeutics) is the first declarative-markdown multi-agent cast and the report that proves the platform. It produces an integrated medication + supplement optimisation report — decision support for treating prescribers — and ships only content: a cast of AGENTS.md members, rubric refs/, a manifest, and a mechanical validate.py.

The seven-member cast

precision-therapeutics.yaml — the orchestration block
output_format: markdown

expected_agent_outputs: [report.md, report.evidence.json] safety_gate: safety-verdict.md fidelity_gate: fidelity-verdict.md mcps: [patient-data, pubmed]

orchestration: kind: workflow steps: - agent: comprehend

writes: comprehend.md

- agent: map-nodes

writes: mechanisms.md

- loop:

author: design
writes: report.md
judge: judge
approve_when: APPROVED
max_iterations: 2

- agent: source-verify
- agent: fact-check

writes: fidelity-verdict.md

- agent: safety-judge

writes: safety-verdict.md

Two quality gates (loop) + two safety gates (terminal)

The design ⇄ judge loop checks the report against two rubrics before downstream work: a completeness rubric (every medication carries its four mandatory fields — mechanism · this-patient applicability · citation · prescriber disposition; tiers risk-ordered; meds + supps interleaved; schedule; monitoring; all S0–S10 sections; the S9 audit covers every current item) and an evidence rubric (no dangling/orphan citations; every claim traces to a citation; med recs anchor to a guideline/CPIC/PMID; frontier items graded honestly). The judge is cheap (Haiku, no patient-data MCP) so revision rounds are fast.

The two terminal gates are fail-secure. fact-check confirms every value, unit, date, and absence-claim is faithful to the patient’s source — it does not judge clinical appropriateness (the report is allowed to override naive reference ranges with N-of-1 reasoning). safety-judge applies the hard-contraindication rubric: autonomous-prescription language, missing prescriber disposition, iron on masked ferritin, DOAC on a mechanical valve, RASi/finerenone without a K⁺ check, abrupt stop of taper-mandatory drugs, supplements against a hard constraint, or any recommendation with no this-patient trigger → BLOCK.

The 11 Framework Rules

Non-negotiable rules govern the framework / workflow / infrastructure boundary. The principle behind every one: shared infrastructure beats isolated implementations. A fix applied once in the framework propagates to every workflow; the same fix made inside a workflow benefits only that workflow and tends to drift. Source of truth: docs/reference/rules.md.

Reliability & Resilience

Fail-secure publishing chain

The two markdown gates plus quarantine-on-verification-failure mean a report only reaches a clinician if it cleared data-fidelity, cleared clinical-safety, rendered, resolved its citations, and passed structural verification. Any break in that chain leaves the report in WORK_DIR (quarantined) and exits non-zero. Default to closed, never open.

Observability

Two complementary signals: structured logs scraped into SigNoz, and OpenTelemetry metrics.

Structured logging

infrastructure/observability/telemetry.py emits JSON log records. consume_events writes one run_event record per agno RunEvent — the SigNoz log scraper picks them up — and a run_metrics line at the end carrying token counts, cost, duration, TTFT, and outcome. Tool failures and the workspace-guard rejections log as security/reliability events.

OTel metrics

forge_runner/metrics.py turns RunOutput.metrics into custom OTel metrics (tokens, cost, duration, outcome) exported over OTLP to SigNoz when OTEL_ENABLED=true. OTEL_EXPORT_FILE writes a local JSONL for debugging without a SigNoz endpoint.

Environment

The runtime is configured entirely via environment variables; bin/run.py reads them at startup and runners + MCP servers inherit them. Full reference: docs/reference/environment.md.

Workflow Catalogue

Adding a Workflow

The declarative path (markdown report)

You write content, never mechanism Python. The framework supplies the engine.

AGENT=my-report MOCK_DATA=1 OUTPUT_DIR=./output
OPENAI_API_KEY=sk-… uv run python bin/run.py

The “what goes where” reflex

Summary

Forge Runner at a glance

One agno-powered container image turns a workflow name into a finished, fail-secure clinical report. The framework owns everything dangerous and everything shared — fetching, cleaning, the patient-data surface, rendering, gating, publishing — and exposes a markdown declaration surface so a new report is authored as content, not code. The reasoning agent only reads patient data and writes its report; the framework does the rest, the same way for every workflow.

Key strengths

Repository pointers

• docs/architecture.md — the narrative mental model.
• docs/reference/ — manifest schema, env vars, prep cascade, MCP registry, patient-data tools, the 11 rules.
• docs/guides/ — task recipes (add a workflow / tool / MCP, run locally, run in Docker, debug with telemetry).
• docs/workflows/ — per-workflow detail pages.
• CLAUDE.md — the rules + “what goes where” table for AI coding assistants.

Forge Runner — internal architecture reference · N1 Healthcare (confidential) · generated from the live forge-runner repo (docs/, CLAUDE.md, and source).