--- title: "N1 Healthcare — CHR Workflow Architecture" --- N1 Healthcare — CHR Workflow Architecture ## Overview The workflow-health-summary** generates a 1-page patient health summary via a 6-stage integrative medicine cascade. It analyzes biomarker data through deterministic scoring, enrichment, and derived marker computation, then uses two LLM calls to identify clinical patterns and produce structured clinical summaries. The final output is deterministically rendered to HTML via `chr-html`. ```text Pipeline Stages LLM Calls Forge Libraries Pydantic Models ``` **Philosophy:** This repo is *pure analysis logic*. No infrastructure, no templates, no API clients. All shared infra comes from forge-sentinel libs. ### Run Modes ##### Local Mode JSON file → cascade → HTML. No credentials required. `INPUT_FILE=tests/fixtures/patient-small.json uv run python bin/run.py` ##### Forge Mode API fetch → cascade → HTML + PDF, with billing and cloud upload. Requires `USER_ID`, `CHR_ID`, LiteLLM and N1 API credentials. ## Pipeline Flowchart 3 deterministic stages + 2 LLM calls + 1 render. The LLM receives the complete enriched picture and reasons about patterns across markers (integrative medicine approach). ```text S0 Pre-filter Deterministic chr-data (reference) • Pure scoring: deviation, trend, recency, density S1 Context Assembly Deterministic chr-data (reference: optimal ranges, body systems, meds, unit conversion) S2 Derived Markers Deterministic chr-data (DERIVED_REGISTRY, unit conversion) S3 Pattern Recognition LLM chr-llm (allm_json_call, load_prompt) • chr-langfuse (tracing) S4 Clinical Summary LLM chr-llm (allm_json_call, load_prompt) • chr-langfuse (tracing) S5 Render Deterministic chr-html (html_document, build_report) • chr-styles (CSS) • chr-pdf (WeasyPrint) ``` ## Stage Details #### Pre-filter — Deterministic Scoring Groups biomarkers by biomarker ID, scores each group, then selects top 50 unhealthy + top 50 healthy biomarker groups (100 total). Each selected group carries its full measurement history. ```text Input `dict[str, BiomarkerInput]` Output `Stage0Output` ``` ##### Unhealthy Group Scoring (max 80) - **worst_deviation (30):** Highest deviation across all measurements vs reference midpoint - **trend (20):** Linear regression slope across full history (3+ data points required) - **recency (10):** ≤2yr = 10, ≤5yr = 5, older = 0 - **data_richness (10):** ≥5 pts = 10, ≥3 = 7, 2 = 4, 1 = 1 - **oor_consistency (10):** Percentage of measurements that are abnormal ##### Healthy Group Scoring (max 80) - **clinical_significance (30):** High-value markers (glucose, HbA1c, TSH, lipids, etc.) = 30; others = 10 - **stability (20):** Fraction of measurements that are normal × 20 - **recency (10):** Same as unhealthy - **data_richness (10):** Same as unhealthy - **system_coverage (10):** Bonus for underrepresented body systems ##### Hard Rules - Severely abnormal groups (worst_deviation ≥ 20) always included - Qualitative biomarkers excluded - If one pool has fewer than 50, overflow fills from the other ```text chr-data ``` #### Context Assembly — Functional Enrichment Enriches scored biomarkers with functional/optimal ranges, body system classification, and medication interaction context. All lookups come from `chr-data`'s reference module. ```text Input `Stage0Output + PatientContext?` Output `ContextEnrichedInput` ``` ##### Functional Classification - **optimal:** Within both lab range and functional/optimal range - **within_lab:** Within lab range, no optimal range defined - **outside_optimal:** Within lab range but outside functional range - **outside_lab:** Outside standard lab reference range - **unknown:** Non-numeric value ##### Key Operations - Unit conversion to conventional units via `convert_to_conventional()` - Optimal range lookup via `get_optimal_range(canonical)` - Body system mapping via `get_body_systems(canonical)` - Medication interaction detection via `get_affected_markers(med_name)` - Data quality assessment (recency, trend availability, system coverage) ```text chr-data ``` #### Derived Markers — Computed Ratios & Indices Computes clinically meaningful derived markers from available biomarkers. Uses `DERIVED_REGISTRY` from `chr-data` which contains formulas, parameter mappings, and interpretation logic. ```text Input `ContextEnrichedInput + PatientContext?` Output `DerivedMarkersOutput` ``` ##### Example Derived Markers - **TG/HDL Ratio:** Triglycerides / HDL — insulin resistance proxy - **HOMA-IR:** (Glucose × Insulin) / 405 — insulin resistance - **eGFR:** CKD-EPI formula (requires age + sex) — kidney function - **NLR:** Neutrophils / Lymphocytes — inflammation marker ##### Key Operations - Builds biomarker_id → numeric_value lookup (highest prefilter score wins) - All values converted to conventional units before computation - Demographics-dependent markers (eGFR) require age and sex from PatientContext - Skipped markers tracked separately for transparency ```text chr-data ``` #### Pattern Recognition — LLM Call #1 LLM identifies 2-6 clinical patterns across enriched biomarkers, providing root cause hypotheses, gap analysis, and cross-system connections. Thinks like a functional medicine doctor. ```text Input `ContextEnrichedInput + DerivedMarkersOutput` Output `PatternAnalysisOutput` ``` ##### Prompt Template Variables - {{enriched_biomarkers_json}} — Full enriched biomarker data - {{derived_markers_json}} — Computed derived markers - {{context_section}} — Patient context (optional) - {{data_quality_json}} — Data quality summary ##### Post-LLM Validation - Filter evidence IDs to only those present in input data - Skip patterns with zero valid supporting evidence - Downgrade confidence from "high" to "medium" when all evidence is stale (>2 years) - Enforce 1-6 pattern count (fallback pattern if none survive validation) - Severity floor: pattern can't be "monitor" if supporting biomarker has high deviation (≥60 score + outside_lab) - Force trajectory to "insufficient_data" when no longitudinal data exists ```text chr-llm chr-langfuse ``` #### Clinical Summary — LLM Call #2 LLM produces the final structured summary: headline, key findings, recommendations, gap recommendations, and overall status. This is the content that gets rendered into the report. ```text Input `PatternAnalysisOutput + ContextEnrichedInput + DerivedMarkersOutput` Output `ClinicalSummaryOutput` ``` ##### Prompt Template Variables - {{patterns_json}} — Clinical patterns from S3 - {{enriched_biomarkers_json}} — Enriched biomarkers (compact) - {{derived_markers_json}} — Derived markers - {{context_section}} — Patient context (optional) ##### Post-LLM Validation - Filter findings to valid measurement_ids only - Force trend to "insufficient_data" when no longitudinal data - Ensure critical pattern biomarkers appear in findings - Cap findings: min 3, max 10 (pad with highest-scored biomarkers if needed) - Cap recommendations: min 2, max 5 (add generic fallbacks if needed) - Deterministic override: overall_status based on pattern severities (critical → attention_needed) - Medication dedup: flag recommendations that suggest starting a medication the patient already takes ```text chr-llm chr-langfuse ``` #### Render — Deterministic HTML Converts `ClinicalSummaryOutput` to a complete HTML document using `chr-html`'s component library and `chr-styles`'s CSS system. No LLM-generated HTML — the presentation layer is fully deterministic. ```text Input `ClinicalSummaryOutput` Output `HTML string → PDF via chr-pdf` ``` ##### Render Components - **Header:** Patient name, date, overall status badge - **Headline:** Key finding + trajectory summary - **Key Findings:** Severity dots, trend arrows, functional notes - **Clinical Patterns:** Pattern cards with severity borders and root cause - **Recommendations:** Category-tagged action items (excluding "test" category) - **Recommended Tests:** Gap recommendations (test-specific) - **Footer:** AI disclaimer + confidentiality notice ##### CSS Pipeline - `html_document(body, report_type="single-page", override="health-summary")` - Base CSS + single-page type CSS + health-summary override CSS layered - PDF rendering via WeasyPrint through `chr-pdf` ```text chr-html chr-styles chr-pdf chr-document-manager ``` ## Full Prompts Both prompts use a system/human split loaded via `chr-llm`'s `load_prompt()`. The system message is everything above the `---` separator, the human template is everything below. Template variables are highlighted in amber. ```text pattern_recognition.md — Stage 3 Prompt (78 lines) S3 • LLM **Copy ``` You are an integrative medicine physician analyzing biomarker data. Your task is to identify clinical patterns — connections across biomarkers that reveal underlying health stories. Think like a functional medicine doctor: look at the whole picture, ask "why", and identify what's missing. Always respond with valid JSON only. No markdown, no explanation, just JSON. --- ## Clinical Pattern Analysis You are analyzing biomarker data for a patient health summary. The biomarkers have been enriched with functional/optimal ranges (tighter than standard lab ranges) and organized by body system. Derived markers (ratios, indices) have been computed where data permits. ### How to Think About This 1. **Look for clusters, not isolated values.** A single mildly elevated marker means little. Three markers in the same direction across two systems tells a story. 2. **Use functional ranges first.** A glucose of 95 is "normal" by lab standards but already suboptimal functionally. When both lab AND functional ranges are abnormal, that's more significant. 3. **Consider root causes.** Elevated glucose + elevated triglycerides + low HDL isn't three separate problems — it's insulin resistance. Name the root cause. 4. **Note what's missing.** If you see elevated inflammatory markers but no thyroid panel, that's a gap worth noting. 5. **Assess trajectory.** A value that's been worsening over 3 data points matters more than a single snapshot. 6. **Cross-system connections matter.** Thyroid dysfunction affects cholesterol. Insulin resistance drives inflammation. Connect the dots. ### Input Data #### Enriched Biomarkers (with functional ranges and body systems) {{enriched_biomarkers_json}} #### Derived Markers (computed ratios and indices) {{derived_markers_json}} {{context_section}} #### Data Quality Summary {{data_quality_json}} ### Rules 1. Identify 2-6 clinical patterns. Each pattern must reference at least 1 biomarker by measurement_id. 2. For each pattern, provide: - A root cause hypothesis (what's driving this pattern?) - Supporting evidence (measurement_ids that support this pattern) - Contradicting evidence (measurement_ids that argue against, if any) - Trajectory assessment - Affected body systems 3. Include gap analysis: what tests are missing that would confirm or refute your hypotheses? 4. Note cross-system connections when they exist. 5. Use ONLY measurement_ids that exist in the input data. Do not invent biomarker IDs. 6. Severity should reflect the clinical significance of the entire pattern, not just individual values. 7. When derived markers are available (HOMA-IR, TG/HDL ratio, etc.), use them to strengthen or modify your pattern assessment. 8. When medication context is available, consider whether abnormal values are expected given the medication (e.g., B12 depletion on metformin is expected, not a new finding). ### Output Format Return a JSON object: ``` { "patterns": [ ```json { "name": "", "severity": "critical" | "significant" | "monitor", "confidence": "high" | "medium" | "low", "narrative": "", "supporting_evidence": ["", ...], "contradicting_evidence": ["", ...], "root_cause_hypothesis": "", "body_systems": ["", ...], "trajectory": "worsening" | "improving" | "stable" | "insufficient_data" } ``` ], "gap_analysis": [ ```json { "test_name": "", "reason": "", "related_pattern": "", "priority": "high" | "medium" | "low" } ``` ], "cross_system_connections": [ ```text "" ``` ] } ``` ```text clinical_summary.md — Stage 4 Prompt (98 lines) S4 • LLM Copy ``` ``` You are writing the final content for a 1-page patient health summary. You have clinical patterns, enriched biomarker data, and derived markers. Now produce the summary content that will be rendered into the report. Always respond with valid JSON only. No markdown, no explanation, just JSON. --- ## Clinical Summary Generation You are producing the final structured output for a patient health summary report. You have: 1. Clinical patterns identified by an integrative medicine analysis 2. Enriched biomarkers with functional/optimal ranges 3. Derived markers (computed ratios and indices) ### Input Data #### Clinical Patterns {{patterns_json}} #### Enriched Biomarkers {{enriched_biomarkers_json}} #### Derived Markers {{derived_markers_json}} {{context_section}} ### Rules #### Headline - Single sentence, max 15 words. - Reference the most clinically significant pattern, not just one biomarker. - Tone: direct, informative, not alarmist. - When patient context is available, acknowledge known diagnoses if they relate to the most significant findings. #### Key Findings (3-10) - Prioritize: critical first, then significant, then monitor. - Each finding needs: biomarker name, value with unit, severity, trend, and context (max 15 words). - Include a functional_note when the value is within lab range but outside functional/optimal range (e.g., "Outside optimal: <90 mg/dL"). - Reference the related pattern when applicable. - When medications are known, include medication context (e.g., "LDL 145 on atorvastatin — above goal"). - Maximum 10 findings. #### Recommendations (2-5) - Categorize each: "test" (order a test), "lifestyle" (diet/exercise/sleep), "discuss" (talk to provider), "monitor" (retest/track), "urgent" (immediate action). - Be specific. Not "improve diet" but "increase omega-3 intake to improve TG/HDL ratio." - Include a brief rationale for each recommendation. - CRITICAL: Do NOT suggest starting a medication the patient is already taking. - Reference the pattern that drives this recommendation. #### Gap Recommendations - Tests that should be ordered to fill data gaps identified in the pattern analysis. - Each with a reason and priority. #### Overall Status - "attention_needed": any critical pattern - "monitoring_recommended": significant patterns but no critical - "generally_healthy": only monitor-level patterns #### Trajectory Summary - One sentence summarizing the overall health trajectory based on pattern trajectories. ### Output Format Return a JSON object: ``` { "headline": "", "key_findings": [ ```json { "measurement_id": "", "display_name": "", "value": "", "severity": "critical" | "significant" | "monitor", "trend": "worsening" | "improving" | "stable" | "insufficient_data", "context": "<1-line explanation, max 15 words>", "functional_note": "", "pattern_ref": "" } ``` ], "recommendations": [ ```json { "priority": 1-5, "category": "test" | "lifestyle" | "discuss" | "monitor" | "urgent", "action": "", "rationale": "", "pattern_ref": "" } ``` ], "gap_recommendations": [ ```json { "test_name": "", "reason": "", "priority": "high" | "medium" | "low" } ``` ], "overall_status": "attention_needed" | "monitoring_recommended" | "generally_healthy", "trajectory_summary": "" } ``` ## Data Contracts All Pydantic models live in `cascade/models.py`. Models are grouped by the stage that produces them. ### Input Models BiomarkerInput — Raw biomarker data ```text Copy ``` class BiomarkerInput(BaseModel): ```text """Biomarker data matching workflow-functional's BiomarkerInsight schema.""" test_name: str measurement_id: str value: str status: str reference_range: str unit: str | None = None test_date: str | None = None all_measurements: list[dict[str, Any]] | None = None file_name: str | None = None biomarker_id: str | None = None record_id: str | None = None ``` CascadeInput — Top-level pipeline input ```text Copy ``` ``` class CascadeInput(BaseModel): ```text """Top-level input to the cascade pipeline.""" patient_name: str = Field(description="Display name for the report header") report_date: str = Field(description="Report date in human-readable format") biomarkers: dict[str, BiomarkerInput] = Field(description="measurement_id -> BiomarkerInput") context: PatientContext | None = Field(default=None) ``` ``` PatientContext — Non-biomarker clinical data ```text Copy ``` ``` class PatientContext(BaseModel): ```text """Non-biomarker patient data passed to LLM stages for clinical context.""" patient_name: str = "Patient" age: int | None = None gender: str | None = None diagnoses: list[dict[str, Any]] = Field(default_factory=list) medications: list[dict[str, Any]] = Field(default_factory=list) procedures: list[dict[str, Any]] = Field(default_factory=list) genetics: list[dict[str, Any]] = Field(default_factory=list) report_date: str = "" enabled_types: set[str] = Field( default_factory=lambda: set(ALL_CONTEXT_TYPES) ) ``` ``` ### Stage 0 Output Stage0Output, ScoredBiomarker, ScoredCanonicalGroup ```text Copy ``` ``` class ScoredBiomarker(BaseModel): ```text """Biomarker with deterministic relevance score.""" biomarker: BiomarkerInput total_score: float components: dict[str, float] = Field(default_factory=dict) ``` class ScoredCanonicalGroup(BaseModel): ```text """Patient biomarker group with all measurements and group-level scoring.""" biomarker_id: str representative: BiomarkerInput all_members: list[BiomarkerInput] pool: Literal["healthy", "unhealthy"] total_score: float components: dict[str, float] = Field(default_factory=dict) ``` class Stage0Output(BaseModel): ```text """Output of the deterministic pre-filter.""" scored_biomarkers: list[ScoredBiomarker] canonical_groups: list[ScoredCanonicalGroup] excluded_count: int input_count: int unhealthy_count: int = 0 healthy_count: int = 0 ``` ``` ### Stage 1 Output ContextEnrichedInput, EnrichedBiomarker, DataQuality, OptimalRange, MedicationInteraction ```text Copy ``` ``` class OptimalRange(BaseModel): ```text low: float | None = None high: float | None = None source: str = "" ``` class MedicationInteraction(BaseModel): ```text medication_name: str interaction_note: str = "" ``` class EnrichedBiomarker(BaseModel): ```text """Biomarker enriched with functional ranges, body systems, medication context.""" biomarker: BiomarkerInput prefilter_score: float numeric_value: float | None = None lab_range_low: float | None = None lab_range_high: float | None = None optimal_range: OptimalRange | None = None functional_status: str = "unknown" # optimal|within_lab|outside_optimal|outside_lab|unknown body_systems: list[str] = Field(default_factory=list) medication_interactions: list[MedicationInteraction] = Field(default_factory=list) ``` class DataQuality(BaseModel): ```text total_biomarkers: int biomarkers_with_trends: int = 0 biomarkers_recent: int = 0 biomarkers_stale: int = 0 systems_covered: list[str] = Field(default_factory=list) systems_missing: list[str] = Field(default_factory=list) has_context: bool = False has_medications: bool = False ``` class ContextEnrichedInput(BaseModel): ```text """Output of Stage 1: Context Assembly.""" enriched_biomarkers: list[EnrichedBiomarker] by_system: dict[str, list[str]] data_quality: DataQuality patient_context: PatientContext | None = None ``` ``` ### Stage 2 Output DerivedMarkersOutput, DerivedMarker ```text Copy ``` ``` class DerivedMarker(BaseModel): ```text """A computed marker derived from multiple biomarkers.""" name: str value: float unit: str status: str # optimal|borderline|elevated|low optimal_range_str: str interpretation: str source_biomarker_ids: list[str] ``` class DerivedMarkersOutput(BaseModel): ```text """Output of Stage 2: Derived Markers.""" computed: list[DerivedMarker] = Field(default_factory=list) skipped: list[str] = Field(default_factory=list) ``` ``` ### Stage 3 Output PatternAnalysisOutput, ClinicalPattern, GapAnalysisItem ```text Copy ``` ``` class ClinicalPattern(BaseModel): ```text """A clinical pattern identified across multiple biomarkers.""" name: str severity: Literal["critical", "significant", "monitor"] confidence: Literal["high", "medium", "low"] narrative: str # max 80 words supporting_evidence: list[str] # measurement_ids, min 1 contradicting_evidence: list[str] root_cause_hypothesis: str # max 30 words body_systems: list[str] trajectory: Literal["worsening", "improving", "stable", "insufficient_data"] ``` class GapAnalysisItem(BaseModel): ```text """A missing lab test that would inform clinical reasoning.""" test_name: str reason: str # max 25 words related_pattern: str priority: Literal["high", "medium", "low"] ``` class PatternAnalysisOutput(BaseModel): ```text """Output of Stage 3: Pattern Recognition.""" patterns: list[ClinicalPattern] # 1-6 gap_analysis: list[GapAnalysisItem] cross_system_connections: list[str] ``` ``` ### Stage 4 Output ClinicalSummaryOutput, SummaryFinding, Recommendation, GapRecommendation ```text Copy ``` ``` class SummaryFinding(BaseModel): ```text """A key finding for the clinical summary.""" measurement_id: str display_name: str value: str severity: Literal["critical", "significant", "monitor"] trend: Literal["worsening", "improving", "stable", "insufficient_data"] context: str # max 15 words functional_note: str pattern_ref: str | None ``` class Recommendation(BaseModel): ```text """A prioritized recommendation.""" priority: int # 1-5 category: Literal["test", "lifestyle", "discuss", "monitor", "urgent"] action: str # max 30 words rationale: str # max 20 words pattern_ref: str ``` class GapRecommendation(BaseModel): ```text """A recommended test to fill a data gap.""" test_name: str reason: str # max 25 words priority: Literal["high", "medium", "low"] ``` class ClinicalSummaryOutput(BaseModel): ```text """Output of Stage 4 — everything needed to render the report.""" headline: str # max 15 words key_findings: list[SummaryFinding] # 1-10 recommendations: list[Recommendation] # 2-5 gap_recommendations: list[GapRecommendation] overall_status: Literal["attention_needed", "monitoring_recommended", "generally_healthy"] trajectory_summary: str # max 20 words patterns: list[ClinicalPattern] html: str = "" # populated by render.py, not LLM ``` ``` ## Library Dependency Map Forge-sentinel libraries used at each stage. = deterministic stage, = LLM stage. ```text Library S0 S1 S2 S3 S4 S5 chr-core chr-data chr-llm chr-html chr-styles chr-frontend chr-logging chr-document-manager chr-pdf chr-langfuse ``` ### Library Details ```text chr-core Exception hierarchy, CHRBaseSettings, PII sanitization. Transitive dependency via chr-llm. chr-data Async N1 API fetching, pagination, typed models, biomarker reference data (optimal ranges, body systems, medication markers, derived formulas, unit conversion). chr-llm LLM calls with billing, JSON parsing, token tracking. `allm_json_call()` for stages 3-4, `load_prompt()` for .md prompt files. chr-html HTML fragment/document builders. `html_document()`, severity dots, trend arrows, escaping utilities. chr-styles CSS loader (base + type + override). `get_report_css("single-page", "health-summary")`. chr-frontend Progress tracking, UI phases, user error messages. `ProgressReporter`, `UIPhase`, `BaseProgressTracker`. chr-logging Structured JSON logging, PII sanitization, context vars. `configure_logging()`, `bind_request_context()`. chr-document-manager S3/GCS upload, signed URLs, publish pipeline. `publish_report()` for HTML write + PDF render + cloud upload. chr-pdf HTML→PDF rendering, compression. `WeasyPrintRenderer` via `publish_report()`. chr-langfuse Langfuse LLM tracing (spans, cost, prompt visibility). `traced_stage()` wraps stages 3-4 LLM calls. ``` ### Not Yet Integrated ```text chr-verify Hallucination detection (name matching, timeline validation). No verification of LLM output before render. chr-checkpoint File-based stage checkpointing, crash recovery. Restarts currently re-run all 6 stages. chr-resilience Retry decorators, error classification, OTEL setup. Not yet integrated. chr-charts Biomarker chart generation (Chart.js, matplotlib). No visualizations in reports currently. ``` ## Configuration Settings model from `config.py` using `pydantic-settings`. All settings can be overridden via environment variables. Settings Model ```text Copy ``` class Settings(BaseSettings): ```json # LLM llm_model: str = "gemini-2.5-pro" llm_temperature: float = 0.1 llm_max_retries: int = 2 llm_timeout: int = 180 # Service identity (for billing metadata) service_name: str = "workflow-health-summary" # Cascade tuning prefilter_max_output: int = 100 # Progress reporting (forge mode) n1_api_base_url: str = "" n1_api_key: str = "" sync_with_cloud: bool = True # Storage (cloud upload via chr-document-manager) bucket_name: str = "" storage_provider: str = "s3" aws_region: str = "us-east-2" skip_uploads: bool = False model_config = {"env_prefix": "", "env_file": ".env", "extra": "ignore"} ``` ### Environment Variables ```text Variable Default Description `LLM_MODEL` `gemini-2.5-pro` LLM model for stages 3-4 (via LiteLLM) `LLM_TEMPERATURE` `0.1` Low temperature for consistent clinical output `LLM_MAX_RETRIES` `2` Retry count for failed LLM calls `LLM_TIMEOUT` `180` Timeout in seconds per LLM call `SERVICE_NAME` `workflow-health-summary` Service identity for billing metadata `PREFILTER_MAX_OUTPUT` `100` Max biomarker groups selected (50 unhealthy + 50 healthy) `N1_API_BASE_URL` *empty* N1 API base URL (forge mode) `N1_API_KEY` *empty* N1 API key (forge mode) `SYNC_WITH_CLOUD` `true` Enable cloud progress sync `BUCKET_NAME` *empty* S3/GCS bucket for report upload `STORAGE_PROVIDER` `s3` Cloud storage provider (s3 or gcs) `AWS_REGION` `us-east-2` AWS region for S3 `SKIP_UPLOADS` `false` Skip cloud upload (local mode) `INPUT_FILE` *none* Path to JSON patient file (local mode) `USER_ID` *none* Patient user ID (forge mode) `CHR_ID` *none* CHR session ID (forge mode) ``` ## Overview The workflow-data-analysis** generates multi-page biomarker-focused Comprehensive Health Reports (CHR) — data visualizations with AI-powered summaries but *no clinical recommendations*. It processes patient biomarker groups through 8 analysis stages: chart discovery, chart rendering, clinical clustering, biomarker summaries, SBAR narrative, pattern & follow-up analysis, timeline analysis, and deterministic HTML assembly. ```text Analysis Stages 7+ LLM Call Types Forge Libraries 20+ Feature Flags ``` **Philosophy:** This repo is a *thin orchestrator*. Chart utilities, HTML builders, retry logic, data fetching, and styling all live in forge-sentinel shared libraries. WDA only does: LLM calls, chart generation, data mapping, and pipeline orchestration. ### Run Modes ##### Local Mode CSV fixtures → charts → HTML. No API credentials required. `uv run python -m src.reportgen.main` with `USE_LOCAL_DATA=true` ##### Staging / Production N1 API fetch → charts + LLM summaries → HTML + PDF upload. Requires `USER_ID`, `CHR_ID`, `N1_API_KEY`, `OPENAI_API_KEY` ## Pipeline Flowchart 3 deterministic + 5 LLM-powered stages. Deterministic data processing bookends the LLM calls. All LLM features have deterministic fallbacks for graceful degradation. ```text S0 Chart Discovery Deterministic chr-data (EntityFetcher) • chr-charts (classification, resolve_category) S1 Chart Rendering LLM + Fallback chr-charts (validate_biomarker_value) • Chart.js config generation or PNG download S2 Smart Clustering Deterministic chr-charts (apply_smart_clustering, subgroups) • Clinical priority ordering S3 Biomarker Summaries LLM chr-llm (billing) • Per-chart trend analysis + per-category chapter summaries S4 SBAR Clinical Narrative LLM chr-llm • Situation, Background, Assessment, Recommendations S5 Pattern & Follow-Up Analysis LLM chr-llm • Cross-biomarker patterns + follow-up test recommendations S6 Timeline Analysis Deterministic + LLM Inflection point detection • LLM narrative generation • Hallucination validation S7 Render Deterministic chr-html (component builders) • chr-styles (CSS) • chr-core (PII sanitization) ``` ## Stage Details #### Chart Discovery — Fetch, Filter & Classify Fetches patient biomarker groups from the N1 API, filters by data richness, optionally classifies uncategorized biomarkers via LLM, and resolves each into a health category. Returns a prioritized list of chartable biomarkers. ```text Input `biomarkers[] + biomarkers_df` Output `List[ChartMetadata]` ``` ##### Filtering - Keep only biomarker groups with `member_count ≥ 2` (minimum for trend line) - Filter out invalid biomarker_id values (NaN, None, empty) - Validate biologically plausible values via `validate_biomarker_value()` - When count exceeds `MAX_BIOMARKERS` (1000), prioritize by: member_count DESC → health_areas count DESC → name ASC ##### Category Resolution (fallback chain) - 1. API `group_name` from patient biomarker metadata - 2. LLM classification via `classify_biomarkers_with_llm()` (optional, uses fast model) - 3. Infer from `health_areas` field - 4. Heuristic name matching (`infer_category_from_name()`) - 5. Fallback: “Uncategorized” ##### LLM Classification (Optional) - Model: `classification_model` (default: gemini-3-flash-preview) - Purpose: Batch-classify uncategorized biomarker names into health categories - Returns `ClassificationResult` with cache + token usage - Cached across calls to avoid re-classifying known biomarkers ```text chr-data chr-charts chr-llm ``` #### Chart Rendering — PNG or Chart.js Generates visual chart representations for each biomarker. Two rendering modes: download PNGs from the N1 chart API, or generate Chart.js configurations via LLM (with deterministic fallback). Chart.js mode produces interactive browser-rendered charts. ```json Input `ChartMetadata[] + biomarkers_df` Output `chart_paths{} (PNG) or chart_configs{} (Chart.js)` ``` ##### PNG Mode (default) - Download chart images from N1 chart API - Skipped entirely when Chart.js mode is enabled - Batch download with error handling per chart ##### Chart.js Mode (LLM-powered) - LLM generates complete Chart.js v4 JSON configs with reference range annotations - Deterministic fallback builds config from data when LLM unavailable or disabled - Batched with semaphore (max 5 concurrent LLM calls) - Deduplicates same-date readings by averaging (`_deduplicate_rows_by_date()`) - Validates values via `validate_biomarker_value()` — filters biologically implausible data - Fallback reference range parsing from row data when API returns null (`_extract_reference_range_from_rows()`) ##### Chart.js Output Format - Line chart with date labels (Mon YYYY format) - Green points (#22c55e) for normal values, red (#ef4444) for out-of-range - Reference range shown as shaded annotation band - Responsive with hover tooltips ```text chr-charts chr-llm chr-resilience ``` #### Smart Clustering — Clinical Priority Ordering Groups charts by health category, orders categories by clinical relevance (most abnormal biomarkers first), and assigns biomarkers to clinical sub-groups within each category. Ensures the most clinically significant data appears first in the report. ```json Input `ChartMetadata[] + biomarkers_df` Output `charts_by_category{} (ordered) + category_order[]` ``` ##### Patient Profile Analysis - `analyze_patient_profile(biomarkers_df)` — counts abnormal biomarkers per category - Categories with more out-of-range values ranked higher - Ensures most clinically relevant sections seen first ##### Category Ordering - `smart_category_order()` — sort by abnormal count descending - `sort_charts_within_category()` — prioritize within each category - `sort_singletons_within_category()` — sort table rows for singleton biomarkers ##### Sub-Group Assignment - 7 category definitions with keyword-matched sub-groups (e.g., Cardiovascular → Lipid Panel, Cardiac Markers, Blood Pressure) - `get_subgroup_for_biomarker(name, category)` — case-insensitive partial match - `group_charts_by_subgroup()` — ordered by clinical priority within category - Unmapped biomarkers fall to “Uncategorized” sub-group ```text chr-charts ``` #### Biomarker Summaries — Per-Chart & Per-Category Generates per-biomarker trend summaries and per-category chapter summaries via LLM. Chart summaries provide individual biomarker analysis while chapter summaries give a holistic view of each health category. All calls are independently toggleable and run with semaphore-based batched concurrency (max 5 concurrent). ```json Input `biomarkers_df + charts_by_category{}` Output `chart_summaries{} + chapter_summaries{}` ``` ##### Chart Summaries (`enable_chart_summaries`) - Per-biomarker trend analysis: description, measurements, clinical meaning, action - Extracts `[CONCLUSION:positive|neutral|negative]` sentiment tag - Deterministic fallback if LLM fails - Batched with semaphore (max 5 concurrent) ##### Chapter Summaries (`enable_chapter_summaries`) - 3-4 sentence analysis per health category - Covers: overall picture, key findings, patterns, clinical context - Constrained to only reference biomarkers in the category ```text chr-llm chr-resilience ``` #### SBAR Clinical Narrative Generates a structured SBAR (Situation, Background, Assessment, Recommendations) clinical narrative from the patient's biomarker data. Uses the medical communication standard adapted for health reports. All biomarker references validated against input data. ```json Input `biomarkers_df + charts_by_category{}` Output `SBARSummary (situation, background, assessment, recommendations)` ``` ##### SBAR Framework - Medical communication standard adapted for health reports ##### Structure - **S**ituation: 1-2 sentences on current health status - **B**ackground: 3-5 bullets with key vitals/labs (actual values) - **A**ssessment: 1-2 sentence interpretation of findings - **R**ecommendations: 3-5 action items starting with **bold verbs** (Discuss, Monitor, Schedule, Review) ##### Safety - All biomarker references validated against input data - No clinical diagnoses — observations only - JSON response format with structured parsing ```text chr-llm chr-resilience ``` #### Pattern & Follow-Up Analysis Discovers cross-biomarker patterns and generates follow-up test recommendations. Identifies multi-biomarker interactions, temporal trends, and cross-system connections. All biomarker names validated against input data to prevent hallucination. ```json Input `biomarkers_df + charts_by_category{}` Output `KeyPattern[] + FollowUpTest[]` ``` ##### Key Patterns (`enable_key_patterns`) - Cross-biomarker pattern discovery (max 10 patterns) - Each pattern: name, description, connected biomarkers, causes, effects, actions, priority (high/medium/low), type (concern/healthy) - All biomarker names validated against input data (hallucination prevention) ##### Category Patterns (`enable_category_patterns`) - Per-category pattern subsets for inline display within category sections ##### Follow-Up Tests (`enable_follow_up_tests`) - 3-5 recommended tests with urgency: priority / recommended / routine - Related biomarkers must exist in patient data - Each test: name, reason, related_biomarkers, urgency ```text chr-llm chr-resilience ``` #### Timeline Analysis — Inflection Detection + Narrative Detects significant biomarker changes (inflection points) over time using deterministic algorithms, then generates an LLM-powered timeline narrative. Three-layer hallucination prevention validates all LLM output against actual patient data. ```text Input `biomarkers_df` Output `HealthEvent[] + timeline_narrative: str` ``` ##### Inflection Point Detection (Deterministic) - **Status changes** (normal ↔ abnormal): Require ≥ 25% value change to avoid minor fluctuations - **Significant value changes** (same status): Require > 40% change - Values validated via `validate_biomarker_value()` before analysis - Signed `change_pct`: positive = increase, negative = decrease - Events classified: *improvement*, *deterioration*, or *significant_change* ##### Event Selection - Sort by absolute change_pct descending (most significant first) - Take top N events (default 5, configurable via `timeline_max_events`) - Re-sort chronologically for narrative coherence ##### LLM Narrative Generation - Bullet-point format: overall trajectory + individual event descriptions - Bold biomarker names and significant changes ##### Hallucination Prevention (3 layers) - `_validate_timeline_narrative()`: All biomarker name mentions must exist in actual data - `_validate_timeline_values()`: All percentage claims checked against real data (±0.5% tolerance) - `_sanitize_percentages()`: Replace hallucinated percentages with correct values from events ```text chr-charts chr-llm ``` #### Render — Deterministic HTML Converts all generated content into a complete HTML report using `chr-html`'s component library and `chr-styles`'s CSS system. No LLM-generated HTML — the presentation layer is fully deterministic. Charts embedded as base64 data URIs (PNG) or inline Chart.js canvases. All LLM output PII-sanitized before rendering. ```text Input `All S0-S6 outputs + demographics` Output `HTML string → file` ``` ##### Render Components (from chr-html) - `html_document()`: Full HTML wrapper with head, body, report-type CSS - `report_header_card()`: Title, patient age/gender, report date - `category_section()`: Health area headers with category icons - `chart_block()` / `chartjs_block()`: Chart + summary text - `chart_grid()`: Multi-column chart layout - `sbar_grid()`: SBAR display (Situation, Background, Assessment, Recommendations) - `pattern_card()`: Key pattern cards with priority/type badges - `follow_up_tests_grid()`: Follow-up test recommendation cards - `timeline()`: Timeline events and narrative - `critical_findings_strip()`: Out-of-range biomarker summary banner ##### CSS Pipeline - `html_document(body, report_type="multi-page")` - Base CSS + multi-page type CSS layered via `chr-styles` - PII sanitization via `chr-core.pii.sanitize_report_content()` ```text chr-html chr-styles chr-core ``` ## Full Prompts Chart Summary Prompt Per Chart • S3 ```text **Copy ``` Analyze the following biomarker data and provide a concise clinical summary. **Description:** Brief technical definition of what this biomarker measures **Measurements:** Trend (direction + brief explanation) followed by all data points **Clinical Meaning:** 1-2 brief points about what the values indicate **Action:** One specific sentence (action required OR "No action required") [Sample Source]: Blood/Urine/Saliva/etc [CONCLUSION:positive|neutral|negative] One-sentence summary CRITICAL: Only reference biomarkers provided in the data below. Do NOT hallucinate values, dates, or biomarker names. Chapter Summary Prompt Per Category • S3 ```text Copy ``` ``` Analyze the following biomarker data for the {category} health area. Write a comprehensive summary (3-4 sentences, max {max_words} words) covering: 1. Overall health picture for this category 2. Key findings (abnormal, borderline, or optimal values) 3. Patterns and multi-biomarker interactions 4. Clinical context and what the results suggest CRITICAL: Only reference biomarkers listed in the data below. ``` SBAR Summary Prompt Global • S4 ```text Copy ``` ``` Generate a clinical SBAR summary from the biomarker data: - SITUATION: 1-2 concise sentences on current health status - BACKGROUND: 3-5 bullets with key vitals/labs (include actual values) - ASSESSMENT: 1-2 sentence interpretation of findings - RECOMMENDATIONS: 3-5 action bullets starting with **bold verb** (Examples: **Discuss**, **Monitor**, **Schedule**, **Review**) Return ONLY valid JSON. No markdown wrapping. ``` Key Pattern Discovery Prompt Global • S5 ```text Copy ``` ``` Analyze biomarker data to discover KEY PATTERNS a physician should know: - Multi-biomarker interactions and correlations - Temporal trends across measurements - Cross-system connections CRITICAL: ONLY reference biomarkers present in the patient's actual report. Return JSON array of patterns: [{ "pattern_name": "3-8 word name", "pattern_discovered": "detailed description", "connected_biomarkers": ["Exact Biomarker Name", ...], "potential_causes": ["cause1", ...], "potential_effects": ["effect1", ...], "possible_actions": ["action1", ...], "priority": "high|medium|low", "pattern_type": "concern|healthy" }] ``` Follow-Up Tests Prompt Global • S5 ```text Copy ``` ``` Suggest 3-5 follow-up tests based on biomarker patterns. CRITICAL: ONLY reference biomarkers from the patient's actual data. For each test: { "test_name": "Test Name", "reason": "Grounded in patient's specific findings", "related_biomarkers": ["Exact Biomarker Name"], "urgency": "priority|recommended|routine" } Return ONLY valid JSON array. ``` Timeline Narrative Prompt Global • S6 ```text Copy ``` ``` CRITICAL: Response MUST be ONLY bullet points. No paragraphs. Analyze chronological biomarker events and create bullet-point summary: 1. EVERY LINE must start with "- " (markdown bullet) 2. Highlight most significant health changes 3. Identify overall trajectory (improving/stable/declining/mixed) 4. Group related biomarker changes FORMAT RULES: - EVERY line MUST start with "- " - NO paragraphs allowed - Use **bold** for biomarker names or important changes - Start with overall trajectory as first bullet Chronological events: {events_text} ``` ## Data Contracts ### Core Models ChartMetadata ```text Copy ``` ``` class ChartMetadata: ```text id: str # biomarker_id canonical_name: str description: str biomarker: str chart_type: str = "line" data_points: int = 0 units: str | None = None category: str = "Uncategorized" # resolved health category health_areas: list[str] = [] member_count: int = 0 # data points in biomarker group reference_range_min: float | None = None reference_range_max: float | None = None ``` ``` HealthEvent (Timeline) ```text Copy ``` ``` class HealthEvent: ```text date: str # "YYYY-MM" format biomarker_name: str biomarker_id: str event_type: str # "improvement" | "deterioration" | "significant_change" old_value: str new_value: str old_status: str # "normal" | "abnormal" new_status: str change_pct: float | None # Signed: positive = increase, negative = decrease reference_range: str unit: str ``` ``` Summary Dataclasses ```text Copy ``` ``` class ChartSummary: ```text biomarker_id: str canonical_name: str summary_text: str conclusion: str # "positive" | "neutral" | "negative" is_fallback: bool = False ``` class ChapterSummary: ```text category: str summary_text: str biomarker_count: int abnormal_count: int ``` class SBARSummary: ```text situation: str background: list[str] assessment: str recommendations: list[str] ``` class KeyPattern: ```text pattern_name: str pattern_discovered: str connected_biomarkers: list[str] potential_causes: list[str] potential_effects: list[str] possible_actions: list[str] priority: str # "high" | "medium" | "low" pattern_type: str # "concern" | "healthy" ``` class FollowUpTest: ```text test_name: str reason: str related_biomarkers: list[str] urgency: str # "priority" | "recommended" | "routine" ``` class ChartJSConfig(BaseModel): ```text biomarker_id: str canonical_name: str config_json: str # Complete Chart.js config as JSON is_fallback: bool = False ``` ``` ## Library Map ### Dependency Grid ```text Library S0 S1 S2 S3 S4 S5 S6 S7 chr-data chr-charts chr-llm chr-resilience chr-core chr-html chr-styles chr-frontend ``` ### Library Details ```text chr-core Exception hierarchy (`RetryableError`, `NonRetryableError`), `CHRBaseSettings`, PII sanitization for report content. chr-data Async N1 API fetching via `EntityFetcher`. Typed models: `Biomarker`, `Biomarker`, `UserProfile`. Local CSV loader for development. chr-charts LLM classification (`classify_biomarkers_with_llm`), smart clustering (`apply_smart_clustering`), sub-group definitions, value validation (`validate_biomarker_value`), reference range formatting. chr-llm LLM billing headers via `get_billing_headers()`, cost tracking (`CostTracker`), token usage recording, model pricing reference. chr-resilience Retry decorators (`@retry_api`, `@retry_llm`), error classification, progress helpers (`compute_adjusted_progress`, `trace_progress_stage`). chr-html HTML component builders: `html_document`, `chart_block`, `chartjs_block`, `sbar_grid`, `pattern_card`, `timeline`, `category_section`, `follow_up_tests_grid`. chr-styles CSS loader with report_type + override layering. Design tokens, responsive layouts, N1 brand colors. chr-logging Structured JSON logging, PII sanitization in logs, `update_log_stage()` for stage-based log context. ``` ## Configuration Settings model from `config.py` using `CHRBaseSettings` (forge). Over 20 feature flags control which analysis stages run and what appears in the report. Settings Model (key fields) ```text Copy ``` class CloudSettings(CHRBaseSettings): ```text # Charts (S0-S1) enable_charts: bool = True enable_tables: bool = False enable_llm_charts: bool = False use_deterministic_charts: bool = True max_biomarkers: int = 1000 max_llm_chart_biomarkers: int = 15 # LLM Summaries (S3-S5) enable_chart_summaries: bool = True enable_chapter_summaries: bool = True enable_key_patterns: bool = True enable_category_patterns: bool = True enable_sbar: bool = True enable_follow_up_tests: bool = True # Timeline (S6) enable_timeline_summary: bool = True # Models summary_model: str = "gemini-2.5-pro" classification_model: str = "gemini-3-flash-preview" temperature: float = 0.2 # Safety enable_pii_sanitization: bool = True enable_llm_classification: bool = True # Retry (per K8s pod, ×3 with restarts) retry_llm_max_attempts: int = 20 retry_api_max_attempts: int = 10 ``` ### Environment Variables ```text Variable Default Description `SUMMARY_MODEL` `gemini-2.5-pro` LLM model for all summary generation (S3-S6) `CLASSIFICATION_MODEL` `gemini-3-flash-preview` Fast model for biomarker categorization (S0) `TEMPERATURE` `0.2` LLM sampling temperature (lower = more deterministic) `MAX_BIOMARKERS` `1000` Maximum charts to generate (0 = unlimited) `ENABLE_LLM_CHARTS` `false` Use Chart.js instead of PNG charts (S1) `ENABLE_CHARTS` `true` Include biomarker chart images in report `ENABLE_SBAR` `true` Generate SBAR clinical summary (S4) `ENABLE_KEY_PATTERNS` `true` Cross-biomarker pattern discovery (S5) `ENABLE_FOLLOW_UP_TESTS` `true` Follow-up test recommendations (S5) `ENABLE_TIMELINE_SUMMARY` `true` Timeline narrative from inflection points (S6) `DEV_MODE` `false` Show patient biomarker IDs on charts `USE_LOCAL_DATA` `false` Use local CSV files instead of N1 API ``` ## Functional Workflow ```text 📋 ``` #### Coming Soon ```text Clinical report with recommendations. Python, LangGraph, LiteLLM. PDF output via Typst. ``` ## Generative Sequential Workflow ```text 📄 ``` #### Coming Soon Detailed narrative report with sequential LLM stages. Python + LiteLLM, PDF via LaTeX. ## Generative Langroid Workflow ```text 🤖 ``` #### Coming Soon Agent-based workflow with lifestyle guides. Python, Langroid, LiteLLM. ## Claude Code Workflow ```text 💻 ``` #### Coming Soon CLI-driven report generation via Claude Code agent. Python. ## Overview The workflow-supplements-optimization** generates a 1-page supplement protocol via a 7-stage cascade. It analyzes biomarker data through deterministic pre-filtering, context enrichment, derived marker computation, and nutrient gap matching, then uses two LLM calls to select supplements with specific dosing and build a daily protocol. The final output is deterministically rendered to HTML via `chr-html`. ```text Pipeline Stages LLM Calls Forge Libraries 20+ Pydantic Models ``` **Philosophy:** Deterministic first, LLM second. Safety screening (drug-supplement interactions, contraindications) is always deterministic — never delegated to an LLM. The LLM only selects from pre-validated candidates and generates patient-friendly text. ### Run Modes ##### Local Mode JSON file → cascade → HTML. No credentials required. `INPUT_FILE=tests/fixtures/patient.json uv run python bin/run.py` ##### Forge Mode API fetch → cascade → HTML + PDF, with billing and cloud upload. Requires `USER_ID`, `CHR_ID`, LiteLLM and N1 API credentials. ## Pipeline Flowchart 4 deterministic stages + 2 LLM calls + 1 render. Stages 0–2.5 are fully deterministic (scoring, enrichment, derived markers, nutrient gap matching). Stages 3–4 use LLM for supplement selection and protocol assembly. Stage 5 renders deterministically. ```text S0 Pre-filter Deterministic Scoring: deviation, trend, recency, density, OOR consistency • Top 50 unhealthy + 50 healthy S1 Context Assembly Deterministic chr-data (optimal ranges, body systems, medication interactions, unit conversion) S2 Derived Markers Deterministic chr-data (DERIVED_REGISTRY) • HOMA-IR, eGFR, LDL:HDL, TG:HDL ratios S2.5 Nutrient Gap Matching Deterministic 41 biomarker→nutrient mappings • ~30 drug-supplement + supplement-supplement interaction rules S3 Supplement Analysis LLM chr-llm (allm_json_call, load_prompt) • Selects 3-8 from pre-validated candidates • Post-LLM validation + hallucination detection S4 Protocol Assembly LLM chr-llm (allm_json_call, load_prompt) • Daily schedule (4 slots) • Safety notes • Retest timeline S5 Render Deterministic chr-html (html_document) • chr-styles (single-page + supplements override) • chr-pdf (WeasyPrint) ``` ## Stage Details #### Pre-filter — Deterministic Scoring Groups biomarkers by biomarker ID, scores each group with 5 components (max 80 pts), then selects top 50 unhealthy + top 50 healthy biomarker groups. Severely abnormal groups are always included. Each selected group carries its full measurement history. ```text Input `dict[str, BiomarkerInput]` Output `Stage0Output` ``` ##### Unhealthy Group Scoring (max 80) - **worst_deviation (0-30):** Highest deviation across all measurements vs reference midpoint - **temporal_trend (0-20):** Linear regression slope across full history (3+ data points required) - **recency (0-10):** ≤2yr = 10, ≤5yr = 5, older = 0 - **data_richness (0-10):** ≥5 pts = 10, ≥3 = 7, 2 = 4, 1 = 1 - **oor_consistency (0-10):** Percentage of measurements that are out-of-range ##### Healthy Group Scoring (max 80) - **clinical_significance (0-30):** High-value markers (glucose, HbA1c, TSH, lipids) = 30; others = 10 - **stability (0-20):** Fraction of measurements that are normal × 20 - **recency (0-10):** Same as unhealthy - **data_richness (0-10):** Same as unhealthy - **system_coverage (0-10):** Bonus for underrepresented body systems ##### Hard Rules - Severely abnormal groups (worst_deviation ≥ 20) always included - Qualitative biomarkers excluded - If one pool has fewer than 50, overflow fills from the other ##### Key References - `_HIGH_SIGNIFICANCE_MARKERS` — clinically important markers list - `_TEST_NAME_ALIASES` — ~50 biomarker ID normalization rules - `get_body_systems()` — body system classification from chr-data ```text chr-data ``` #### Context Assembly — Functional Enrichment Enriches scored biomarkers with functional/optimal ranges, body system classification, medication interaction context, and data quality assessment. All lookups come from `chr-data`'s reference module. ```text Input `Stage0Output + PatientContext?` Output `ContextEnrichedInput` ``` ##### Functional Classification - **outside_lab:** Outside standard lab reference range (highest priority) - **outside_optimal:** Within lab range but outside functional/optimal range - **within_optimal:** Within both lab and functional ranges ##### Key Operations - Optimal range lookup via `get_optimal_range(canonical)` - Unit conversion to conventional units via `convert_to_conventional()` - Medication interaction detection via `get_affected_markers(med_name)` - Body system mapping via `get_body_systems(canonical)` - Data quality assessment (coverage, staleness, body systems) ```text chr-data ``` #### Derived Markers — Computed Ratios & Indices Computes clinically meaningful derived markers from available biomarkers. Uses `DERIVED_REGISTRY` from `chr-data` which contains formulas, parameter mappings, and interpretation logic. ```text Input `ContextEnrichedInput + PatientContext?` Output `DerivedMarkersOutput` ``` ##### Example Derived Markers - **HOMA-IR:** (Glucose × Insulin) / 405 — insulin resistance - **eGFR:** CKD-EPI formula (requires age + sex) — kidney function - **LDL:HDL Ratio:** LDL / HDL — cardiovascular risk proxy - **TG:HDL Ratio:** Triglycerides / HDL — insulin resistance proxy ##### Key Operations - Builds biomarker_id → numeric_value lookup (highest prefilter score wins) - All values converted to conventional units before computation - Demographics-dependent markers (eGFR) require age and sex from PatientContext - Skipped markers tracked separately for transparency ```text chr-data 2.5 ``` #### Nutrient Gap Matching — Deterministic Candidate Selection Maps enriched biomarkers to supplement candidates using 41 biomarker-to-nutrient rules, then screens all candidates against drug-supplement and supplement-supplement interaction databases. Contraindicated supplements are blocked before any LLM call. ```text Input `ContextEnrichedInput + DerivedMarkersOutput + PatientContext` Output `list[NutrientCandidate] + list[InteractionFlag]` ``` ##### Biomarker → Nutrient Mapping - 41 mappings in `BIOMARKER_NUTRIENT_MAP` - Direct markers (vitamin D, B12, ferritin) map 1:1 - Indirect markers (homocysteine, CRP, TSH) map to multiple nutrients with relevance + evidence scores ##### Severity Classification - **severe:** ≥30% deviation from optimal midpoint - **moderate:** 10-30% deviation - **mild:** <10% deviation ##### Interaction Screening - **~30 drug-supplement rules:** Levothyroxine+Iron, Metformin+B12, Warfarin+Omega-3, etc. - **Supplement-supplement rules:** Iron+Calcium absorption, Zinc+Copper competition, etc. - **Blocking logic:** `contraindicated` → blocked, diagnosis contraindications → blocked - **Caution flags:** flagged with spacing/timing instructions, not blocked ```text reference/nutrient_gaps.py reference/supplement_specs.py reference/interactions.py ``` #### Supplement Analysis — LLM Call #1 LLM selects 3-8 supplements from pre-validated candidates, assigns specific doses, and writes clinical rationales. Extensive post-LLM validation catches hallucinated supplements, enforces dose caps, and ensures safety. ```text Input `NutrientCandidates + ContextEnrichedInput + DerivedMarkersOutput + PatientContext` Output `SupplementGapAnalysisOutput` ``` ##### LLM Selection - Selects 3-8 supplements from pre-validated candidates that passed S2.5 screening - Assigns specific doses based on biomarker severity, patient context, and supplement specs - Writes per-supplement clinical rationale ##### Post-LLM Validation - **Name resolution:** exact match → fuzzy match → alias lookup against SUPPLEMENT_SPECS - **Blocking check:** re-verify against interaction database (LLM may hallucinate cleared supplements) - **Deduplication:** merge duplicate supplement entries - **Dose capping:** if LLM proposes dose above max_daily, cap to max_daily - **Timing sanitization:** normalize timing values to known slots - **Hallucination detection:** verify supplement names against ground truth in SUPPLEMENT_SPECS ##### Fallback - If post-validation produces 0 valid supplements: pick top candidate from highest-severity gap deterministically ```text chr-llm chr-langfuse chr-verify ``` #### Protocol Assembly — LLM Call #2 LLM builds the daily protocol: a 4-slot schedule, patient-friendly safety notes, retest timeline, and headline + summary. Extensive post-LLM validation backfills missing data and resolves timing conflicts deterministically. ```text Input `SupplementGapAnalysisOutput + InteractionFlags + PatientContext` Output `SupplementProtocolOutput` ``` ##### Daily Schedule - 4 time slots: morning, afternoon, evening, bedtime - Per-slot instructions (e.g. “take with fat for D3 absorption”) ##### Post-LLM Validation - **Auto-assign:** supplements missing from schedule are assigned to a slot based on supplement_specs timing - **Conflict resolution:** Iron + Calcium never in same slot (absorption interference) - **Safety note backfill:** interaction flags from S2.5 are added as safety notes if LLM omitted them - **Retest backfill:** severe-gap biomarkers get retest entries if LLM omitted them - **Headline hallucination check:** validated against supplement data; deterministic fallback if hallucinated ##### Overall Status (Deterministic) - **actionable_gaps:** any severe gaps detected - **minor_optimization:** moderate gaps only - **no_significant_gaps:** no deficiencies found - Status is always determined deterministically from gap data, never by LLM ```text chr-llm chr-langfuse chr-verify ``` #### Render — Deterministic HTML Converts `SupplementProtocolOutput` to a complete HTML document using `chr-html`'s component library and `chr-styles`'s CSS system. No LLM-generated HTML — the presentation layer is fully deterministic. ```text Input `SupplementProtocolOutput + patient_name + report_date` Output `HTML string → PDF via chr-pdf` ``` ##### HTML Sections - **Header:** Patient name, date, overall status badge - **Headline + summary:** Protocol overview from S4 - **Daily schedule (hero):** 4 time-slot cards (morning/afternoon/evening/bedtime) with per-slot instructions - **Supplement theme cards:** Individual supplement cards with dose, form, rationale, retest - **Interaction warnings:** Amber callouts for flagged interactions - **Safety notes:** base .callout (amber for important, neutral for informational) - **Retest timeline:** base .data-table with biomarker, weeks, reason - **Disclaimer + footer:** base .report-disclaimer + .report-footer ##### CSS Pipeline - `html_document(body, report_type="single-page", override="supplements")` - Base CSS + single-page type CSS + supplements override CSS layered - PDF rendering via WeasyPrint through `chr-pdf` ##### Edge Case: No Gaps - When pipeline finds no deficiencies: renders a simplified “No Significant Gaps” variant ```text chr-html chr-styles chr-pdf chr-document-manager ``` ## Full Prompts Both prompts use a system/human split loaded via `chr-llm`'s `load_prompt()`. The system message is everything above the `---` separator, the human template is everything below. ### Stage 3 — Supplement Gap Analysis Instructs the LLM to select 3-8 supplements from pre-validated candidates, assign specific doses, and write clinical rationales for each selection. Prompt template: `src/supplements_optimization/prompts/supplement_gap_analysis.md` ### Stage 4 — Protocol Assembly Instructs the LLM to build a daily schedule (4 time slots), write patient-friendly safety notes, set a retest timeline, and compose the headline and summary. Prompt template: `src/supplements_optimization/prompts/protocol_assembly.md` ## Data Contracts All Pydantic v2 models live in `src/supplements_optimization/cascade/models.py`. Models are grouped by the stage that produces them. ModelStagePurposeKey Fields ```text `BiomarkerInput`InputRaw biomarker measurementtest_name, value, unit, reference_range, biomarker_id `PatientContext`InputPatient demographics + medicationspatient_name, age, gender, medications, diagnoses `ScoredBiomarkerGroup`S0Biomarker group with scoringbiomarker_id, pool, total_score, components `Stage0Output`S0Pre-filter resultscanonical_groups, excluded_count, unhealthy_count, healthy_count `EnrichedBiomarker`S1Biomarker with functional contextfunctional_status, optimal_range, body_systems, medication_interactions `ContextEnrichedInput`S1Full enriched datasetenriched_biomarkers, by_system, data_quality `DerivedMarker`S2Computed ratio/indexname, value, unit, status, interpretation, source_biomarker_ids `DerivedMarkersOutput`S2All derived markerscomputed, skipped `NutrientGapMatch`S2.5Biomarker → nutrient matchbiomarker_id, nutrient, severity, deviation_pct `NutrientCandidate`S2.5Pre-validated supplement candidatesupplement_name, gap_matches, priority_score, safety_status `InteractionFlag`S2.5Drug/supplement interactionsupplement, interacts_with, severity, mechanism `PrioritizedSupplement`S3LLM-selected supplementrank, supplement_name, specific_dose, form, rationale, retest_weeks `SupplementGapAnalysisOutput`S3Analysis resultspriority_supplements, validation_notes `DailyScheduleSlot`S4Time slot in scheduletime_of_day, supplements, instructions `SafetyNote`S4Patient safety notecategory, severity, note `RetestItem`S4Retest recommendationbiomarker_name, retest_weeks, reason `SupplementProtocolOutput`S4Final protocol (render input)headline, protocol_summary, overall_status, daily_schedule, safety_notes, retest_timeline ``` ## Library Dependency Map Forge-sentinel libraries used at each stage. = deterministic stage, = LLM stage. ```text Library S0 S1 S2 S2.5 S3 S4 S5 chr-core chr-data chr-llm chr-html chr-styles chr-frontend chr-logging chr-document-manager chr-pdf chr-langfuse chr-verify chr-resilience ``` ### Library Details ```text chr-core Exception hierarchy, CHRBaseSettings, PII sanitization. Transitive dependency via chr-llm. chr-data Async N1 API fetching, pagination, typed models, biomarker reference data (optimal ranges, body systems, medication markers, derived formulas, unit conversion). chr-llm LLM calls with billing, JSON parsing, token tracking. `allm_json_call()` for stages 3-4, `load_prompt()` for .md prompt files. chr-html HTML fragment/document builders. `html_document()`, supplement theme cards, schedule slots, escaping utilities. chr-styles CSS loader (base + type + override). `get_report_css("single-page", "supplements")`. chr-frontend Progress tracking, UI phases, user error messages. `ProgressReporter`, `UIPhase`, `BaseProgressTracker`. chr-logging Structured JSON logging, PII sanitization, context vars. `configure_logging()`, `bind_request_context()`. chr-document-manager S3/GCS upload, signed URLs, publish pipeline. `publish_report()` for HTML write + PDF render + cloud upload. chr-pdf HTML→PDF rendering, compression. `WeasyPrintRenderer` via `publish_report()`. chr-langfuse Langfuse LLM tracing (spans, cost, prompt visibility). `traced_stage()` wraps stages 3-4 LLM calls. chr-verify Hallucination detection (name matching, supplement validation). Post-LLM verification in stages 3-4. chr-resilience Retry decorators, error classification, OTEL setup. LLM call retries in stages 3-4. ``` ## Configuration Settings model from `config.py` using `pydantic-settings`. All settings can be overridden via environment variables. Settings Model ```text **Copy ``` class Settings(BaseSettings): ```json # LLM llm_model: str = "gemini-2.5-pro" llm_temperature: float = 0.1 llm_max_retries: int = 2 llm_timeout: int = 180 # Service identity (for billing metadata) service_name: str = "workflow-supplements-optimization" # Cascade tuning prefilter_max_output: int = 100 max_supplements: int = 8 deidentify_pii: bool = True # Progress reporting (forge mode) n1_api_base_url: str = "" n1_api_key: str = "" sync_with_cloud: bool = True # Storage (cloud upload via chr-document-manager) bucket_name: str = "" storage_provider: str = "s3" aws_region: str = "us-east-2" skip_uploads: bool = False model_config = {"env_prefix": "", "env_file": ".env", "extra": "ignore"} ``` ### Environment Variables ```text Variable Default Description `LLM_MODEL` `gemini-2.5-pro` LLM model for stages 3-4 (via LiteLLM) `LLM_TEMPERATURE` `0.1` Low temperature for consistent dosing `LLM_MAX_RETRIES` `2` Retry count for failed LLM calls `LLM_TIMEOUT` `180` Timeout in seconds per LLM call `SERVICE_NAME` `workflow-supplements-optimization` Service identity for billing metadata `PREFILTER_MAX_OUTPUT` `100` Max biomarker groups selected (50 unhealthy + 50 healthy) `MAX_SUPPLEMENTS` `8` Maximum supplements in final protocol (3-8 range) `DEIDENTIFY_PII` `true` Sanitize PII before LLM calls `N1_API_BASE_URL` *empty* N1 API base URL (forge mode) `N1_API_KEY` *empty* N1 API key (forge mode) `SYNC_WITH_CLOUD` `true` Enable cloud progress sync `BUCKET_NAME` *empty* S3/GCS bucket for report upload `STORAGE_PROVIDER` `s3` Cloud storage provider (s3 or gcs) `AWS_REGION` `us-east-2` AWS region for S3 `SKIP_UPLOADS` `false` Skip cloud upload (local mode) `INPUT_FILE` *none* Path to JSON patient file (local mode) `USER_ID` *none* Patient user ID (forge mode) `CHR_ID` *none* CHR session ID (forge mode) ``` ## Overview The workflow-predictive-aging** generates a multi-page predictive health report via a 7-stage integrative medicine cascade with 10/20/30-year biomarker projections. It extends the health-summary pipeline with deterministic trend analysis (Stage 2.5), predictive LLM prompts, and projection chart generation (Stage 4.5). The final output is a multi-page HTML/PDF with inline projection charts. ```text Pipeline Stages LLM Calls Forge Libraries 25+ Pydantic Models ``` **Philosophy:** Same as health-summary — *pure analysis logic*. The key differentiator is Stage 2.5 (deterministic projections via linear regression + population norm blending) and Stage 4.5 (chart generation via `chr-charts`). Dual-mode: legacy prompts (single snapshot) or predictive prompts (trajectory-focused). ### Run Modes ##### Local Mode JSON file → cascade → HTML + PDF. No credentials required. `INPUT_FILE=tests/fixtures/patient-small.json uv run python bin/run.py` ##### Forge Mode API fetch → cascade → HTML + PDF, with billing, PII de-identification, and cloud upload. Requires `USER_ID`, `CHR_ID`, LiteLLM and N1 API credentials. ## Pipeline Flowchart 4 deterministic stages + 2 LLM calls + 1 chart generation + 1 render. Stages 2.5 and 4.5 are unique to predictive-aging (not present in health-summary). The LLM receives projections and reasons about aging trajectories. ```text S0 Pre-filter Deterministic chr-data (reference) • Pure scoring: deviation, trend, recency, density S1 Context Assembly Deterministic chr-data (reference: optimal ranges, body systems, meds, unit conversion) S2 Derived Markers Deterministic chr-data (DERIVED_REGISTRY, unit conversion) S2.5 Projections Deterministic Linear regression + population norm blending (NHANES/Framingham/CKD-EPI) • 16 clinical thresholds S3 Pattern Recognition LLM chr-llm (allm_json_call, load_prompt) • chr-langfuse (tracing) • chr-resilience (retry) • chr-verify S4 Clinical Summary LLM chr-llm (allm_json_call, load_prompt) • chr-langfuse (tracing) • chr-resilience (retry) • chr-verify S4.5 Chart Generation Deterministic chr-charts (matplotlib → base64 PNG) • Priority scoring: threshold crossings × 10 + severity S5 Render Deterministic chr-html (multi-page) • chr-styles (CSS) • chr-pdf (WeasyPrint) ``` ## Stage Details #### Pre-filter — Deterministic Scoring Groups biomarkers by biomarker ID, scores each group, then selects top 50 unhealthy + top 50 healthy biomarker groups (100 total). Same logic as health-summary. ```text Input `dict[str, BiomarkerInput]` Output `Stage0Output` ``` ##### Scoring (same as health-summary) - **worst_deviation (30):** Highest deviation across all measurements vs reference midpoint - **trend (20):** Linear regression slope across full history (3+ data points required) - **recency (10):** ≤2yr = 10, ≤5yr = 5, older = 0 - **data_richness (10):** ≥5 pts = 10, ≥3 = 7, 2 = 4, 1 = 1 - **oor_consistency (10):** Percentage of measurements that are abnormal ```text chr-data ``` #### Context Assembly — Functional Enrichment Enriches scored biomarkers with functional/optimal ranges, body system classification, and medication interaction context. Same logic as health-summary. ```text Input `Stage0Output + PatientContext?` Output `ContextEnrichedInput` ``` ##### Key Operations - Unit conversion to conventional units via `convert_to_conventional()` - Optimal range lookup via `get_optimal_range(canonical)` - Body system mapping via `get_body_systems(canonical)` - Medication interaction detection via `get_affected_markers(med_name)` ```text chr-data ``` #### Derived Markers — Computed Ratios & Indices Computes clinically meaningful derived markers (HOMA-IR, TG/HDL, eGFR, NLR) from available biomarkers using `DERIVED_REGISTRY`. Same logic as health-summary. ```text Input `ContextEnrichedInput + PatientContext?` Output `DerivedMarkersOutput` ``` ##### Example Derived Markers - **HOMA-IR:** (Glucose × Insulin) / 405 — insulin resistance - **TG/HDL Ratio:** Triglycerides / HDL — insulin resistance proxy - **eGFR:** CKD-EPI formula (requires age + sex) — kidney function - **NLR:** Neutrophils / Lymphocytes — inflammation marker ```text chr-data 2.5 ``` #### Projections — Trend Analysis & 10/20/30-Year Forecasting Fully deterministic stage unique to predictive-aging. Computes blended trends (patient history + population norms) and projects each biomarker 10, 20, and 30 years into the future. Checks 16 clinical thresholds for disease-onset crossings. ```text Input `ContextEnrichedInput + DerivedMarkersOutput` Output `ProjectionOutput` ``` ##### Blending Weights (Patient vs Population) - **5+ data points:** 90% patient / 10% population - **3-4 data points:** 70% patient / 30% population - **2 data points:** 50% patient / 50% population - **1 data point:** 20% patient / 80% population - **0 data points:** 0% patient / 100% population norm ##### Population Norm Sources - NHANES age-stratified means (general biomarkers) - Framingham equations (cardiovascular markers) - CKD-EPI age-related decline (kidney function) ##### Clinical Thresholds (16 rules) - **Glucose:** ≥100 pre-diabetic, ≥126 diabetic - **HbA1c:** ≥5.7% pre-diabetic, ≥6.5% diabetic - **eGFR:** <60 CKD Stage 3, <30 CKD Stage 4 - **LDL:** ≥160 high, ≥190 very high - **Triglycerides:** ≥200 high, ≥500 very high - **TSH:** >4.5 hypothyroid, <0.4 hyperthyroid - **ALT:** >56 elevated - **Creatinine:** >1.3 elevated - **Hemoglobin:** <12 anemia ##### Key Operations - Linear regression on patient measurement history (slope units/year, r²) - Blended slope = (patient_weight × patient_slope) + (pop_weight × pop_slope) - projected_value = current + blended_slope × years - Threshold crossing detection: current below threshold → projected above (or vice versa) - System-level aggregation: trajectory = "declining" if risk worsens 10y→30y - Max 50 biomarkers projected (configurable via `projection_max_biomarkers`) ```text chr-data ``` #### Pattern Recognition — LLM Call #1 LLM identifies clinical patterns. In **predictive mode**, it uses projections to assess aging acceleration, threshold crossings, and projected severity at 10/20/30 years. Outputs `PredictivePatternOutput` with `AgingRiskAssessment` per pattern and an `aging_acceleration_score` (0–100). In **legacy mode**, it produces a standard single-snapshot `PatternAnalysisOutput`. ```text Input (predictive) `ContextEnrichedInput + DerivedMarkersOutput + ProjectionOutput` Output (predictive) `PredictivePatternOutput` ``` ##### Predictive Mode Additions - Per-pattern `AgingRiskAssessment`: risk narratives at 10y, 20y, 30y + acceleration/deceleration factors - Projected severity at each horizon (critical/significant/monitor) - Focus on trajectories over snapshots: where is each biomarker heading? - Cross-system compounding: multiplicative, not additive risks - `aging_acceleration_score` (0 = protective, 50 = on track, 100 = accelerated) ##### Post-LLM Validation - Filter evidence IDs to only those present in input data - Skip patterns with zero valid supporting evidence - Enforce 1–4 pattern count - Hallucination verification via `chr-verify` ```text chr-llm chr-langfuse chr-resilience chr-verify ``` #### Clinical Summary — LLM Call #2 LLM produces the final structured summary. In **predictive mode**, findings include projected values at 10/20/30 years, threshold alerts, and recommendations carry `time_sensitivity`. Outputs `PredictiveSummaryOutput` with `aging_outlook` (accelerated/on_track/favorable). In **legacy mode**, produces standard `ClinicalSummaryOutput`. ```text Input (predictive) `PredictivePatternOutput + ContextEnrichedInput + DerivedMarkersOutput + ProjectionOutput` Output (predictive) `PredictiveSummaryOutput` ``` ##### Predictive Mode Additions - `aging_outlook`: accelerated | on_track | favorable - Findings (1–8) include `projected_10y`, `projected_20y`, `projected_30y` values - `threshold_alert` when clinical threshold crossing is projected - Recommendations (1–5) include `time_sensitivity`: immediate | within_1y | within_5y | long_term - `impact_description` per recommendation (how it affects trajectory) - `aging_acceleration_score` (0–100) echoed from S3 - Headline focuses on projected trajectory, not current snapshot ##### Post-LLM Validation - Filter findings to valid measurement_ids only - Cap findings: min 1, max 8 - Cap recommendations: min 1, max 5 - Hallucination verification via `chr-verify` - Deterministic override: overall_status based on threshold crossings within 10y → attention_needed ```text chr-llm chr-langfuse chr-resilience chr-verify 4.5 ``` #### Chart Generation — Projection Visualizations Fully deterministic stage unique to predictive-aging. Selects key biomarkers for visualization via priority scoring, then renders projection charts using `chr-charts` (matplotlib → base64 PNG). Two chart types: finding charts (Page 1) and system charts (Page 2). ```text Input `PredictiveSummaryOutput + ProjectionOutput + ContextEnrichedInput` Output `list[ProjectionChart]` ``` ##### Finding Charts (Page 1) — max 3 - Priority: findings with `threshold_alert` or severity ≥ significant - Score = threshold_crossings × 10 + severity_weight - Shows: historical data points + trend line + projected curve + threshold lines ##### System Charts (Page 2) — max 4 - One chart per declining system (trajectory = "declining") - Picks worst biomarker per system: most threshold crossings, then highest risk at 30y - Shows: historical data + projection + lab/optimal range bands + threshold lines ##### Render Pipeline - Patient measurement history → (dates_as_fractional_years, values) - chr-charts matplotlib renderer → base64 PNG - Output: `ProjectionChart(biomarker_id, display_name, base64_png)` ```text chr-charts chr-data ``` #### Render — Multi-Page HTML Converts `PredictiveSummaryOutput` + charts to a multi-page HTML document using `chr-html` and `chr-styles`. No LLM-generated HTML — the presentation layer is fully deterministic. ```text Input `PredictiveSummaryOutput + list[ProjectionChart]` Output `HTML string → PDF via chr-pdf` ``` ##### Page Layout - **Page 1 (Executive):** Headline, aging outlook, aging acceleration score, key findings with projected values, up to 3 finding charts - **Page 2 (System Projections):** System-level trajectory summaries, up to 4 system charts, threshold crossing alerts - **Page 3+ (Details):** Clinical patterns with aging risk assessments, recommendations with time sensitivity, gap recommendations ##### CSS Pipeline - `html_document(body, report_type="multi-page", override="predictive-aging")` - Base CSS + multi-page type CSS + predictive-aging override CSS layered - Inline chart images (base64 PNG — no external dependencies) - PDF rendering via WeasyPrint through `chr-pdf` ```text chr-html chr-styles chr-pdf chr-document-manager ``` ## Full Prompts Four prompts in dual-mode configuration: predictive (with projections) and legacy (snapshot only). Loaded via `chr-llm`'s `load_prompt()`. System message above `---`, human template below. Template variables in amber. ```text predictive_patterns.md — Stage 3 Predictive Mode (aging trajectories) S3 • LLM **Copy ``` You are an integrative medicine physician analyzing biomarker data with a focus on aging acceleration. Your task is to identify clinical patterns through the lens of trajectories, threshold crossings, and projected health outcomes at 10, 20, and 30 years. Always respond with valid JSON only. No markdown, no explanation, just JSON. --- ## Predictive Pattern Analysis You are analyzing biomarker data enriched with projections for a predictive aging report. The biomarkers have been enriched with functional/optimal ranges, organized by body system, and projected forward using trend analysis. Derived markers and clinical thresholds have been pre-computed. ### How to Think About This 1. **Focus on trajectories, not snapshots.** A glucose of 95 matters less than a glucose trending from 85 to 95 over 5 years with a projected crossing of 100 mg/dL in 8 years. 2. **Use projections as the anchor.** Where is each biomarker heading? Which clinical thresholds will be crossed? 3. **Identify aging acceleration.** Which patterns make biological aging faster than chronological aging? 4. **Consider threshold crossings.** A projected crossing from "normal" to "pre-diabetic" in 10 years is clinically actionable NOW. 5. **Assess risk at each horizon.** Current severity may be "monitor" but projected 20-year severity may be "critical". 6. **Cross-system compounding.** Insulin resistance + declining kidney function + rising inflammation is multiplicative, not additive. ### Input Data #### Enriched Biomarkers (with functional ranges and body systems) {{enriched_biomarkers_json}} #### Derived Markers (computed ratios and indices) {{derived_markers_json}} #### Biomarker Projections (10/20/30-year trend analysis) {{projections_json}} #### System Projection Summaries {{system_projections_json}} {{context_section}} #### Data Quality Summary {{data_quality_json}} ### Rules 1. Identify 1-4 clinical patterns focused on aging acceleration. 2. For each pattern, provide: - Root cause hypothesis (max 30 words) - Supporting evidence (measurement_ids from input) - Current trajectory AND projected severity at 10y, 20y, 30y - Aging risk assessment: risk narratives at each horizon, acceleration and deceleration factors - Affected body systems 3. Include gap analysis: what tests would confirm or refute projections? 4. Note cross-system connections with compounding risk. 5. Use ONLY measurement_ids from input data. 6. Provide an aging_acceleration_score (0-100): 0 = protective, 50 = on track for age, 100 = maximally accelerated. ### Output Format Return a JSON object: ``` { "patterns": [ ```json { "name": "<3-8 words>", "severity": "critical" | "significant" | "monitor", "confidence": "high" | "medium" | "low", "narrative": "", "supporting_evidence": ["", ...], "contradicting_evidence": ["", ...], "root_cause_hypothesis": "", "body_systems": ["", ...], "trajectory": "worsening" | "improving" | "stable" | "insufficient_data", "projected_severity_10y": "critical" | "significant" | "monitor", "projected_severity_20y": "critical" | "significant" | "monitor", "projected_severity_30y": "critical" | "significant" | "monitor", "aging_risk": { "risk_10y": "", "risk_20y": "", "risk_30y": "", "acceleration_factors": ["", ...], "deceleration_factors": ["", ...] } } ``` ], "gap_analysis": [ ```json { "test_name": "", "reason": "", "related_pattern": "", "priority": "high" | "medium" | "low" } ``` ], "cross_system_connections": ["", ...], "aging_acceleration_score": 0-100 } ``` ```text predictive_summary.md — Stage 4 Predictive Mode (multi-page, projected values) S4 • LLM Copy ``` ``` You are writing the final content for a multi-page predictive aging health report. You have clinical patterns with aging risk assessments, enriched biomarkers, derived markers, and 10/20/30-year projections. Produce the summary content for a report focused on projected health trajectory. Always respond with valid JSON only. No markdown, no explanation, just JSON. --- ## Predictive Summary Generation You are producing the final structured output for a predictive aging report. This report spans multiple pages and focuses on WHERE the patient's health is heading, not just where it is now. ### Input Data #### Clinical Patterns (with aging risk assessments) {{patterns_json}} #### Enriched Biomarkers {{enriched_biomarkers_json}} #### Derived Markers {{derived_markers_json}} #### Biomarker Projections {{projections_json}} {{context_section}} ### Rules #### Headline - Max 15 words. Focus on PROJECTED trajectory. - Example: "Metabolic markers project diabetes risk within 10 years without intervention" #### Aging Outlook - "accelerated": aging faster than chronological age (score > 65) - "on_track": aging at expected rate (score 35-65) - "favorable": aging slower than expected (score < 35) #### Key Findings (1-8) - Include current value AND projected values at 10y, 20y, 30y (from deterministic projections) - Include threshold_alert when a clinical threshold crossing is projected - Do NOT invent projected values — use only from projections input - Prioritize findings with threshold crossings and high severity #### Recommendations (1-5) - Include time_sensitivity: "immediate" | "within_1y" | "within_5y" | "long_term" - Include impact_description (max 25 words): how this affects the aging trajectory - CRITICAL: Do NOT suggest medications the patient is already taking #### Healthy Highlights (0-4) - Biomarkers with favorable projected trajectories #### Overall Status - "attention_needed": threshold crossings within 10y - "monitoring_recommended": threshold crossings within 20y - "generally_healthy": no threshold crossings or only 30y+ #### Trajectory Summary - Max 25 words. Overall aging trajectory summary. ### Output Format Return a JSON object: ``` { "headline": "", "aging_outlook": "accelerated" | "on_track" | "favorable", "aging_acceleration_score": 0-100, "key_findings": [ ```json { "measurement_id": "", "display_name": "", "value": "", "severity": "critical" | "significant" | "monitor", "trend": "worsening" | "improving" | "stable" | "insufficient_data", "context": "", "functional_note": "", "pattern_ref": "", "projected_10y": "", "projected_20y": "", "projected_30y": "", "threshold_alert": "" } ``` ], "healthy_highlights": [ ```json { "display_name": "", "value": "", "note": "" } ``` ], "recommendations": [ ```json { "priority": 1-5, "category": "test" | "lifestyle" | "discuss" | "monitor" | "urgent", "action": "", "rationale": "", "pattern_ref": "", "time_sensitivity": "immediate" | "within_1y" | "within_5y" | "long_term", "impact_description": "" } ``` ], "gap_recommendations": [ ```json { "test_name": "", "reason": "", "priority": "high" | "medium" | "low" } ``` ], "overall_status": "attention_needed" | "monitoring_recommended" | "generally_healthy", "trajectory_summary": "" } ``` ``` ```text pattern_recognition.md — Stage 3 Legacy Mode (single snapshot, no projections) S3 • LLM Copy ``` ``` You are an integrative medicine physician analyzing biomarker data. Your task is to identify clinical patterns — connections across biomarkers that reveal underlying health stories. Think like a functional medicine doctor: look at the whole picture, ask "why", and identify what's missing. Always respond with valid JSON only. No markdown, no explanation, just JSON. --- ## Clinical Pattern Analysis You are analyzing biomarker data for a patient health summary. The biomarkers have been enriched with functional/optimal ranges (tighter than standard lab ranges) and organized by body system. Derived markers (ratios, indices) have been computed where data permits. ### How to Think About This 1. **Look for clusters, not isolated values.** Three markers in the same direction across two systems tells a story. 2. **Use functional ranges first.** A glucose of 95 is "normal" by lab standards but already suboptimal functionally. 3. **Consider root causes.** Elevated glucose + elevated triglycerides + low HDL isn't three problems — it's insulin resistance. 4. **Note what's missing.** Elevated inflammatory markers but no thyroid panel is a gap worth noting. 5. **Assess trajectory.** A value worsening over 3 data points matters more than a single snapshot. ### Input Data #### Enriched Biomarkers {{enriched_biomarkers_json}} #### Derived Markers {{derived_markers_json}} {{context_section}} #### Data Quality Summary {{data_quality_json}} ### Rules 1. Identify 1-4 clinical patterns. Each must reference ≥1 biomarker by measurement_id. 2. Provide root cause hypothesis, supporting/contradicting evidence, trajectory, body systems. 3. Include gap analysis and cross-system connections. 4. Use ONLY measurement_ids from input data. ### Output Format Return JSON: `{ "patterns": [...], "gap_analysis": [...], "cross_system_connections": [...] }` ``` ```text clinical_summary.md — Stage 4 Legacy Mode (single page, 6 findings max) S4 • LLM Copy ``` ``` You are writing the final content for a 1-page patient health summary. You have clinical patterns, enriched biomarker data, and derived markers. Produce the summary content that will be rendered into the report. Always respond with valid JSON only. No markdown, no explanation, just JSON. --- ## Clinical Summary Generation You are producing the final structured output for a patient health summary report. This is a single-page report — content must fit on one A4 page. ### Input Data #### Clinical Patterns {{patterns_json}} #### Enriched Biomarkers {{enriched_biomarkers_json}} #### Derived Markers {{derived_markers_json}} {{context_section}} ### Rules - **Headline** (max 15 words): most clinically significant finding - **Key Findings** (1-6): severity, trend, context, functional_note, pattern_ref - **Healthy Highlights** (0-4): biomarkers in good health - **Recommendations** (1-3): category, rationale, pattern_ref. Do NOT suggest meds already being taken. - **Gap Recommendations** (0-5): missing tests - **Overall Status**: attention_needed | monitoring_recommended | generally_healthy - **Trajectory Summary** (max 20 words) ### Output Format Return JSON: `{ "headline": "...", "key_findings": [...], "healthy_highlights": [...], "recommendations": [...], "gap_recommendations": [...], "overall_status": "...", "trajectory_summary": "..." }` ``` ## Data Contracts All Pydantic models live in `cascade/models.py`. Models grouped by the stage that produces them. Shared models (Input, S0-S2) are identical to health-summary; predictive-specific models highlighted below. ### Input Models (shared) BiomarkerInput, CascadeInput, PatientContext — same as health-summary ```text Copy ``` class BiomarkerInput(BaseModel): ```json test_name: str measurement_id: str value: str status: str reference_range: str unit: str | None = None test_date: str | None = None all_measurements: list[dict[str, Any]] | None = None file_name: str | None = None biomarker_id: str | None = None record_id: str | None = None additional_data: dict[str, Any] = {} unit_mismatch: bool = False ``` class PatientContext(BaseModel): ```text patient_name: str = "Patient" age: int | None = None gender: str | None = None diagnoses: list[dict[str, Any]] = [] medications: list[dict[str, Any]] = [] procedures: list[dict[str, Any]] = [] genetics: list[dict[str, Any]] = [] report_date: str = "" enabled_types: set[str] ``` ### Stage 2.5 Output — Projections TrendAnalysis, TimeHorizonProjection, BiomarkerProjection, SystemProjectionSummary, ProjectionOutput ```text Copy ``` ``` class TrendAnalysis(BaseModel): ```text """Blended trend from patient data + population norms.""" slope: float # annual rate of change in native units direction: Literal["increasing", "decreasing", "stable"] = "stable" confidence: Literal["high", "medium", "low"] = "low" source: Literal["patient_data", "population_norm", "blended"] = "population_norm" r_squared: float | None = None data_points: int = 0 ``` class TimeHorizonProjection(BaseModel): ```text """Single time-horizon projection for a biomarker.""" years: int # 10, 20, or 30 projected_value: float projected_status: Literal["optimal", "within_lab", "outside_optimal", "outside_lab", "unknown"] = "unknown" risk_level: Literal["low", "moderate", "high", "critical"] = "low" threshold_crossing: str = "" # clinical threshold label if crossed, else "" ``` class BiomarkerProjection(BaseModel): ```text """Full projection for a single biomarker across all horizons.""" biomarker_id: str display_name: str current_value: float unit: str trend: TrendAnalysis projections: list[TimeHorizonProjection] = [] # at 10, 20, 30 years clinical_significance: str = "" ``` class SystemProjectionSummary(BaseModel): ```text """System-level aggregation of biomarker projections.""" system_name: str trajectory: Literal["declining", "stable", "improving"] = "stable" risk_10y: Literal["low", "moderate", "high", "critical"] = "low" risk_20y: Literal["low", "moderate", "high", "critical"] = "low" risk_30y: Literal["low", "moderate", "high", "critical"] = "low" threshold_crossings: list[str] = [] biomarker_ids: list[str] = [] ``` class ProjectionOutput(BaseModel): ```json """Output of Stage 2.5: Projections.""" biomarker_projections: list[BiomarkerProjection] = [] system_summaries: list[SystemProjectionSummary] = [] projection_horizons: list[int] = [10, 20, 30] patient_age: int | None = None data_source_summary: dict[str, int] = {} # count by source type ``` ``` ### Stage 3 Output — Predictive Patterns AgingRiskAssessment, PredictivePattern, PredictivePatternOutput ```text Copy ``` ``` class AgingRiskAssessment(BaseModel): ```text """Per-pattern aging risk narratives at each projection horizon.""" risk_10y: str = "" # max 30 words risk_20y: str = "" # max 30 words risk_30y: str = "" # max 30 words acceleration_factors: list[str] = [] deceleration_factors: list[str] = [] ``` class PredictivePattern(ClinicalPattern): ```text """Extends ClinicalPattern with aging risk projections.""" aging_risk: AgingRiskAssessment = AgingRiskAssessment() projected_severity_10y: Literal["critical", "significant", "monitor"] = "monitor" projected_severity_20y: Literal["critical", "significant", "monitor"] = "monitor" projected_severity_30y: Literal["critical", "significant", "monitor"] = "monitor" ``` class PredictivePatternOutput(BaseModel): ```text """Output of Stage 3 in predictive mode.""" patterns: list[PredictivePattern] # 1-4 gap_analysis: list[GapAnalysisItem] = [] cross_system_connections: list[str] = [] aging_acceleration_score: int = 50 # 0-100 ``` ``` ### Stage 4 Output — Predictive Summary PredictiveFinding, PredictiveRecommendation, PredictiveSummaryOutput ```text Copy ``` ``` class PredictiveFinding(SummaryFinding): ```text """Extends SummaryFinding with projected values.""" projected_10y: str = "" projected_20y: str = "" projected_30y: str = "" threshold_alert: str = "" ``` class PredictiveRecommendation(Recommendation): ```text """Extends Recommendation with time sensitivity.""" time_sensitivity: Literal["immediate", "within_1y", "within_5y", "long_term"] = "long_term" impact_description: str = "" # max 25 words ``` class PredictiveSummaryOutput(BaseModel): ```text """Output of Stage 4 in predictive mode — everything for multi-page render.""" headline: str # max 15 words aging_outlook: Literal["accelerated", "on_track", "favorable"] = "on_track" key_findings: list[PredictiveFinding] # 1-8 healthy_highlights: list[HealthyHighlight] = [] # up to 4 recommendations: list[PredictiveRecommendation] # 1-5 gap_recommendations: list[GapRecommendation] = [] # 0-5 overall_status: Literal["attention_needed", "monitoring_recommended", "generally_healthy"] trajectory_summary: str = "" # max 25 words patterns: list[PredictivePattern] = [] aging_acceleration_score: int = 50 # 0-100 html: str = "" # populated by render, not LLM ``` ``` ### Stage 4.5 Output — Charts ProjectionChart (NamedTuple) ```text Copy ``` ``` class ProjectionChart(NamedTuple): ```text """A rendered projection chart for a single biomarker.""" biomarker_id: str display_name: str base64_png: str # full data URI: data:image/png;base64,... ``` ``` ## Library Dependency Map Forge-sentinel libraries used at each stage. = deterministic stage, = LLM stage, = predictive-aging unique stage. ```text Library S0 S1 S2 S2.5 S3 S4 S4.5 S5 chr-core chr-data chr-llm chr-charts chr-html chr-styles chr-pdf chr-verify chr-resilience chr-langfuse chr-frontend chr-logging chr-document-manager ``` ### Library Details ```text chr-core Exception hierarchy (RetryableError, NonRetryableError), CHRBaseSettings, PII sanitization. Transitive dependency via chr-llm. chr-data Async N1 API fetching, transforms, biomarker reference data (optimal ranges, body systems, medication markers, derived formulas, unit conversion). Used in S0-S2, S2.5 (population norms), S4.5 (historical data lookup). chr-llm LLM calls with billing, JSON parsing, token tracking. `allm_json_call()` for S3-S4, `load_prompt()` for .md prompt files. Auto-prefix bare model names with `openai/` when proxy enabled. chr-charts Chart generation via matplotlib. Renders projection charts with historical data points, trend lines, projected curves, lab/optimal range bands, and clinical threshold lines. Output: base64 PNG. chr-html HTML fragment/document builders. `html_document()` with multi-page support for predictive-aging. chr-styles CSS loader. `get_report_css("multi-page", "predictive-aging")`. chr-pdf HTML→PDF rendering, compression. `WeasyPrintRenderer` via `publish_report()`. chr-verify Hallucination detection: name matching, timeline validation, projected value verification. Integrated at S3-S4 post-LLM validation. chr-resilience Retry decorators (`@retry_llm`, `@retry_api`) on LLM calls in S3-S4. RetryBudget for cascading failure protection. chr-langfuse Langfuse LLM tracing (spans, cost, prompt visibility). `traced_stage()` wraps S3-S4 LLM calls. chr-frontend Progress tracking, UI phases. `ProgressReporter`, `UIPhase`. Used across all stages. chr-document-manager S3/GCS upload, signed URLs, publish pipeline. `publish_report()` for HTML + PDF + cloud upload. ``` ## Configuration Pydantic Settings in `src/predictive_aging/config.py`. All settings overridable via environment variables. Predictive-aging-specific fields highlighted. Settings Model ```text Copy ``` class Settings(BaseSettings): ```json # LLM llm_model: str = "gemini-2.5-pro" llm_temperature: float = 0.1 llm_timeout: int = 180 # Service identity service_name: str = "workflow-predictive-aging" # Cascade tuning prefilter_max_output: int = 100 # Projection settings (predictive-aging specific) projection_horizons: list[int] = [10, 20, 30] projection_max_biomarkers: int = 50 # PII de-identification (predictive-aging specific) deidentify_pii: bool = True # Progress reporting (forge mode) n1_api_base_url: str = "" n1_api_key: str = "" sync_with_cloud: bool = True # Storage (cloud upload) bucket_name: str = "" storage_provider: str = "s3" aws_region: str = "us-east-2" skip_uploads: bool = False model_config = {"env_prefix": "", "env_file": ".env", "extra": "ignore"} ``` ### Environment Variables ```text Variable Default Description `LLM_MODEL` `gemini-2.5-pro` LLM model for stages 3-4 (via LiteLLM) `LLM_TEMPERATURE` `0.1` Low temperature for consistent clinical output `LLM_TIMEOUT` `180` Timeout in seconds per LLM call `SERVICE_NAME` `workflow-predictive-aging` Service identity for billing metadata `PREFILTER_MAX_OUTPUT` `100` Max biomarker groups selected (50 unhealthy + 50 healthy) `PROJECTION_HORIZONS` `[10, 20, 30]` Years to project forward (predictive-aging specific) `PROJECTION_MAX_BIOMARKERS` `50` Max biomarkers to project (predictive-aging specific) `DEIDENTIFY_PII` `true` Replace patient names with pseudonyms (predictive-aging specific) `N1_API_BASE_URL` *empty* N1 API base URL (forge mode) `N1_API_KEY` *empty* N1 API key (forge mode) `SYNC_WITH_CLOUD` `true` Enable cloud progress sync `BUCKET_NAME` *empty* S3/GCS bucket for report upload `STORAGE_PROVIDER` `s3` Cloud storage provider (s3 or gcs) `AWS_REGION` `us-east-2` AWS region for S3 `SKIP_UPLOADS` `false` Skip cloud upload (local mode) `INPUT_FILE` *none* Path to JSON patient file (local mode) `USER_ID` *none* Patient user ID (forge mode) `CHR_ID` *none* CHR session ID (forge mode) ``` ## Overview Easy CHR** generates an interactive, multi-section health report from raw medical PDFs (lab results, imaging, clinical notes) via a **7-phase agentic pipeline**. Four phases use iterative LLM tool-calling loops (where the model explores data, forms hypotheses, and builds output incrementally), one phase uses a single LLM call, and the final rendering is fully deterministic (Nunjucks templates — no LLM involvement). The output is a self-contained HTML report with Chart.js visualisations and a 3D body-twin viewer. ```text Pipeline Phases Agentic Phases 32+ LLM Tools 25+ TypeScript Interfaces ``` **Philosophy:** This is a *deep agentic reasoning system*, not a simple prompt→response pipeline. Each phase uses an iterative tool-calling loop where the LLM decides which tool to call, receives results, reasons about them, and iterates — often for 25–50 cycles. External state (Maps, Sets) preserves work product through chat compression, so the agent can lose conversation memory but never its output. ### Run Modes ##### Local Mode Upload PDFs via React UI → SSE-streamed pipeline → HTML report. `POST /api/realm` with multipart files. Requires `GEMINI_API_KEY`. ##### Forge Mode (K8s Job) N1 API data fetch → pipeline → HTML + S3 upload + signed URL. Requires `USER_ID`, `CHR_ID`, N1 API + S3 credentials. ### Three Entry Points ##### `execute()` Full pipeline: Raw PDFs → OCR extraction → all 7 phases. Used by HTTP upload path. ##### `executeWithExtractedContent()` Skips Phase 1: Pre-extracted markdown from N1 API → Phases 2–7. Used by job runner. ## Pipeline Flowchart 4 agentic phases (iterative LLM tool-calling loops) + 1 single-shot LLM call + 1 OCR extraction + 1 deterministic render. Each agentic phase uses external state that **survives chat compression** — the agent can lose conversation history but never its work product. ```text P1 Document Extraction Deterministic + OCR Gemini Vision • PDF → Markdown with page numbers P2 Agentic Medical Analysis Agentic • 50 iter 8+ tools • Explore → Hypothesise → Cross-reference → Synthesise P3 Agentic Research & Validation Agentic • 50 iter 5 tools • Web search → Fetch primary sources → Record verdicts P4 Agentic Data Structuring Agentic • 25 iter 8 tools • Builds structured_data.json incrementally (SOURCE OF TRUTH) P5 Agentic Validation Agentic • 15 iter 11+ tools • Verification-focused: verify_value_exists, compare_date_ranges P6 Organ Insights Single LLM Call Per-organ markdown • Bridge between CHR and 3D body twin P7 Deterministic HTML Render Deterministic Nunjucks + Tailwind (inline) + Chart.js + Cytoscape.js • Milliseconds, 100% fidelity ``` ### How Each Agentic Phase Works Phases 2–5 all follow the same iterative tool-calling pattern. The LLM is given tools, decides which to call, receives results, and iterates until it calls a completion tool. ##### The Agentic Loop (shared pattern) - **1. Send history + tools to LLM** via `generateContentStreaming()` (keeps LiteLLM proxy alive) - **2. LLM responds** with tool calls (e.g. `search_data("platelets")`) or text - **3. Execute each tool** against external state (Maps, Sets); add results to conversation - **4. Check completion** — did LLM call the `complete_*` tool? If yes, validate and return - **5. Check compression** — if tokens > 50% of 1M limit, compress oldest 70%, keep newest 30% - **6. Repeat** until completion or max iterations reached ##### External State (survives compression) - **Medical Analyst:** `currentAnalysis: Map` — exploration notes and synthesis sections - **Research Agent:** `findings: Map`, `searchesPerformed: Set`, `documentsRead: Set` - **Data Structurer:** `currentJson: Map` — the JSON being built section-by-section - **Validator:** `validationIssues: Array` — accumulated issues ##### Why Streaming? - Gemini 3 Pro thinking takes 2–3 minutes on complex function calls - LiteLLM gateway has a ~5 minute TTFT (time-to-first-token) timeout - `generateContentStreaming()` sends first token in seconds, preventing timeout cascade ## Stage Details #### Document Extraction — PDF to Markdown Converts raw PDFs into a single combined markdown file. Each document gets a `## [filename.pdf]` header. For N1 API sources, pre-extracted markdown is fetched directly (skipping OCR). ```text Input `PDF/TXT/CSV files *or* N1 API record IDs` Output `extracted.md (~460KB combined)` ``` ##### How It Works - **PDFs:** Gemini Vision OCR (`gemini-2.5-flash`) processes each page, adds page numbers - **Text files:** Direct read with `## [FileName]` section markers - **N1 API:** Pre-extracted markdown via `/records/{id}/markdown` (fast path, avoids OCR cost) - **Mixed mode:** Combine pre-extracted + OCR for records with partial extraction - All content combined into one `extracted.md` file ##### Challenges - Multi-language content (English, Chinese, Malay) in same table - Inconsistent units across labs (`g/dL` vs `g/L`, `mmol/L` vs `mg/dL`) - OCR artefacts and formatting inconsistencies - Reference ranges vary by lab provider ```text @google/genai gemini-2.5-flash ``` #### Agentic Medical Analysis — Explore, Hypothesise, Synthesise An **Integrative Systems Physician** agent iteratively explores 17+ years of medical records. It reads documents oldest→newest, forms hypotheses, cross-references findings across body systems, and produces a comprehensive medical narrative. ```text Input `extracted.md (via tools) + patient question` Output `analysis.md (medical narrative)` ``` ##### How the Agent Reasons (Two Modes) - **MODE A — Exploration (Phases 1–4 internally):** Read documents oldest→newest via `read_document()` - Write descriptive exploration notes (e.g. "OAT Panel Jul 2025", "Cross-System Pattern: Methylation → Coagulation") - For every marker with ≥2 occurrences: call `get_value_history()` to track trends - Perform ≥10 searches, write ≥8 exploration notes ```text **MODE B — Synthesis (Phase 5 internally):** ``` - Write final synthesis sections with `replace=true` - Required: Executive Summary, System-by-System Analysis, Timeline, Root Cause Hypothesis, Causal Chain, Keystone Findings, Recommendations, Missing Data ##### Available Tools (8+) - `list_documents()` — inventory of all source docs - `read_document(name)` — full text of one document section - `search_data(query)` — case-insensitive substring search with ±2 line context - `get_value_history(marker)` — all historical values, chronologically sorted (for trend detection) - `get_analysis()` / `get_section(name)` — read current progress - `update_analysis(section, content, replace?)` — write to external state Map - `get_date_range()` / `list_documents_by_year()` / `extract_timeline_events()` — temporal awareness - `complete_analysis(summary, confidence)` — validates synthesis sections exist, signals done ##### Data Fidelity Rule - Every extracted value must include: exact number + unit + reference range + flag - Format: `TSH: 2.3 mIU/L (ref 0.4-4.0) *L` ```text gemini-3-pro-preview 50 max iterations chat compression ``` #### Agentic Research & Web Validation — Evidence Grounding An **Agentic Medical Research Specialist** searches the web to ground claims from the analysis with authoritative external evidence. It searches iteratively, fetches and reads primary sources, and records honest verdicts (supported / unsupported / contested). ```text Input `analysis.md` Output `research.json (claims + sources)` ``` ##### How the Agent Reasons - **Survey:** Identify 12–20 claim candidates from analysis, prioritise by centrality and actionability - **Search:** Multiple searches per claim with different formulations (mechanism, evidence quality, guideline position) - **Read:** Fetch primary sources (papers, guidelines, institution pages) via `fetch_document()` — up to 8KB each - **Record:** `record_finding(claim, verdict, confidence, evidence, sources)` with honest verdicts - **Audit:** `get_research_state()` to check coverage before completing - "Depth over speed" — `contested` and `unsupported` verdicts are valuable findings, not failures ##### Available Tools (5) - `search_web(query)` — Gemini built-in web search; resolves Google redirect URLs - `fetch_document(url)` — reads up to 8KB of content, strips HTML - `record_finding(claim, verdict, confidence, evidence, sources)` — stores in external Findings Map - `get_research_state()` — audit progress (claims covered, searches done, findings count) - `complete_research(summary)` — validates substantive summary, minimum coverage ##### Source Classification - `journal`: pubmed, ncbi, nejm, lancet - `guideline`: who.int, cdc.gov, nih.gov - `institution`: mayoclinic, clevelandclinic - `education`: uptodate, medscape ```text gemini-3-pro-preview 50 max iterations web search ``` #### Agentic Data Structuring — Building the Source of Truth A **Data Extraction Specialist** agent transforms the medical narrative and research findings into a single, precisely structured JSON object (`structured_data.json`). This JSON drives the deterministic HTML renderer — every field must be accurate and complete. ```text Input `analysis.md (inline) + research.json (inline) + extracted.md (tools)` Output `structured_data.json (25+ fields, Zod-validated)` ``` ##### Source Priority (conflict resolution) - **Priority 1 — Analysis** (inline): Primary for clinical reasoning, interpretations, causal chains - **Priority 2 — Research** (inline): Citations, validated claims - **Priority 3 — Raw Source** (via tools): Original values, dates, reference ranges ##### How JSON is Built Incrementally - Agent reads analysis + research in context window - Calls `update_json_section("executive_summary", '{...}')` for object sections - Calls `append_to_section("data_gaps", '[{...}, {...}]')` for arrays (3–5 items/batch) - Calls `search_source()` and `get_value_history()` to verify values from raw source - Periodically calls `get_json_draft()` to check completeness - Calls `complete_structuring()` when all required sections present ##### Mandatory Tool Calls for detailed_findings - For EVERY body system: `search_source("[system] diagnosis")` + `search_source("[system] biomarker")` - For EVERY biomarker trend: **MUST** call `get_value_history("[marker]")` — never write trend data from memory - If `get_value_history()` returns <2 data points → exclude from `biomarker_trends[]` (charts need ≥2) ##### Available Tools (8) - `search_source(query)` — search extracted.md - `get_value_history(marker)` — all historical values for trend building - `get_date_range()` / `list_source_documents()` — orientation - `update_json_section(section, data)` — set object fields in external Map - `append_to_section(section, items)` — add to array fields (1–5 per call) - `get_json_draft()` — review progress - `complete_structuring(summary)` — validates required sections: executiveSummary, criticalFindings, timeline, diagnoses, systemsHealth ##### Post-Processing: Zod Validation - Output validated through `report.zod.ts` with intelligent coercion - `normalizeKeysToSnakeCase()` — camelCase → snake_case recursively - `z.coerce.number()` — string "6.5" → number 6.5 - `.catch("neutral")` — invalid enum values default instead of failing - `flatEventsToEras()` — flat event arrays → hierarchical era/group/findings - Reference filtering — strips entries without `https://` URIs ```text gemini-3-pro-preview 25 max iterations zod ``` #### Agentic Validation — Verification-Focused QA A **QA Specialist** agent verifies that `structured_data.json` accurately reflects the source data. Unlike the analyst which *explores*, the validator *verifies*. Tools return summaries and verification results (not full content) to prevent payload bloat. ```text Input `structured_data.json + extracted.md + analysis.md` Output `validation.md + corrected JSON (if needed)` ``` ##### Verification Workflow (7 Steps) - **1. Orientation:** `list_documents()` + `get_json_overview()` + `compare_date_ranges()` - **2. Timeline check:** `find_missing_timeline_years()` → flag gaps as critical issues - **3. Numeric check:** For each document → `get_document_summary()` → for each abnormal value → `verify_value_exists()` - **4. Array check:** Verify counts in criticalFindings, diagnoses, trends - **5. Qualitative check:** Search for medications, supplements, symptoms - **6. PII check:** Any full name, DOB, SSN in JSON → critical issue - **7. Complete:** `complete_validation(status)` — pass / pass_with_warnings / needs_revision ##### Primary Verification Tool - `verify_value_exists(marker, expected_value?)` returns: `{ existsInSource, existsInJson, sourceValue, jsonValue, match }` - Combined check: is value in extracted.md AND in structured_data.json? Do they agree? ##### Correction Mechanism - If `needs_revision`: surgical JSON patches via `deepMergeJsonPatch()` - Faster and more precise than full regeneration ```text gemini-3-pro-preview 15 max iterations 11+ tools ``` #### Organ Insights — Bridge to 3D Body Twin Single LLM call that generates per-organ markdown from `structured_data.json`. This is the bridge between the HTML report and the 3D body twin visualisation. ```text Input `structured_data.json` Output `organ_insights.md → body-twin.json` ``` ##### Per-Organ Sections - `## Organ Name` headers with Status (critical/warning/stable/optimal), Confidence, Markers table - Clinical findings, cross-organ connections, clinical implications - `BodyTwinTransformer` parses markdown into typed `BodyTwinData` for Three.js viewer - 14 body systems, per-organ health scores, cross-organ connection edges ```text gemini-3-pro-preview single call ``` #### Deterministic HTML Render — Nunjucks Templates Pure data transformation — **no LLM calls**. Takes validated JSON, computes a section manifest (which sections have enough data to render), renders Nunjucks templates, and compiles Tailwind CSS inline. Output is a self-contained HTML file with no external CDN dependencies. ```text Input `StructuredReportParsed (Zod-validated) + organ_insights.md` Output `index.html (self-contained, interactive)` ``` ##### Rendering Pipeline - **1. Compute Section Manifest:** Boolean map of which sections render (threshold-based rules) - **2. Nunjucks Templates:** `base.html.njk` + 13 section includes gated by manifest - **3. Custom Filters:** `format_bold`, `format_citations`, `paragraphs`, `trend_chart_data`, `score_level` - **4. Tailwind CSS:** PostCSS compiles used classes into inline `