---
title: "AWS cloud infrastructure — N1 IT Landscape"
---

N1 IT Landscape — The Blueprint · CF-Access internal

# AWS cloud infrastructure

N1 IT Landscape · Owner: Arun Venkataraman · source of truth: markdown SSOT

# N1 IT Landscape — AWS cloud infrastructure

> **CURRENT-STATE** as of **2026-06-09** · part of the **N1 IT Landscape** (start at **[Overview](/landscape/overview/)**). Owner: **Arun Venkataraman** (Principal Engineer).

_The cloud account model and a deep inventory of the production account that holds both the staging and production clusters and all data._

* * *

## 8\. AWS — CLOUD ACCOUNTS & PRODUCTION INFRASTRUCTURE

### 8.1 Account model (3 accounts)

Account | ID | Role | In scope here?  
---|---|---|---  
Management | `971422719525` | AWS **IAM Identity Center** (SSO) + **billing** only | No clusters  
Development / AI | `083636777616` | Holds **`n1-ai-cluster`** — `litellm.n1-research.com`, **SigNoz** , **Langfuse** , AI inference | Out of scope (own account)  
**Production** | **`191421493115`** | Holds **BOTH** `n1-staging-cluster` and `n1-production-cluster` \+ all data | ✅ **Fully inventoried**  
  
> Note: the development/AI account follows the same pattern — no external IPs other than its NAT gateway; its services (LiteLLM, SigNoz, Langfuse) are reachable only through Cloudflare Tunnels. Sign-in/access for all three accounts is via Google IdP + permission sets; only Arun, Eyad, Darpan have AWS access. **Primary region for everything:`us-east-2`.**

Everything below is the **production account`191421493115`** (where staging _and_ prod both live).

### 8.2 Compute — EKS, Karpenter, spot, Bottlerocket

Property | n1-staging-cluster | n1-production-cluster  
---|---|---  
Kubernetes version | **v1.35.4-eks** (platform eks.13) | **v1.35.4-eks** (platform eks.13)  
Control-plane logging | api, audit, authenticator | api, audit, authenticator  
etcd encryption | 🔒 KMS CMK envelope (`alias/eks/…`) | 🔒 KMS CMK envelope  
Nodes (live) | 12 | 10  
Container host OS | **Bottlerocket** (100%) | **Bottlerocket** (100%)  
Container runtime | containerd 2.1.x+bottlerocket | containerd 2.1.x+bottlerocket  
  
  * **Node provisioning:** **Karpenter`1.9`** (image `dhi.io/karpenter:1.9`, sha256-pinned). 🔒 **`dhi.io` = Docker Hardened Images (DHI)** — Docker, Inc.'s paid offering of minimal, continuously patched, signed, near-zero-CVE base/tool images. N1 pulls DHI images through an **ECR pull-through cache** under the `dhi-mirror/` prefix, authenticated by a **DHI token in Secrets Manager** , so the hardened images are cached in N1's private ECR rather than fetched live. A sound pattern for a HIPAA estate; the only caveat is keeping that token and the mirror source controlled. Static EKS managed node-group provides 2 baseline system nodes per cluster (t4g, Bottlerocket **1.59.0**); everything else is Karpenter-provisioned.
  * 🔒 **Spot-first:** Karpenter NodePools prefer **spot** , falling back to **on-demand** only when spot is unavailable. Live fleet is ~50/50 (22 EC2 instances: 11 spot / 11 on-demand) because stateful pools pin on-demand. **Bottlerocket versions in flight: 1.59.0 / 1.61.0 / 1.62.0** (Karpenter drift).
  * **Karpenter NodePools:** staging 5 (`application`, `chr-generators`, `cpu-inference`, `gpu`, `on-demand-stateful`); production 6 (adds a dedicated **`redis`** pool). All use `amiFamily=Bottlerocket`, `alias: bottlerocket@latest`. Arch is **arm64 (Graviton)** for application/CHR/stateful/redis pools; **amd64** for `cpu-inference` and `gpu`.
  * **GPU:** 2 GPU spot nodes (staging `g5.2xlarge`, prod `g6.2xlarge`) on the NVIDIA Bottlerocket variant, for the inference servers (mineru).
  * **EKS add-ons (identical both clusters):** vpc-cni `v1.21.1`, coredns `v1.13.2`, kube-proxy `v1.35.0`, aws-ebs-csi-driver `v1.56.0`, metrics-server `v0.8.1`.

### 8.3 Container registry — ECR

**101 ECR repositories** under prefixes `n1-prod/`, `n1-staging/`, `docker-hub/` (pull-through cache), `dhi-mirror/`. **16 are empty** (unused prod repos: several `twin-*`, `chr-evaluator`, evals, some workflows). Image tags follow `V<YYYYMMDD>[_<sha7>]`. The deployed image tag per service (the "what's actually running") is in [§9.4](/landscape/kubernetes/). ECR pull-through cache mirrors Docker Hub (auth'd via the `ecr-pullthroughcache/docker-hub` secret).

### 8.4 Storage — S3 (15 buckets, all private)

🔒 All 15 buckets have **full public-access block** enabled.

Bucket (per env: `n1-{staging,prod}-…`) | Purpose  
---|---  
`…-user-data` | Patient uploads / user data (the medical records doctors upload land here)  
`…-kyc-documents` | KYC documents  
`…-qdrant-backups` | Qdrant vector-DB snapshots (7-day lifecycle)  
`…-memgraph-backups` | Memgraph graph-DB snapshots (7-day lifecycle)  
`n1-terraform-state-{staging,prod}` | Terraform remote state  
`twin-staging-*` (us-east-1) | Retired Twin product artifacts  
  
### 8.5 Databases — RDS Aurora (manual, NOT Terraform-managed)

🔒 Databases are **intentionally not automated** — there are **zero`aws_rds_*` resources in Terraform** (confirmed). They are provisioned **by hand** so that nobody can accidentally destroy them via a `terraform apply`. They live in **dedicated, isolated "database" VPCs** peered into the cluster VPCs (§8.7).

**Engine: Aurora PostgreSQL 17.7, Serverless v2 (`db.serverless`)** for every cluster. Per-service databases (each major service owns its own DB):

Aurora cluster (per env) | Owning service | Multi-AZ  
---|---|---  
`n1-{production,staging}-app` | **api-backend** (the main platform DB) | No  
`auth-{production,staging}` | **authentication-service** \+ **Kratos** \+ **Hydra** (Ory) | No  
`billing-{production,staging}` | **billing-service** | 🔒 **prod = Yes** (only Multi-AZ cluster)  
`litellm-{production,staging}` | **LiteLLM** | No  
`forge-runner-{production,staging}` | **forge-runner** (the **agno** database) | No  
  
The `forge-runner-{env}` Aurora cluster is the **agno** database for the CHR workflow runs: it stores agno session/memory (`AsyncPostgresDb`) and LangGraph checkpoints (`AsyncPostgresSaver`). It is wired in both staging and production via `FORGE_RUNNER_DB_URL`, delivered as a key in the `forge-sentinel-secrets` ExternalSecret (`{staging,production}-cluster/forge-sentinel-agno-db-url`), read by `forge_runner/memory.py`. The chart's base default falls back to per-pod ephemeral SQLite only when that variable is absent (it is set in both environments).

### 8.6 Cache — ElastiCache Valkey

The cache and inter-service stream layer (often referred to internally as "Redis") is **Valkey 8.2.0** on ElastiCache (the open-source Redis fork adopted after Redis's license change). 2 replication groups, auto-failover enabled:

Group | Node type  
---|---  
`production-valkey` | `cache.m7g.large`  
`staging-valkey` | `cache.r7g.xlarge` (larger node than production)  
  
These Valkey instances carry **all the Redis Streams** that drive the parsing/CHR pipeline ([§10](/landscape/app-architecture/)).

### 8.7 Network — VPCs, peering, egress

VPC | CIDR | Role  
---|---|---  
`n1-production-cluster-vpc` | 10.30.0.0/16 | Prod EKS  
`n1-production-database` | 10.70.0.0/16 | Prod Aurora (isolated)  
`n1-staging-cluster-vpc` | 10.10.0.0/16 | Staging EKS  
`n1-staging-database` | 10.80.0.0/16 | Staging Aurora (isolated)  
  
  * **VPC peering (both active):** prod-cluster ↔ prod-db (`pcx-043ae3b0…`), staging-cluster ↔ staging-db (`pcx-0b0be348…`). The cluster reaches its database across the peering link only.
  * 🔒 **All 18 subnets are private** (no public-IP-on-launch). **One NAT gateway per cluster VPC** (both in `us-east-2a` only — no AZ-redundant NAT). Internet **egress** is via NAT; internet **ingress** is **only** via Cloudflare Tunnels. No ALB/NLB, no ACM certs, no Route53 zones.
  * **DynamoDB:** `n1-api-vitals-staging` (patient vitals/appointment notes, PAY_PER_REQUEST) + Terraform lock tables.

### 8.8 Secrets & keys in AWS

  * **AWS Secrets Manager: ~150 secrets** , grouped `production-cluster/*` (~57), `staging-cluster/*` (~57), auto-managed `rds!cluster-*` (8), `rds-db-credentials/*` (8), plus the ECR pull-through cred. Categories include DB URLs, JWT/RS256 signing keys, LiteLLM/Langfuse keys, GCP/Vertex service accounts, OpenAI/Stripe/Postmark keys, Kratos/Hydra encryption keys, Cloudflare tunnel tokens, ArgoCD/KEDA Slack webhooks, GitHub/HuggingFace/DHI PATs. **(Values never read; names only.)**
  * 🔒 **KMS:** 2 customer EKS-envelope CMKs (`alias/eks/n1-{production,staging}-cluster`); RDS/S3/Secrets use AWS-managed keys (no customer-managed CMK for those).
  * These secrets reach pods via **External Secrets Operator** ([§9.3](/landscape/kubernetes/)), not by hand.

### 8.9 What Terraform manages (and what it doesn't)

Repo `n1-infrastructure` (`aws/n1/`), Terraform **`>= 1.3`**, provider **`hashicorp/aws>= 6.30.0`**, EKS module `terraform-aws-modules/eks ~> 21.15`. Layout: `bootstrap/` (state bucket + lock table), `environments/{staging,production,ai}/`, `modules/{database-vpc, network, eks-core, karpenter, irsa, external-secrets, ecr, github-oidc-ecr}`. It manages EKS, both VPCs + peering + NAT, Karpenter (IAM + SQS interruption queue + nodepools via Helm), IRSA roles, External Secrets Operator + ClusterSecretStore, ECR + pull-through cache, GitHub OIDC→ECR, the 4 S3 data buckets/env, the DynamoDB vitals table, and the **Cloudflare Tunnel** Helm release. 🔒 It does **NOT** manage the Aurora databases (manual, §8.5) and the environment root modules declare no `required_providers` of their own (only `bootstrap` does).

* * *