--- title: Phase 2 — Plan & Subscription Model + Usage Ledger + Frontend Swap --- # Phase 2 — Plan & Subscription Model + Usage Ledger + Frontend Swap ## Problem → Solution **Problem:** users see raw counts with no meaning ("412 pages — of what?"), there's no data model tying a doctor to a billing tier, and usage data lives in api-backend — billing-service can't answer "how many pages did Dr. Smith commit this month?" without a cross-service round-trip. Separately, the frontend is still wired to the pre-subscription model: the current `BillingPage`, balance widgets, "credits remaining" copy, and "insufficient funds" error paths all read off the old LiteLLM-budget layer. Leaving that UI live during cutover would render stale data against a source of truth that no longer exists. **Solution:** introduce `plans` + `subscriptions` + `quota_events` in billing-service. Every extract click calls `/quota/charge`, which writes one immutable row in `quota_events`. Billing-service becomes the single source of truth for usage: `SUM(quota_events.count)` scoped to the current period. Existing users are hard-migrated to **Starter** with `charge_exempt = true`. `charge_exempt` is a **narrow flag on the overage billing path only** — when true, billing-service suppresses Stripe metered-usage posts for overage (Phase 3 consumer). Nothing else changes. Migrated users are full subscribers with real subscription rows, normal quota enforcement, normal app functionality, and the standard billing UI — they simply have no Stripe customer/subscription yet because they haven't gone through Checkout (no payment method at cutover). Admin endpoint lets ops flip `charge_exempt` off per-user so overages get billed normally when they're ready. **Same phase swaps the frontend off the old billing surfaces entirely onto the new quota UI** — old `BillingPage`, balance widgets, credit/top-up flows, and "insufficient funds" UX are deleted; `QuotaStatusCard` + `QuotaIndicator` are the only billing surface. Phase 2 ships with **no enforcement** — `/quota/charge` writes rows but doesn't 429, there's no Stripe side, and there's no replacement for the old "insufficient funds" block until Phase 3's paid-plan branches land. Because all five phases deploy in one cutover (master spec), the un-enforced gap is a code state inside the PR stack, not a prod state. Backend-side legacy billing code (LiteLLM-budget / balance checks in api-backend's extract + report-gen paths, parser-router, workflow repos) stays live in Phase 2 — nothing reads the frontend-side billing surfaces, so those backend checks aren't user-visible; they get swept in Phase 6. ## What this phase adds to the user experience - Billing page shows plan name ("Your plan: Starter" / "Professional" / etc.). - Pages and reports used this period rendered as "412 of 2,000" against the plan quota. - Migrated users see the standard Starter layout with "500 of 500 pages remaining" from the moment they log in post-cutover — no separate "legacy" or "complimentary" UX. - Always-on header pill shows remaining quota from anywhere in the app. - Old billing page is gone. All billing UI reads from Contract B — no "credits remaining", no "insufficient funds" modals, no balance top-up widgets. - Admin dashboard gains a checkbox per subscription to toggle `charge_exempt` — non-user-facing, ops only. No enforcement, no Stripe checkout, no plan picker, no overage — that's Phase 3. Phase 2 stands up the data model + the write path and moves the frontend onto the new surface; Phase 3 layers paid-plan branches on top without revisiting the frontend. ## Goal - Every user has a `subscriptions` row pointing at a `plans` row. - Every existing user lands on `Starter` with `charge_exempt = true` (migrated at cutover). New signups land on `Starter` with `charge_exempt = false` — they'll pick a plan and complete Checkout via Phase 3. - Every extract and report-generate click writes a `quota_events` row. The `charge_exempt` flag only suppresses overage Stripe posts (Phase 3 consumer); quota enforcement, row-writes, and all other functionality run identically for charge-exempt users. - Doctor's billing card shows "X of Y pages / reports" against Starter's quota. - Admin can toggle `charge_exempt` per user via the admin dashboard. - **Zero legacy billing UI left in react-frontend.** Old `BillingPage` layout, balance widgets, "credits remaining" copy, "add credits" / top-up flows, and "insufficient funds" error paths deleted. `QuotaStatusCard` + `QuotaIndicator` are the only billing surface. ## Repos touched | Repo | Changes | |---|---| | **billing-service** | `plans` + `subscriptions` + `quota_events` tables, Contract B (`/quota/status/{user_id}`), Contract C (`/quota/charge` — write path only at this phase; paid-plan enforcement branches ship in Phase 3), registration hook endpoint, admin endpoint for `charge_exempt` toggle. **All routes are N1-API-key only — billing-service never accepts user JWT.** | | **api-backend** | Inject `billing_client.charge` into `POST /records/batch-extract` (from Phase 1) and into the report-generation endpoint; expose `GET /billing/quota-status/me` as the user-facing pass-through for Contract B. Backend-side legacy budget / balance guards in the extraction pipeline are not touched in this phase — they stay live until Phase 6. | | **authentication-service** | Registration hook calls billing-service; one-shot backfill script at cutover | | **react-frontend** | **Full swap** — delete old `BillingPage` layout + balance widgets + credit top-up flows + "insufficient funds" error paths. Add the new `BillingPage` with `QuotaStatusCard` + always-on `QuotaIndicator` in the app shell. No feature flag to toggle between old and new — the old surfaces are gone as of this PR stack. | | **admin-dashboard** | `charge_exempt` checkbox on the subscription detail view, calling billing-service's admin endpoint | --- ## Step 1 — `plans` table **Problem:** billing-service has no catalog of tiers — code + queries have nowhere to resolve "what quota does Professional grant?" **Solution:** create a `plans` table seeded with the five rows from the pricing spec (`free_trial`, `starter`, `professional`, `practice`, `enterprise`). | Column | Type | Notes | |---|---|---| | `id` | UUID PK | | | `code` | text UNIQUE | | | `name` | text | | | `monthly_price_cents` | int NULL | Null for enterprise | | `stripe_price_id` | text NULL | Recurring price. Null for `free_trial` and `enterprise` | | `pages_quota` | int NULL | Null = unlimited (enterprise only) | | `reports_quota` | int NULL | | | `pages_overage_cents` | int NULL | Null if overage not available | | `reports_overage_cents` | int NULL | | | `pages_overage_price_id` | text NULL | Stripe metered price, Phase 3 consumer | | `reports_overage_price_id` | text NULL | | | `is_lifetime_cap` | bool | True for `free_trial` | | `is_active` | bool | | | `created_at` / `updated_at` | timestamptz | | No Legacy plan. Existing users migrate to `starter` with `charge_exempt = true` (Step 4). Stripe IDs filled by runbook after Stripe products exist. --- ## Step 2 — `subscriptions` table **Problem:** billing-service has no per-doctor record — nowhere to store which plan a user is on, their period bounds, Stripe IDs, or whether they're exempt from charging. **Solution:** create a `subscriptions` table with one row per user. | Column | Type | Notes | |---|---|---| | `id` | UUID PK | | | `user_id` | UUID UNIQUE | | | `plan_id` | UUID FK | | | `status` | text | `active`, `past_due`, `canceled`, `incomplete` | | `stripe_subscription_id` | text NULL | Null for users who haven't gone through Checkout yet (migrated users pre-transition, `free_trial`). Orthogonal to `charge_exempt` — a `charge_exempt=true` user can have a live Stripe subscription. | | `stripe_customer_id` | text NULL | Null for users who haven't gone through Checkout yet. Orthogonal to `charge_exempt`. | | `charge_exempt` | bool NOT NULL DEFAULT false | Narrow flag on the overage billing path only. When true: the Phase 3 overage branch skips the Stripe metered-usage post — no overage charge is ever billed to this user. Everything else is unchanged: full subscription row, normal quota enforcement, normal app access, normal billing UI, base-plan Stripe invoicing (where a Stripe subscription exists) continues untouched. Flipped via admin endpoint (Step 10). Default false for new signups; set true for migrated existing users (Step 4) and ops-comped accounts. | | `current_period_start` | timestamptz | Anchor for `compute_current_period()`. Set to `created_at` on row creation; rolled forward lazily by compute-on-read (see below). | | `cancel_at_period_end` | bool default false | | | `canceled_at` | timestamptz NULL | | | `tc_accepted_version` | text NULL | T&C version the user agreed to. Date-stamped (e.g. `"2026-05-01"`). Stamped at Checkout (paid), at trial signup (free trial), or at migration backfill (Step 4). Only null between Kratos registration and plan selection. | | `tc_accepted_at` | timestamptz NULL | Server-stamped at acceptance. Null iff `tc_accepted_version` is null. | | `created_at` / `updated_at` | timestamptz | | `overage_enabled`, `pages_stripe_item_id`, and `reports_stripe_item_id` are added in Phase 3 (Step 1) — they first do something when paid-plan enforcement + Contract E land. ### Compute-on-read period rolling No `current_period_end` column, no cron, no background job to roll periods. The active period is computed from `current_period_start` at read time: ```python def compute_current_period(sub: Subscription, now: datetime = utcnow()) -> tuple[datetime, datetime]: """Roll monthly anniversary-based periods forward from sub.current_period_start.""" anchor = sub.current_period_start if now < anchor: return (anchor, anchor + relativedelta(months=1)) months_elapsed = (now.year - anchor.year) * 12 + (now.month - anchor.month) if now.day < anchor.day or (now.day == anchor.day and now.time() < anchor.time()): months_elapsed -= 1 start = anchor + relativedelta(months=months_elapsed) end = start + relativedelta(months=1) return (start, end) ``` Used everywhere a period bound is needed: Contract B (period start/end/days_remaining), Contract C's charge transaction (stamps `quota_events.period_start` = computed start), Phase 3's quota SUM scope. Paid plans: Stripe webhooks (`customer.subscription.updated`) overwrite `current_period_start` with the authoritative value from Stripe at each renewal — the compute-on-read is only load-bearing between webhook events (and for `charge_exempt=true` users who have no Stripe renewal events). **Why compute-on-read over a cron:** no scheduled job to fail or drift, no "stuck on last month" bugs if a cron misses a fire, and `quota_events.period_start` becomes the authoritative historical record of which period every charge belonged to (immutable by design — Step 3). The compute is a few arithmetic ops against an indexed row; cost is irrelevant. **T&C columns are nullable by design.** The row is created at Kratos-registration (Step 5) *before* the user sees the T&C checkbox on `PickPlanPage` (Phase 3 Step 11) or the trial signup form (Phase 5). Acceptance writes the columns later via Contract D (paid) or carried through register-user (trial). The current T&C version string lives in billing-service config (`TC_CURRENT_VERSION` env var) — single source of truth, referenced at Checkout to enforce that the user agreed to the version that's live when they pay. Migrated users are stamped with the current version at backfill time (Step 4). --- ## Step 3 — `quota_events` table (append-only usage ledger) **Problem:** usage data lives in api-backend (`record_requests.page_count`, `report_gen_requests` count). For billing-service to answer "X of Y pages used this month" without a cross-service call, and for Phase 3 to enforce quota transactionally, the usage has to live where the subscription lives. **Solution:** one immutable row per extract/generate click, in billing-service. Period balance is `SUM(count)` scoped to `(subscription_id, period_start, kind)`. Phase 3 adds paid-plan enforcement branches to the write path without touching the schema. | Column | Type | Notes | |---|---|---| | `id` | UUID PK | | | `subscription_id` | UUID FK → `subscriptions` | | | `period_start` | timestamptz | Frozen at write from `compute_current_period(subscription).start`. Never updated. | | `kind` | text | `pages` \| `reports` | | `count` | int | Always positive. | | `overage_count` | int NOT NULL DEFAULT 0 | Always 0 in Phase 2 — populated by the overage branch added in Phase 3. | | `source_type` | text | `record_request` \| `report_gen` | | `source_id` | UUID | FK target is in api-backend — no hard FK across services. | | `occurred_at` | timestamptz | When the user clicked Extract. | | `recorded_at` | timestamptz NOT NULL DEFAULT now() | When the DB wrote the row. | No `period_end` column — period bounds are always derivable from `period_start` + "1 month" (anniversary billing, spec §4). Storing both risks drift; compute the end when rendering. Indexes / constraints: - `UNIQUE (source_type, source_id)` — one charge per record, one per report. Retries and double-clicks replay the same row; no cross-record collision possible. This is the idempotency guarantee. - `(subscription_id, period_start, kind)` — the summation index. Contract B's balance read uses it. **Immutability:** application role gets `INSERT` and `SELECT` only. A separate migration role owns `ALTER`. No `UPDATE` or `DELETE` grant anywhere. Period "close" is implicit — compute-on-read rolls `current_period_start` forward on the subscription row (Step 2), so new writes stamp the new period; old rows keep the old `period_start` forever. **No materialized per-period summary at launch.** Sub-second SUM over a few thousand rows per subscription per month, backed by the composite index, is fine at our scale. A `(subscription_id, period_start, kind, total)` cache refreshed via trigger is a future optimization if reads get slow. --- ## Step 4 — Backfill existing users to Starter + `charge_exempt=true` (run from authentication-service) **Problem:** the moment `subscriptions` is in production, every existing user needs a row — Contract B reads against a missing row would 404. billing-service doesn't own the user directory, and authentication-service has **no local users table** either: Kratos is the identity authority. The backfill must iterate identities from Kratos's Admin API, not from a SQL table. **Solution:** one-shot script in authentication-service pages through Kratos's `GET /admin/identities` and POSTs `/subscriptions/register-user` on billing-service with `charge_exempt=true` **and** `tc_accepted_version=` for each identity — the same endpoint the Kratos post-registration webhook uses (Step 5), but the migration flag routes to the Starter plan and stamps the T&C version carried in Phase 6's migration email (sent 14 days before cutover). No Stripe provisioning happens here — that's Checkout's job (Phase 3), and migrated users haven't picked a paid plan yet. `charge_exempt=true` on the row means overage billing will be suppressed once enforcement turns on in Phase 3. Idempotent end-to-end: the UNIQUE constraint on `subscriptions.user_id` + Step 5's early-return on duplicate make re-running safe. ### Script Pattern follows the existing `scripts/migrate_identity_metadata.py` in authentication-service (same Kratos-pagination approach, same httpx + `asyncio.Semaphore` shape). ```python # authentication-service/scripts/backfill_migrated_subscriptions.py # Run once at cutover. Idempotent — safe to re-run. import asyncio, sys import httpx from src.services import backend_sync # existing billing-service client from src.config import settings async def iter_kratos_identities(client: httpx.AsyncClient): url = f"{settings.KRATOS_ADMIN_URL}/admin/identities?page_size=250" while url: r = await client.get(url, timeout=30) r.raise_for_status() for identity in r.json(): yield identity url = _next_page_url_from_link_header(r.headers.get("link")) async def main(): sem = asyncio.Semaphore(20) errors: list[tuple[str, str]] = [] async with httpx.AsyncClient() as client: identities = [i async for i in iter_kratos_identities(client)] async def register_one(identity): n1_user_id = identity["metadata_public"].get("n1_user_id") email = identity["traits"]["email"] if not n1_user_id: errors.append((identity["id"], "missing n1_user_id in metadata_public")) return async with sem: try: await backend_sync.register_subscription( n1_user_id, email, charge_exempt=True, tc_accepted_version=settings.TC_CURRENT_VERSION, timeout=5, ) except Exception as e: errors.append((n1_user_id, str(e))) log.error("backfill.failed", n1_user_id=n1_user_id, error=str(e)) await asyncio.gather(*(register_one(i) for i in identities)) print(f"processed {len(identities)} identities, {len(errors)} failures") if errors: for uid, err in errors: print(f" {uid} {err}") sys.exit(1) if __name__ == "__main__": asyncio.run(main()) ``` `backend_sync.register_subscription` is a thin new sibling to the existing `create_billing_user` in `src/services/backend_sync.py` — reuses the same `httpx.AsyncClient`, `X-API-Key: settings.BILLING_SERVICE_KEY` header, and 30s client timeout. The `charge_exempt` kwarg defaults to `false` and `tc_accepted_version` defaults to `None` so the real-time registration hook (Step 5) doesn't accidentally mint comped accounts or pre-accept the T&C for new signups — only the migration script passes these flags. **Why stamp T&C at migration.** Phase 6 sends a migration-announcement email 14 days before cutover describing the new subscription model and linking the rewritten T&C. Every migrated user has both received that email and had 14 days with an in-app banner + cancel-before-cutover path. Stamping `tc_accepted_version = settings.TC_CURRENT_VERSION` at backfill treats that 14-day window + continued use as acceptance of the migration-era T&C. No in-app re-prompt surface is built — new signups accept at Checkout (Phase 3 Contract D), the migration cohort was stamped here, and any future T&C revision is handled by the Phase 6 revision runbook (email notice, cancel-before-renewal path — no code). ### Why a script, not a SQL migration A database migration runs per-deploy; this runs exactly once and crosses both a service boundary and an identity-provider boundary. Keeping it as an explicit script makes re-running after a partial failure an obvious operation — the error log names the failed n1_user_ids, rerun the script, Step 5's idempotent early-return handles the successful ones. ### Filtering Kratos's Admin API only lists active identities by default — deactivated / pending identities in recovery states are excluded via Kratos's own lifecycle. The script passes every listed identity to the hook; Step 5 is responsible for the idempotency guarantee. Identities without an `n1_user_id` in `metadata_public` (legacy onboarding gaps) are logged and skipped — they already can't use api-backend, so they can't generate usage either. ### Verification Two checks run during the cutover window: ```bash # authentication-service — count identities from Kratos Admin API curl -s "$KRATOS_ADMIN_URL/admin/identities?per_page=1&page=1" -I | grep -i x-total-count ``` ```sql -- billing-service DB — every migrated user lands on Starter with charge_exempt=true SELECT count(*) FROM subscriptions WHERE charge_exempt = true AND plan_id = (SELECT id FROM plans WHERE code = 'starter'); ``` Counts must match (excluding the skipped "missing n1_user_id" log lines). Any gap → the script's error log names the n1_user_ids; rerun the script against those. ```sql -- billing-service DB — at cutover no migrated user has Stripe IDs yet -- (they haven't gone through Checkout). This is a one-shot migration invariant, -- not a property of charge_exempt — later, an ops-comped user with charge_exempt=true -- and a live Stripe subscription is a valid state. SELECT count(*) FROM subscriptions WHERE charge_exempt = true AND (stripe_customer_id IS NOT NULL OR stripe_subscription_id IS NOT NULL); -- expected at cutover: 0. ``` ### Concurrency with Step 5's real-time hook The script runs at cutover — the same window Step 5 starts accepting new registrations via the Kratos post-registration webhook. If an identity registers while the script is running, the webhook writes first, the script's call finds the row via the early-return path, returns the existing subscription — no duplicate. The UNIQUE constraint on `subscriptions.user_id` is the DB-level guarantee. --- ## Step 5 — Registration hook for new users (synchronous, blocking) **Problem:** users created after the backfill won't have a subscription — the backfill is a one-shot migration, not a recurring process. authentication-service already blocks Kratos's post-registration webhook on calls to api-backend (`create_user`) and billing-service (`create_billing_user` → existing `POST /users` in billing-service). The existing `POST /users` endpoint writes a LiteLLM user + `users` row but returns **409 on duplicate** (not idempotent) — we can't just extend it to also insert a subscription row, because the backfill would 409 on every existing user. **Solution:** new sibling endpoint on billing-service, `POST /subscriptions/register-user`, idempotent by design. authentication-service's registration webhook gains it as a **third blocking step** after `create_billing_user`. If the call fails after bounded retries, the webhook returns non-2xx to Kratos → the user sees a signup failure and retries. No half-created state. Auth: `Depends(verify_api_key)` — same dependency billing-service already uses on every other N1-API-key-only route (defined at `app/auth.py:49`). Header: `X-API-Key`. No JWT path. The Step 4 backfill script uses the same endpoint with the same credential. ```python # billing-service: POST /subscriptions/register-user # body: { # n1_user_id: UUID, # email: EmailStr, # plan_code: Literal["starter", "free_trial"] = "starter", # Phase 5's trial signup passes "free_trial" # tc_accepted_version: str | None = None, # charge_exempt: bool = False, # true only from the Step 4 backfill script # } # Idempotent on n1_user_id. 200 whether it wrote a new row or returned an existing one. async def register_user(body: RegisterUserRequest, db: AsyncSession = Depends(get_db), _auth: str = Depends(verify_api_key)): existing = await subscriptions_repo.get_by_user(db, body.n1_user_id) if existing: return subscription_response(existing) plan_id = await plans_repo.get_id_by_code(db, body.plan_code) # T&C: accept only if client passed the current version. Stale or missing → null # (user will accept later via Contract D or the trial signup path). tc_version = body.tc_accepted_version if body.tc_accepted_version == settings.tc_current_version else None row = await subscriptions_repo.create( db, user_id=body.n1_user_id, plan_id=plan_id, status="active", charge_exempt=body.charge_exempt, current_period_start=utcnow(), # compute-on-read rolls periods forward tc_accepted_version=tc_version, tc_accepted_at=utcnow() if tc_version else None, ) return subscription_response(row) ``` **`charge_exempt` has no effect on this endpoint.** The endpoint never calls Stripe — it only writes a DB row. `charge_exempt` is stamped onto the row and read later by the Phase 3 overage branch of `/quota/charge` to decide whether to post metered usage to Stripe. Quota enforcement (Phase 3) runs normally for charge-exempt users against their plan's quota; the only difference between a charge-exempt user and a non-exempt user is that overage Stripe posts are suppressed for the former. An admin flipping `charge_exempt=false` on a migrated user (Step 10) is a signal that they should pick a paid plan — they still have no Stripe subscription, so until they complete Checkout their overages can't actually be billed. Step 10 surfaces that prompt via `status=incomplete`. **`tc_accepted_version` is optional.** The Kratos post-registration hook doesn't pass one — the new user hasn't seen the T&C yet; they'll accept on `PickPlanPage` (paid) or the trial signup form (free trial). The Step 4 backfill script passes the current version (migration-era T&C accepted via the Phase 6 email + 14-day window). Phase 5's trial path passes the version directly, since trial signup accepts T&C *before* registration completes. Server stamps `tc_accepted_at = utcnow()` when a version is accepted — client-supplied timestamps are ignored. Submitting a stale `tc_accepted_version` is treated as no acceptance (silently null) rather than a hard error — the user gets prompted at Checkout (Contract D) when they next pay. **Why a new endpoint, not an extension of `POST /users`:** the existing endpoint returns 409 on duplicate. Making it idempotent would change its contract for every existing caller. Keeping the subscription write in a separate, explicitly-idempotent endpoint is cheaper and safer — the webhook just makes one extra call. **Reliability posture:** - **Blocking.** If billing-service returns non-2xx after retries, Kratos's webhook returns 5xx → Kratos surfaces the signup failure. No half-created state. - **Bounded client-side retry** in auth → billing: 3 attempts, 500ms / 2s / 8s backoff, 2s per-call timeout. Matches the pattern already in `src/services/backend_sync.py`. - **Idempotent on `n1_user_id`.** `UNIQUE` on `subscriptions.user_id` enforces it at the DB level; the early return is the fast path for retries. **Why no reconciliation cron / outbox:** every successful registration creates a subscription row by construction. A billing-service outage blocks new signups during the outage — acceptable at n1's signup rate, and far cleaner than a polling scan that masks silent webhook failures. **Failure visibility:** auth-side ERROR log on exhausted retries carries `n1_user_id` + last billing-service response code. SigNoz alert → on-call investigates. --- ## Step 6 — Contract C — `POST /quota/charge` (endpoint shell + write path) **Problem:** for Contract B to show real usage, something has to write `quota_events` rows when a doctor extracts or generates. No endpoint does that yet. **Solution:** one endpoint, `POST /quota/charge`, that writes one `quota_events` row per call inside a single transaction. Phase 2 owns the endpoint shell, input validation, auth, the transaction boundary, the row-write path, and observability. Phase 3 fills in the paid-plan + overage + 429 branches; Phase 5 fills in the lifetime-cap branch. The union is what ships to prod. `charge_exempt` is orthogonal to the charge path — it only suppresses the overage Stripe post in Phase 3's overage branch; quota checks, 429s, and row writes are identical regardless of the flag. ### Auth **N1 API key (service-to-service from api-backend) only.** User JWT is rejected — a doctor's JWT reaching this endpoint directly would let them mint their own overage. Verified by `Depends(verify_api_key)` at the route (same dependency billing-service uses today at `app/auth.py:49`; header `X-API-Key`). `verify_jwt_or_api_key` — the dual-mode dep on some existing billing-service routes — is NOT used here. ### Request ```json { "user_id": "uuid", "kind": "pages", "count": 12, "source_type": "record_request", "source_id": "uuid" } ``` ### Input validation (Pydantic, fail at boundary) | Field | Rule | On violation | |---|---|---| | `user_id` | UUID | 422 | | `kind` | `Literal["pages", "reports"]` | 422 | | `count` | `int`, `1 <= count <= 1000` (upper bound sanity — a larger batch is either a bug or abuse; real extracts sit well under this) | 422 | | `source_type` | `Literal["record_request", "report_gen"]` | 422 | | `source_id` | UUID | 422 | ### Transaction (single atomic unit) All steps run inside one DB transaction. The lock acquired in step 2 is held through the INSERT in step 3. 1. **Idempotency lookup.** `SELECT … FROM quota_events WHERE source_type = :t AND source_id = :id` — hit → return the existing row as a 200, no mutation. `UNIQUE(source_type, source_id)` makes this the natural idempotency unit: one row per record_request, one per report_gen. 2. **Lock + fetch subscription.** `SELECT … FROM subscriptions WHERE user_id = :user_id FOR UPDATE`, joined to `plans`. Compute `(period_start, _)` via `compute_current_period(sub)` for use in steps 3–4. - No row → 404 `subscription_missing` (ERROR log; registration hook should prevent this — alert fires). 3. **Branch on plan:** - `plan.pages_quota IS NULL` (Enterprise custom contract) → INSERT `quota_events` row (`overage_count = 0`, computed `period_start`). Return 200 with `remaining = null`. - Paid plan (Starter / Professional / Practice — includes every migrated user, since they're on Starter) → **Phase 3 branch** (filled in Phase 3's Step 3: quota check under the same `FOR UPDATE`, 429 on exhaustion, overage when toggled on). - `plan.is_lifetime_cap = true` → **Phase 5 branch** (filled in Phase 5's Step 3). ### Responses **200 (Enterprise / unlimited):** ```json { "event_id": "uuid", "in_quota_count": 12, "overage_count": 0, "overage_charge_cents": 0, "remaining": { "pages": null, "reports": null } } ``` `remaining: null` → frontend renders "Unlimited." **404:** `{ "error": "subscription_missing", "user_id": "..." }` — registration gap, alert. **422:** Pydantic's default validation payload — the caller (api-backend) gets enough to log the bad input. ### Rate limit 60 req/min per `user_id` in the body (not per API key — a bug in api-backend mustn't be able to burn one specific doctor's quota even if the service credential is fine). Exceeded → 429 with `Retry-After: ` header. ### Observability Every call emits one structured log line on success: ``` charge.committed user_id=... kind=pages count=12 overage_count=0 event_id=... plan=starter charge_exempt=true duration_ms=8 replayed=false ``` `replayed=true` when the `(source_type, source_id)` lookup hit an existing row. Failures log at ERROR with the same shape plus `error_code`. SigNoz dashboards read off `charge.committed` + `charge.error_code` labels; alerts on `subscription_missing` (should be zero) and elevated 429 rate. ### Why `SELECT FOR UPDATE` on the subscription row, not the events table Concurrent extracts for the same doctor serialize cleanly on her subscription row; different doctors don't contend. Phase 3's quota check (is there budget?) reads committed events under the same lock, so no read/write race. --- ## Step 7 — Inject `/quota/charge` into api-backend **Problem:** Contract C is live but nothing calls it. `record_requests` and `report_gen_requests` rows are created without writing `quota_events` — Contract B would read `SUM = 0` forever. **Solution:** insert one `billing_client.charge` call per record into the Phase 1 `POST /records/batch-extract` loop (before `queue_parse_job`), and one into the report-generation endpoint (`routes/reports.py` → `POST /generate`, before the Cloud Run enqueue). Single cutover: Phase 3's paid-plan branch ships in the same deploy, so every charge — including migrated users on Starter — runs through the quota-enforcement path from day one. Migrated users have `charge_exempt=true`, which (in Phase 3 consumer semantics) suppresses only the overage metered-usage post to Stripe; in-quota enforcement, 429s, and row writes behave identically to any other Starter subscriber. api-backend already has a `services/billing_client.py` for other billing calls — extend that module with a `charge(...)` function. It uses `aiohttp.ClientSession`, header `X-API-Key: settings.BILLING_SERVICE_API_KEY`, matching the pattern already in place. A `BillingUnavailable` exception (Phase 3 will catch it) is added as a sibling of the existing `ExternalServiceError` base — billing_client maps 5xx / network failures → `BillingUnavailable`, 409 / 404 / validation → `ExternalServiceError`. ```python # routes/records.py — INSIDE the loop shipped in Phase 1. for record in records: if record.status != "SYNCED": results.append({"record_id": record.id, "status": "not_extractable"}) continue charge = await billing_client.charge( user_id=caller.caller_id, kind="pages", count=record.page_count, source_type="record_request", source_id=record.id, timeout=2, ) # QuotaExceeded/BillingUnavailable handling layered on in Phase 3 Step 6. await queue_parse_job(record.id) results.append({ "record_id": record.id, "status": "queued", "event_id": charge.event_id, }) ``` Report-generation endpoint: same pattern with `kind="reports", count=1, source_type="report_gen", source_id=`. **Why charge-at-extract, not charge-at-upload:** matches spec §5. A doctor who uploads and then cancels never gets a `quota_events` row. Reprocess (Phase 3) replays the same `(source_type, source_id)` tuple → `UNIQUE` hits the existing row, no new charge. **Source ownership — no cross-user charges.** The `records` iterated here are the ones Phase 1's `POST /records/batch-extract` already loaded via `records WHERE owner_user_id = caller.caller_id` (the existing scoping on the endpoint). A `record_id` from a request body that doesn't belong to the caller never enters the loop — it 404s before reaching `billing_client.charge`. Same invariant on the report-generate endpoint: the `gen_id` is created inside the request handler from the caller's JWT, not accepted from the client. Combined with billing-service's `user_id`-body rate limit (60 req/min — master spec *User isolation*), no request path allows user A to write a `quota_events` row against user B's subscription. --- ## Step 8 — Contract B — `/quota/status` via api-backend **Problem:** frontend has no endpoint returning plan + period + usage + remaining as one read — stitching on the client risks inconsistent views. Separately, the frontend must not hit billing-service directly — every user-facing billing call routes through api-backend, the same pattern `/quota/charge` already uses. **Solution:** two endpoints, one boundary each. - **api-backend** exposes `GET /billing/quota-status/me` — user JWT, resolves user_id from `jwt.sub`, calls billing-service with N1 API key, passes the response back unchanged. - **billing-service** exposes `GET /quota/status/{user_id}` — N1 API key only. Composes plan + period + usage + remaining from its own tables. No cross-service call. ### billing-service — `GET /quota/status/{user_id}` Auth: `Depends(verify_api_key)`. User JWT is rejected — same auth rule as `/quota/charge` and `/subscriptions/register-user`, so every route billing-service introduces in this rollout has exactly one auth path. Usage read (where `:period_start` is `compute_current_period(sub).start`): ```sql SELECT kind, COALESCE(SUM(count), 0) AS total_count, COALESCE(SUM(overage_count), 0) AS overage_count FROM quota_events WHERE subscription_id = :sub_id AND period_start = :period_start GROUP BY kind; ``` Derived in Python: ```python in_quota = row.total_count - row.overage_count remaining = None if plan.pages_quota is None else max(0, plan.pages_quota - in_quota) ``` **Forward-correct:** in Phase 2 every row has `overage_count = 0`, so this reduces to `SUM(count)`. Phase 3 starts writing `overage_count > 0` rows and `remaining` stays accurate without touching Contract B. Response: ```json { "user_id": "uuid", "plan": { "code": "professional", "name": "Professional", "pages_quota": 2000, "reports_quota": 20, "pages_overage_cents": 15, "reports_overage_cents": 2200, "is_lifetime_cap": false }, "usage": { "pages_used": 412, "reports_used": 14 }, "remaining": { "pages": 1588, "reports": 6 }, "period": { "start": "2026-04-01T00:00:00Z", "end": "2026-05-01T00:00:00Z", "days_remaining": 9 }, "status": "active", "tc": { "current_version": "2026-05-01", "user_accepted_version": "2026-05-01" } } ``` Rules: - `plan.pages_quota = null` → `remaining.pages = null` → frontend renders "Unlimited" (Enterprise only). - `plan.pages_overage_cents = null` → plan has no overage path; frontend hides overage UI (Phase 3's OverageToggleCard, "Y will be billed as overage" copy). - `is_lifetime_cap = true` → `period: null`; frontend renders lifetime layout. Phase 5 wires this branch — in Phase 2 no user has `is_lifetime_cap=true`, so the branch is never hit. - `period.start` / `period.end` come from `compute_current_period(sub)`; `days_remaining` = `max(0, (end - now).days)`. - Response exposes a derived `billing_setup_complete: bool = charge_exempt OR stripe_subscription_id IS NOT NULL OR plan.is_lifetime_cap`. True means the user is in a "settled" subscription state (a real Stripe subscription, an ops-comped account, or an active free trial) and the frontend should not prompt them to pick a plan. The raw `charge_exempt` flag is not in the response — only the derived state. - `tc.current_version` — read from billing-service env (`TC_CURRENT_VERSION`, date-stamped, e.g. `"2026-05-01"`). Single source of truth for what "current" means. - `tc.user_accepted_version` — mirror of `subscriptions.tc_accepted_version`. Null only between Kratos registration and plan selection (brand-new paid signups pre-`PickPlanPage`); every post-migration user has a stamped version. - Frontend uses the pair for diagnostic rendering only ("you agreed to T&C 2026-05-01") — Contract D enforces version currency at Checkout. No in-app re-prompt surface in this rollout. Errors: - No subscription for `user_id` → 404 `subscription_missing` + ERROR log + alert. Step 4's backfill + Step 5's hook should have caught everyone; 404 fires → investigate. ### api-backend — `GET /billing/quota-status/me` Auth: user JWT (`Depends(get_caller)`). ```python @router.get("/billing/quota-status/me") async def quota_status_me(caller: CallerContext = Depends(get_caller)): try: return await billing_client.quota_status(caller.caller_id, timeout=2) except BillingUnavailable: raise HTTPException(503, "billing temporarily unavailable") ``` Pass-through shape — api-backend doesn't rewrite the response. If billing-service 404s (missing subscription) api-backend returns 500 + alerts; that's a backend bug the user can't fix and shouldn't see. **No `/billing/quota-status/{user_id}` on api-backend.** No legitimate use case for a user to read another user's quota. Admin/ops use billing-service's endpoint directly with the ops API key. --- ## Step 9 — Frontend full swap: delete legacy billing UI, ship new quota surface **Problem:** Contract B returns plan + period + remaining, but the current frontend has an entire billing surface — `BillingPage` with balance widgets, "credits remaining" copy, credit top-up flows, "insufficient funds" error handling on upload/extract — all driven off the pre-subscription LiteLLM-budget model. Leaving both UIs live during cutover splits the user's mental model ("is this my credit balance or my quota?"), and the old UI would render stale data against a source of truth that no longer exists post-cutover (api-backend still has the legacy budget code live until Phase 6, but nothing user-facing reads it). **Solution:** delete the legacy surface in the same PR stack and ship the new one as the only billing UI. Add `QuotaStatusCard` to the new `BillingPage` (the detailed view) **and** a compact `QuotaIndicator` in the app header (always visible, links to billing). Both driven by the same Contract B response via one shared `useQuotaStatus` hook. No feature flag, no toggle, no "keep old surface as fallback" — the old surfaces are gone as of this PR. ### What gets deleted from react-frontend - Old `BillingPage` layout + balance components. - `useBalance` / `useCredits` / `useBilling` (old) hooks — grep and remove. - Credit top-up / "add credits" UI. - "Insufficient funds" / "balance exhausted" error strings in upload / extract / report-gen error handlers — Phase 3's `ReadOnlyBanner` + upgrade-modal replace them, but at Phase 2 the error can't surface because enforcement isn't live yet. - Any route, sidebar entry, or menu item that linked to the old billing UI — repointed to the new `BillingPage` or removed. Verification: `rg '(useBalance|useCredits|insufficient_funds|add_credits|credit_balance)'` across `src/` returns zero hits. ### What gets added ### `useQuotaStatus` hook ```ts // src/features/billing/hooks/useQuotaStatus.ts import { useQuery } from "@tanstack/react-query"; import { useAuthStore } from "@/stores/authStore"; import { api } from "@/api/client"; // the api-backend axios instance (/proxy-api) import type { QuotaStatus } from "@/api/types"; export function useQuotaStatus() { const user = useAuthStore((s) => s.user); return useQuery({ queryKey: ["quota-status", user?.n1_user_id], // n1_user_id — matches Contract B's key, same convention as useProcedures / patient hooks queryFn: async () => (await api.get("/billing/quota-status/me")).data, staleTime: 30_000, enabled: !!user?.n1_user_id, // don't fire before auth resolves or for users missing n1_user_id }); } ``` Cache key uses `n1_user_id` (the N1 backend ID) rather than `user.id` (the Kratos identity ID) because Contract B's `GET /billing/quota-status/me` resolves the user from `jwt.sub` → N1 backend ID server-side, and every other user-scoped query in the app keys on `n1_user_id` (`PatientDashboardPage`, `useProcedures`). Using the Kratos id here would silently diverge from the rest of the cache graph. ### Invalidation on user actions `staleTime: 30_000` alone would leave the counter lagging up to 30s after an Extract click. Spec §11 requires real-time updates. Fix: invalidate the query at every user action that mutates quota. In Phase 2, that's the batch-extract success handler (wired into Phase 1's code path) and the report-generate success handler: ```ts onSuccess: async () => { await queryClient.invalidateQueries({ queryKey: ["quota-status", user?.n1_user_id] }); } ``` Same pattern already in use in `features/reports/hooks/useShareLinks.ts` and `features/medical-records/UploadModal.tsx`. Phase 3's overage toggle + reprocess mutations inherit it. ### Render decision rule One discriminator function picks the layout — explicit so the branch choice isn't lost in mockups: ```ts function quotaLayout(plan: Plan): "trial" | "unlimited" | "paid" { if (plan.is_lifetime_cap) return "trial"; if (plan.pages_quota === null) return "unlimited"; return "paid"; } ``` All numbers rendered with `Intl.NumberFormat("en-US")` — `"2,000"` not `"2000"`. ### `QuotaIndicator` (header pill) Compact pill in the app shell next to the user menu. Rendered as a `` (keyboard-focusable, screen-reader-announced as a link — not a `
`). Shares the `useQuotaStatus` cache with `QuotaStatusCard`, so the pill and the billing-page card stay in lockstep. - Paid: `412 / 2,000 pages · 14 / 20 reports` - Enterprise: `Unlimited` - Free Trial: `77 / 100 pages (lifetime)` - **Loading / error: collapse to nothing.** The header must never fail — a broken pill blocking the entire app shell would be worse than a momentarily missing counter. ### `QuotaStatusCard` (billing page detailed view) Loading: skeleton. Error: inline error message with a Retry button — this is the surface the user navigated to for billing data; silently hiding it confuses them. Render branches: ``` Paid plan (plan.pages_quota != null, is_lifetime_cap = false): ┌──────────────────────────────────────────────┐ │ Your plan: Professional │ │ Pages ▓▓▓▓▓▓▓░░░░░░ 412 of 2,000 │ │ Reports ▓▓▓▓▓▓▓▓▓░░░░ 14 of 20 │ │ Resets Apr 30 · 9 days remaining │ └──────────────────────────────────────────────┘ Enterprise (plan.pages_quota = null): ┌──────────────────────────────────────────────┐ │ Your plan: Enterprise │ │ Pages used this month: 412 │ │ Reports generated: 14 │ │ No monthly limit │ └──────────────────────────────────────────────┘ Free Trial (plan.is_lifetime_cap = true): ┌──────────────────────────────────────────────┐ │ Your plan: Free Trial │ │ Pages used (lifetime): 23 of 100 │ │ Reports generated: 0 of 1 │ └──────────────────────────────────────────────┘ ``` Migrated users (Starter + `charge_exempt=true`) render under the "paid plan" branch — visually identical to a paying Starter subscriber. The `charge_exempt` flag is not exposed to the frontend. --- ## Step 10 — Admin endpoint: toggle `charge_exempt` **Problem:** the migration writes `charge_exempt=true` for every existing user, and ops may want to exempt specific users (internal team, demo, partner) from overage billing permanently. Without an admin endpoint, both flips require hand-running SQL against the billing-service DB — error-prone and not auditable. **Solution:** a single admin-authenticated endpoint on billing-service. When flipping false on a user with no Stripe IDs (typical migration case), also set `status = 'incomplete'` so the frontend surfaces the `ReadOnlyBanner` → `PickPlanPage` → Checkout path on that user's next refresh — without a Stripe subscription, overage charges can't be billed even once `charge_exempt` is off, so the user needs to complete Checkout before overage enforcement is meaningful. ### billing-service — `PATCH /admin/subscriptions/{user_id}/charge-exempt` Auth: admin API key (separate credential from the regular N1 API key; scoped to admin-only ops endpoints). Reuses the existing `verify_api_key` dependency pattern but checks a distinct `ADMIN_API_KEY` env var to prevent a compromised api-backend credential from granting comp status. Request: `{ "enabled": bool }`. Response: the updated subscription row. ```python @router.patch("/admin/subscriptions/{user_id}/charge-exempt") async def set_charge_exempt( user_id: UUID, body: SetChargeExemptRequest, db: AsyncSession = Depends(get_db), _auth: str = Depends(verify_admin_api_key), ): sub = await subscriptions_repo.get_by_user_for_update(db, user_id) if not sub: raise HTTPException(404, "subscription_missing") sub.charge_exempt = body.enabled # Flipping false on a user with no Stripe IDs → they still have no way to be # billed for overage. Mark as incomplete so the frontend surfaces # ReadOnlyBanner → PickPlanPage → Checkout, which provisions a Stripe sub. if not body.enabled and sub.stripe_subscription_id is None: sub.status = "incomplete" # Flipping true on a user with a live Stripe subscription is valid — base-plan # invoicing continues via Stripe; only overage metered-usage posts are # suppressed. No 409, no special handling. await db.commit() log.info( "admin.charge_exempt_set", user_id=str(user_id), enabled=body.enabled, status_after=sub.status, ) return subscription_response(sub) ``` No api-backend pass-through — this is not a user-facing endpoint. Admin-dashboard calls billing-service directly with the admin API key (same gateway exception Stripe webhooks use). --- ## Step 11 — Admin dashboard `charge_exempt` toggle **Problem:** Step 10 ships the endpoint, but ops has no UI for it — flipping the flag from psql is fine for one-offs but won't scale to migration cleanup or comp-account management. **Solution:** single checkbox on the subscription detail view in the admin-dashboard repo. Scope: extend the existing subscription detail page (or add one if none exists yet — check before writing). Checkbox labelled "Charge exempt (suppress overage billing)" with helper text "Base plan continues invoicing normally via Stripe. Only overage metered-usage posts are suppressed."; on flip, calls `PATCH /admin/subscriptions/{user_id}/charge-exempt` with the admin API key. Optimistic update; revert + toast on error. Show the current `status` + `stripe_subscription_id` nearby so the operator has full context before flipping (e.g. flipping false on a no-Stripe user transitions them to `status=incomplete`). Non-goal: no bulk toggle UI, no filter by `charge_exempt`. Migration cleanup is individual-user work; if it's not, we'd revisit the whole complimentary strategy rather than bulk-flip flags. --- ## Accuracy invariant Two derived quantities from `quota_events`, both scoped to `(subscription_id, period_start, kind)`: - **Total committed** = `SUM(count)` — pages/reports the doctor authorized at Extract click, including any that overflowed into overage. In Phase 2 this equals the in-quota portion (every row has `overage_count = 0`); in Phase 3+ it includes overage. - **In-quota committed** = `SUM(count - overage_count)` — drives Contract B's `remaining = plan.quota - in_quota_committed`. `SUM(record_requests.page_count)` over the same user + period in api-backend is an upper bound on *total committed* — it counts uploaded-but-not-extracted records that never reached `/quota/charge`. Divergence between the two is expected. Divergence between Phase 1's hand-sample (uploaded pages) and Contract B's display (committed pages) is not a bug. ## Testing **Unit** - Contract C write path: one row per call with `overage_count = 0`; duplicate `(source_type, source_id)` returns existing row without insert. - Contract B returns `pages_quota = null → remaining = null` for Enterprise and correctly fetches both `SUM(count)` and `SUM(overage_count)` (forward-correct for Phase 3). - `compute_current_period` rolls correctly across month boundaries, leap years, and the "anchor day > month length" edge (e.g. anchor on the 31st in a 30-day month). - Backfill script: idempotent re-run produces 0 new rows; every row created with `charge_exempt=true` and `plan_id = starter`; semaphore bounds parallel requests to 20; error accumulator reports failed user_ids. - Admin endpoint: flipping `charge_exempt=false` on a no-Stripe user sets `status='incomplete'`; flipping `charge_exempt=true` on a user with `stripe_subscription_id` succeeds without status change (base-plan invoicing continues); admin API key required (regular API key → 401). - `useQuotaStatus` hook: cache key changes on user change; query disabled when no user. **Integration** - New user registers → auth registration hook fires → subscription exists before the endpoint returns 2xx, with `charge_exempt=false`. Hook failure after 3 retries → registration returns 5xx (no half-created state). - Existing user after backfill → Contract B returns Starter shape (500 pages / 5 reports); no Stripe IDs on the row. - Extract click on a 12-page record → one `quota_events` row → Contract B shows `pages_used = 12`. Second extract on same record id (retry) → no new row, same response. - Report-generate click → one row with `kind = "reports", count = 1` → Contract B `reports_used += 1`. - Post-Extract, `QuotaIndicator` in the header updates without a tab refresh (invalidation fires on the mutation's success handler). - Admin flips `charge_exempt=false` on a migrated user → next Contract B refresh shows `status=incomplete` → frontend surfaces `ReadOnlyBanner` (Phase 4) with the resume-Checkout CTA. **Auth boundary** - billing-service `GET /quota/status/{user_id}` with user JWT → 401 (never accepts JWT on any route). - billing-service same endpoint with N1 API key → 200. - api-backend `GET /billing/quota-status/me` with no / expired JWT → 401. - api-backend with valid JWT → Contract B for exactly that user; no path for a user JWT to read another user's data. **Failure handling** - billing-service down → api-backend `/billing/quota-status/me` returns 503; frontend pill collapses, `QuotaStatusCard` shows inline error + Retry. - `subscription_missing` from billing-service → api-backend 500 + alert (not leaked to user). ## Done when - 100% of users have a subscription row (Step 4's verification queries return 0 missing, 0 non-Starter among migrated users). - All five plan rows seeded with correct values including overage rates. - Every extract and report-generate click in staging writes a `quota_events` row. - billing-service rejects user JWT on every route (validated with a staging probe). - api-backend `GET /billing/quota-status/me` returns Contract B shape for a JWT user. - `QuotaStatusCard` renders correctly for paid, unlimited, and lifetime-cap layouts in staging with real usage numbers. - `QuotaIndicator` pill renders in the app shell and updates without a tab refresh after an Extract click. - Registration hook failure after retry exhaustion surfaces as a 5xx from auth — no silent half-created users. - Admin dashboard can flip `charge_exempt` per user; flip off on a no-Stripe user cleanly transitions them to `status=incomplete`. - Legacy billing UI fully removed from react-frontend (`rg '(useBalance\|useCredits\|insufficient_funds\|add_credits\|credit_balance)' src/` returns zero). Old `BillingPage`, balance widgets, and credit top-up flows gone. Sidebar / menu entries repointed or removed. ## Depends on - Phase 1 (uses `record_requests.page_count` and the `/records/batch-extract` endpoint). ## Feeds - Phase 3 (adds paid-plan branches to `/quota/charge`, overage, Stripe, checkout). - Phase 4 (reads `subscriptions.status` for `ReadOnlyBanner`). - Phase 5 (adds lifetime-cap branch to `/quota/charge`).