AI & model access — N1 IT Landscape

N1 IT Landscape — The Blueprint · CF-Access internal

AI & model access

N1 IT Landscape · Owner: Arun Venkataraman · source of truth: markdown SSOT

CURRENT-STATE as of 2026-06-09 · part of the N1 IT Landscape (start at Overview). Owner: Arun Venkataraman (Principal Engineer).

How services reach AI models — everything brokered through LiteLLM, running on Vertex AI with Bedrock backup.

🔒 All model access is brokered through LiteLLM — no N1 service holds raw provider keys.

LiteLLM runs in-cluster in the llmproxy namespace, three deployments : staging (V20260515_2), production (V20260506_1), and the AI-cluster one that backs litellm.n1-research.com (development account). Each has its own Aurora litellm-{env} DB and is Cloudflare-Access-gated. A litellm-dlq-processor sidecar handles dead-letter retries.
Where the models actually run: LiteLLM forwards Claude + Gemini requests to GCP Vertex AI , with AWS Bedrock as backup. There are two separate GCP projects — one for staging + developers , one dedicated to production that nobody else can access.
Sign-in with Google for the product uses two GCP OAuth clients (one per project, staging/prod) — separate from the Cloudflare Access Google IdP (§4.2/§6.6).
Developer key self-service (30-day expiry, invalidate-on-reissue):
- A LiteLLM Slack app lets developers mint LiteLLM API keys that expire every 30 days.
- The n1-api-slack-bot (staging) lets developers mint N1 service tokens for staging.
- 🔒 When a developer requests a new key (either type), all their existing keys are invalidated/expired first , then a new one is issued.
release-scribe (CI) and other AI-powered services call models through this same LiteLLM proxy (Gemini 2.5 Flash by default for release notes).