AI Tier
AI Tier — Gemini on Vertex AI
Decision: the entire AI tier runs on Google Gemini via Vertex AI. One Google Cloud BAA covers reasoning, embeddings, and voice — in the same cloud as the app, with PHI staying in-boundary. We break a subcomponent out to another provider only when a concrete, evidenced need forces it (see Escalation). The full provider bake-off lives in the Compute Stack Plan (Notion).
The AI tier is inside the trust zone — unlike the durable-execution layer, it does receive PHI (that's the point of clinical reasoning), under the GCP BAA. It is not on the PHI-free service list. PHI must still be scrubbed from logs and spans.
Subcomponents
Each is separable — the boundary and BAA are shared, so swapping one (e.g. the reasoning model) doesn't disturb the others.
| Subcomponent | Service | Model / API | Notes |
|---|---|---|---|
| Reasoning & agent orchestration | Vertex AI | Gemini 3.1 Pro | The agent brain — tool-calling + clinical-context reasoning. Runs as a durable workflow (see Backend). |
| Fast / high-volume | Vertex AI | Gemini 3.1 Flash / Flash-Lite | Cheap, high-throughput tasks: classification, extraction, routing, summaries. |
| Embeddings & retrieval | Vertex AI + Neon | gemini-embedding-001 + pgvector | In-boundary embeddings; vector search lives in Neon under RLS (see Data Pipeline). |
| Voice / realtime | Vertex AI | Gemini Live API + Chirp (STT/TTS) | Low-latency speech for the voice agent. Phase-2 — see the Voice plan (Notion). |
Client SDK: the Google Gen AI SDK (google-genai), configured for Vertex. All three subcomponents — reasoning, embeddings, and voice — go through the single google-genai Python SDK, initialised in Vertex mode (genai.Client(vertexai=True, project=..., location=...)), never the ai.google.dev Gemini Developer API (not BAA-covered). It's a runtime dependency of backend/worker (the agent runtime) and any embedding jobs; see Libraries.
Why one provider
- One boundary, one BAA. Reasoning + embeddings + voice all sit inside the Google Cloud BAA, in the same cloud as the Cloud Run app — no third-party inference hop, no second contract.
- Gemini uniquely closes two gaps that bite single-model shops: in-GCP HIPAA-covered embeddings (
gemini-embedding-001) and a realtime voice path with named-covered STT/TTS (Chirp). Claude, for example, offers neither and would force a second provider for both. - Fewer moving parts beats marginal per-task model quality at this stage.
HIPAA posture (required)
- BAA: Vertex AI is covered under the Google Cloud BAA (on the covered-products list as "Generative AI on Gemini Enterprise Agent Platform"). Never use the
ai.google.devGemini Developer API or AI Studio — not BAA-covered. - Region: pin to a US region, not
global(global lacks CMEK / Access Transparency). - Retention: request abuse-logging opt-out / Zero Data Retention (default abuse logging retains prompts up to 90 days).
- Isolation: VPC-SC + CMEK; a separate GCP project for HIPAA workloads.
- Schema hygiene: never put PHI in JSON-schema field names / enums / regex (schemas are cached without PHI protections).
Escalation — break out only when forced
Default everything to Gemini/Vertex. Reach for another provider only when a specific, evidenced need demands it — and even then, prefer to stay inside a BAA boundary. Both of these are subcomponent-level swaps, not a tier rewrite:
- Reasoning quality — if an eval shows Gemini losing materially on our concierge / clinical tasks, escalate that subcomponent to Claude on Vertex AI (still the same GCP BAA, same boundary).
- Retrieval latency — if RAG traffic needs to be co-located with Neon, run embeddings on Bedrock (us-west-2) (the AWS BAA, same region as the DB).
The bake-off and trade-offs behind this are documented in the Compute Stack Plan (Notion).
Open item — caregiver-scheduling AI is not locked. A care-operations AI layered on WellSky to auto-match and schedule caregivers is an open bake-off: Zingage vs Phoebe vs Alden. It is an operational tool on top of WellSky — distinct from the in-house Gemini agent tier above — so it does not change this AI-tier decision, but it is an open GTM / ops decision to resolve before scaling caregiver operations.