The Compliant AI Gateway
Enterprise LLM spend reached $8.4B in 2025 and essentially none of it is governed at the MSP layer. This paper lays out the control plane a managed service provider needs to resell LLM access safely: per-tenant virtual keys and budgets, model routing, regional egress, prompt caching, audit logs, and PII redaction — with EU AI Act and HIPAA paths designed in from day one.
Key findings →
- Cloudflare AI Gateway has no RBAC; Portkey sells enterprise-direct, not through the MSP channel.
- Per-tenant spend visibility cut one client's LLM bill 38% in the first month.
- PII redaction before the model call is the only pattern that survives a HIPAA audit.
- A shared prompt cache across tenants is a data-leak primitive. Isolate by default.





