AI Gateway

Why Marketing AI Needs a Gateway

Every production AI system needs a control plane between the user and the LLM. For marketing intelligence, the stakes are uniquely high:

The AI Gateway handles all of this as a LangGraph.js state graph — a directed graph where each step is a node, transitions are conditional edges, and the entire execution is traceable.

LangGraph Architecture

Why a Graph, Not If/Else

Traditional approaches use nested conditional logic:

if (isCached(query)) return cache.get(query);
if (hasConfidentialData(query)) return reject();
if (isContentDraft(query)) model = "opus";
// ... more nesting

This becomes unmaintainable as you add guardrails, caching, routing, and tracking. LangGraph replaces it with a visible graph:

classify → cache_check → [hit] → return_cached
                       → [miss] → guardrails → route → llm → brand_check → format → track

Each node is a pure function. Edges are conditional transitions. You can visualize the graph, trace any request through it, and add new nodes without touching existing ones.

State Object

The GatewayState type flows through every node:

interface GatewayState {
  query: string;
  intent: "competitive" | "trend" | "performance" | "content_draft" | "strategic";
  complexity: "simple" | "moderate" | "complex";
  sensitivity: "normal" | "confidential" | "restricted";
  retrievedContext: string;
  llmResponse: string;
  brandCompliance: { score: number; violations: string[] };
  model: string;
  cached: boolean;
  tokensUsed: number;
  latencyMs: number;
}

Each node reads what it needs from state and writes its outputs back. The classify node writes intent, complexity, and sensitivity. The route node reads those to write model. The brand check node reads llmResponse and writes brandCompliance.

The Nodes

Classify

Analyzes the incoming query to determine intent, complexity, and sensitivity. For marketing queries:

Classification drives every downstream decision — which model to use, which guardrails to apply, how to format the output.

Cache Check

Semantic caching: embed the query and check if a similar query (cosine similarity > 0.95) was answered recently. Cache TTLs vary by query type:

Cache hits save 80-90% of the cost and return instantly. For a marketing team running the same competitor analysis multiple times a day, this is critical.

Guardrails

This node runs a series of checks. If any check fails, the query is either sanitized or rejected.

Brand Voice Compliance:

Load the brand guidelines from the data lake. Check that any content in the query or response uses approved terminology, maintains the right tone, and doesn't contradict messaging pillars.

Example: If brand guidelines say "We never use fear-based messaging," flag any query or response containing scare tactics.

Confidential Data Gate:

Detect and block queries that would expose internal data in external contexts. Pattern matching for:

Competitor Mention Sanitizer:

Many brands have policies about naming competitors in public content. The sanitizer:

Route

Select the right model based on classification:

LLM Call

Invoke the selected model with the retrieved context from Module 3. Key patterns:

Brand Check (Post-LLM)

After the LLM generates a response, re-validate against brand guidelines:

The brand check returns a compliance score (0-100) and a list of violations. If the score is below threshold, the response is sent back through the LLM with specific fix instructions.

Format

Structure the final output with:

Usage Tracking

Log every request:

Check per-team daily budgets. Alert at 80% and reject at 100%.

Cost Controls

Marketing teams can run hundreds of queries per day. Without cost controls, a single enthusiastic analyst can burn through a month's AI budget in a week.

The gateway enforces:

What You'll Build

Glossary