Vercel AI SDK: Wiring Up a Custom OpenAI-Compatible Provider

The AI SDK splits into two halves that people conflate. The core half runs on the server and makes model calls. The UI half runs in the browser and manages message state over a streaming connection between them. Most setup problems come from wiring one half correctly and assuming the other followed.

Pointing it at a non-OpenAI endpoint is a provider factory call. Keeping streaming alive through a real deployment is the part that takes an afternoon.

Create the provider

The SDK ships a dedicated OpenAI-compatible provider factory. Use it rather than the OpenAI provider with an overridden URL — the OpenAI provider carries assumptions about model-specific features that do not hold elsewhere.

import { createOpenAICompatible } from '@ai-sdk/openai-compatible'

export const provider = createOpenAICompatible({
  name: 'cozy',
  baseURL: 'https://api.example.com/v1',
  apiKey: process.env.LLM_API_KEY,
})

export const model = provider('kimi-k3')

The base URL ends at /v1. The SDK appends the route. As with every OpenAI-compatible client, a doubled path segment produces a 404 that reads like an auth failure, so check that first when a working key stops working.

Keep this in one module and import the exported model everywhere. Constructing a provider per request works but scatters the base URL across your codebase, which makes switching endpoints a find-and-replace instead of a one-line change. That flexibility is the main practical benefit of the OpenAI-compatible API shape and it is worth preserving.

Server: generate or stream

For a one-shot call where nothing is displayed incrementally — a classification, a summary written to a database — use the non-streaming call and take the text:

import { generateText } from 'ai'
import { model } from '@/lib/ai'

const { text, usage } = await generateText({
  model,
  prompt: 'Classify this ticket: ' + body,
  maxRetries: 0,
})

Set maxRetries explicitly. The SDK retries by default, and an inherited retry policy is one you cannot see in your metrics or your bill. Decide the number, put it in one place, and log when it fires.

For anything a user watches, stream. The route handler returns the SDK's response helper directly, which sets the headers the client half expects:

import { streamText } from 'ai'
import { model } from '@/lib/ai'

export async function POST(req: Request) {
  const { messages } = await req.json()
  const result = streamText({ model, messages })
  return result.toDataStreamResponse()
}

The runtime and buffering traps

This is where a working local setup fails in production. Streaming needs every layer between the model and the browser to forward bytes as they arrive, and several common layers do not by default.

The usual culprits are a reverse proxy buffering the proxied response, compression middleware accumulating before it compresses, and serverless runtimes that collect the whole response before returning it. Symptom: the answer appears all at once after a long pause, instead of token by token. Diagnose from outside your app first with a raw curl -N against the route — if curl trickles and the browser does not, the problem is client-side. The full list of causes is in streaming and server-sent events.

Two more that are specific to this stack. Function timeouts: a serverless function with a ten-second limit will cut off a long generation mid-stream, and the client sees a truncated answer rather than an error. Raise the limit for streaming routes or move them to a runtime without one. And any middleware that reads the request or response body to log it will consume the stream before the client gets it.

The client half

The React hook manages an array of messages, an input value, and the fetch to your route. It expects the data stream protocol the server helper produces, so a hand-rolled route returning raw SSE will connect and then fail to parse.

Two things belong in every real implementation. First, handle the error state — the hook exposes one, and an unhandled failure otherwise looks like the assistant simply never replying. Second, wire up the abort path: when a user navigates away or clicks stop, cancel the request so generation stops upstream rather than running to completion and billing you for text nobody reads.

Cancellation only works if it propagates all the way. Aborting the browser fetch while your route keeps consuming the provider stream saves nothing. Pass the incoming request's abort signal through to the model call.

Tool calling and multi-step loops

Tools are defined with a schema and a handler, and the SDK converts them into the JSON schema the endpoint expects:

import { tool } from 'ai'
import { z } from 'zod'

const searchDocs = tool({
  description: 'Search internal documentation',
  parameters: z.object({
    query: z.string(),
    limit: z.number().default(5),
  }),
  execute: async ({ query, limit }) => search(query, limit),
})

Whether this works depends on the endpoint honouring the tool call contract, not on the SDK. Test it before building on it: one trivial tool, a prompt that clearly requires it, and an inspection of the raw tool call in the result. The recognisable failure is a model that describes the tool it would call in prose — that means tool calls are not being negotiated at all. Background in how tool calling works and which models do it reliably.

By default the SDK stops after the model emits a tool call. To run the loop — call tool, feed the result back, let the model continue — you opt into multiple steps with a step limit. Set that limit low and deliberately. An unbounded loop is how a chat route quietly makes forty model calls for one user message, which is the core of why agent costs are unpredictable.

Usage, cost and what the SDK will not tell you

Both the generate and stream paths expose token usage, though with streaming it arrives at the end rather than up front, and only if the provider reports it. Verify that your endpoint returns a usage block while streaming before you build billing on it — several do not by default.

The SDK reports tokens, not money. Pricing is yours to apply, and it is worth logging counts per request keyed by route so you can see which feature is expensive rather than only that the total went up. In a multi-step tool loop, log per step: the loop, not the model, is usually where the tokens go.

Note also that the usage figure covers the model call and not your own overhead — the system prompt you inject on every request, the retrieved documents, the conversation history you replay. Those are the parts you control, and reducing token usage starts with measuring them separately.

A setup order that saves time

Get a non-streaming generateText call working from a server route first. That proves base URL, key and model alias in isolation, with a readable error if any of them is wrong.

Then switch that same route to streaming and test it with curl -N before involving React. Then connect the client. Then add tools, one at a time, checking the raw tool call each time.

Doing it in that order means each failure has exactly one plausible cause. The alternative — building the whole feature and then debugging why nothing appears — leaves you guessing between four layers, and the symptom is identical in all four.