Model APIs

danwo's model API emulator answers the APIs your app's AI features call: OpenAI, Anthropic, the Vercel AI Gateway, Gemini, Perplexity and Bedrock. The openai and @anthropic-ai/sdk packages and the AI SDK work against it unchanged, streaming or not.

Each call is answered from a script you write, so a test knows exactly what the model will say. That makes AI features testable like any other: same input, same answer, every run.

This page covers setting it up and reading the calls back. Writing replies is on Scripting replies, and Gemini, Perplexity and Bedrock on More model APIs.

Set upCopy Anchor Link

Add llmService with a script and the prices you want cost reported at. Any key is accepted, so your app keeps its own:

import { appEnvironment, defineConfig, llmService } from "@danwo/sdk"

export default defineConfig({
  testDir: "tests",
  environment: () =>
    appEnvironment({
      app: { command: "npm run dev", url: "http://localhost:3000" },
      services: [
        llmService({
          seed: {
            scripts: [
              {
                name: "summary",
                match: { user: /summari[sz]e/ },
                reply: { text: "Three orders shipped late this week." },
              },
            ],
            prices: { "gpt-5": { input: 1.25, output: 10 } },
          },
        }),
      ],
    }),
})

Seed itCopy Anchor Link

Use it in a testCopy Anchor Link

Ask for a summary, then check the call the app made and what it cost:

import { type LlmFixtures, llmCalls, test as base } from "@danwo/sdk"

const test = base.extend<LlmFixtures>()

test("summarises the week in one call", async ({ app, screen, expect, llm }) => {
  await app.open("/reports")
  await screen.getByRole("button", { name: "Summarise this week" }).click()
  await expect(screen.getByText("Three orders shipped late this week.")).toBeVisible()

  const calls = llmCalls(llm)

  expect(calls.all()).toHaveLength(1)
  expect(calls.last()?.model).toBe("gpt-5")
  expect(calls.cost()).toBeLessThan(0.01)
})

CostCopy Anchor Link

Token counts are an estimate, about four characters a token, and the same for the same text every time, so a test of your app's usage tracking can check exact numbers. With prices seeded, each call's cost is in dollars, and the AI Gateway's answers carry that cost in their provider metadata, as the real gateway's do.

What's coveredCopy Anchor Link

FaultsCopy Anchor Link

The last row is what only an emulator can do: the real API won't refuse on cue.

FidelityCopy Anchor Link

Live for the AI Gateway. danwo fidelity gateway runs 6 scenarios against the real gateway, from text and tool calls to streams, embeddings and errors. All 6 match.

Gemini, Perplexity and Bedrock are checked offline, against their published descriptions. OpenAI and Anthropic have no suite of their own yet.

Not covered yetCopy Anchor Link

• Images, audio and video
• OpenAI's Assistants, Realtime and batch APIs
• Fine-tuning and file uploads

NextCopy Anchor Link