REC

Record once.
Replay forever.

Stonetape turns real LLM and tool interactions into hermetic regression tests for TypeScript apps. Record a real agent run once. Replay it in CI forever: zero tokens, zero flakiness, no API keys, fully parallel.

GitHub ↗
Live test (measured)
5.2 s · API keys
Replayed (measured)
17 ms · $0 · offline
Terminal recording: a test suite recorded against the real API once, then replayed instantly with no network
A real run, replayed. No network, no keys.

Testing LLM code gives you three bad options

  1. TAKE 01

    Hit the real APIs in CI

    Costs real money on every push. Slow, flaky under rate limits, non-deterministic by nature, and every runner needs production secrets.

  2. TAKE 02

    Hand-write mocks

    They drift from reality the day you write them. When a provider changes its response shape, your mocks keep passing. They lie silently.

  3. TAKE 03

    Skip the tests

    The most common choice. The code path that talks to the model, the most fragile part of the app, ships unverified.

Mocks test scenarios you imagined. Stonetape preserves the messy ones that actually happened.

No monkeypatching. Just fetch.

OpenAI and Anthropic SDKs accept a custom fetch. The Vercel AI SDK has an official middleware API. Stonetape intercepts at that layer, records everything that crosses it, and replays it byte for byte. Streaming chunks and whole agent chains included.

STEP 01

Point your SDK at the tape

import { openCassette } from "stonetape";
import OpenAI from "openai";

const tape = openCassette("tests/cassettes/weather-agent.yaml");
const client = new OpenAI({ fetch: tape.fetch });

// ... run your agent ...

tape.close();
STEP 02

Record once, replay forever

STONETAPE_MODE=record vitest   # hits the real API once, writes the cassette
vitest                         # replays forever: no network, no keys

Or let the vitest helper handle the cassette lifecycle: 

import { cassette } from "stonetape/vitest";

test("weather agent", cassette("weather-agent", async ({ fetch }) => {
  const client = new OpenAI({ fetch });
  const result = await runAgent(client, "What's the weather in Cluj?");
  expect(result.city).toBe("Cluj");
}));
STEP 03

The cassette is a file. Commit it.

# tests/cassettes/weather-agent.yaml  (committed to git)
version: 1
recorded_at: 2026-02-11T21:47:03Z
interactions:
  - request:
      url: https://api.openai.com/v1/chat/completions
      body:
        model: gpt-4o
        messages: [ ... ]
    response:
      status: 200
      body:
        choices:
          - message:
              tool_calls:
                - function:
                    name: get_weather
                    arguments: '{"city":"Cluj"}'

Versioned, language-agnostic schema. Secrets auto-redacted. When your prompt changes behavior, the replay fails with an explicit chain diff, not a cryptic error.

SIDE B

Not in your test process? Proxy it.

# record once (real upstream):
stonetape proxy --cassette agent.yaml \
  --target https://api.openai.com --mode record
OPENAI_BASE_URL=http://127.0.0.1:8787/v1  your-agent-cli "do the thing"

# replay forever — upstream not contacted, no keys:
stonetape proxy --cassette agent.yaml \
  --target https://api.openai.com --port 8787
OPENAI_BASE_URL=http://127.0.0.1:8787/v1  your-agent-cli "do the thing"

Agent CLIs, sidecar services, Python or Go processes — run stonetape proxy and point them at it via their base-URL env. Any language records; replay works with the upstream dead. Same matching, same redaction, same fail-closed semantics: unrecorded requests answer 501 with the full mismatch explanation.

Behavior diffs on every PR

The library stays MIT, free forever. Stonetape Cloud is what comes next: when a PR changes what your agent actually does, the diff shows up in review. Nightly re-records catch provider drift before your users do.