Cut LLM Token Costs with Anthropic Prompt Caching in TypeScript

Most teams discover prompt caching the same way: the monthly bill arrives, and the finance channel wants to know why a chatbot that answers short questions is metering tens of thousands of input tokens per call. The answer is almost always the same. Every request ships the same fat preamble — a long system prompt, a tool catalog, a retrieved knowledge base — and every request pays full price for it, over and over, even though those tokens never change between calls.

Prompt caching is the fix, and on paper it is trivial: mark the stable part of your prompt, and the provider serves it from a warm cache at roughly a tenth of the price on subsequent requests. In practice, most teams either never turn it on, or turn it on and quietly break it — a stray timestamp in the system prompt, a tool list that serializes in a different order each boot, and the cache read rate sits at zero while everyone assumes it is working.

This tutorial builds a small, focused TypeScript library that makes prompt caching correct by construction and, just as importantly, measurable. It plans requests so the cache boundary lands in the right place, audits prompts for the handful of mistakes that silently invalidate the cache, and turns the raw usage numbers a response comes back with into a dollars-and-cents savings report. Everything is dependency-injected and tested offline, so you can run the whole thing without an API key. The complete, runnable project lives in the CrashBytes ByteSizedExamples repository.

We build against Anthropic's prompt caching because it gives you explicit control over where the cache boundary sits — you place a breakpoint, you can reason about exactly what gets cached. The same discipline applies to any provider with prefix caching; the numbers and the cache_control syntax are Anthropic's, but the design lessons transfer directly.

What you will build

The library has five responsibilities, and each maps to a single module you will implement one at a time:

• Model facts — pricing per million tokens, the cache multipliers, and the minimum cacheable prefix per model, in one place you can trust.
• Plan — take a stable system prompt, a deterministic tool list, a large context blob, and a volatile question, and produce a request whose cache boundary is in the correct place.
• Audit — scan the stable prefix for the specific patterns that silently invalidate a cache, and serialize tool definitions deterministically.
• Measure — read the usage fields off a response and compute the cache hit rate and the realized dollar savings versus paying full price.
• Wire — a thin client that composes the four above around any object shaped like the Anthropic SDK, so it is trivial to test and trivial to adopt.

Prerequisites

You will want Node.js 20 or newer, npm, and a working knowledge of TypeScript — generics, discriminated unions, and strict mode should feel routine. No Anthropic API key is required to follow along or to run the test suite; the demo degrades gracefully to an offline simulation when no key is present. If you have never wired up a resilient LLM client before, the companion resilient multi-provider LLM client in TypeScript is a good primer on the request-construction patterns we build on here.

Project setup

Clone the examples repository and move into the project directory:

git clone https://github.com/CrashBytes/ByteSizedExamples.git
cd ByteSizedExamples/prompt-caching-cost-optimizer-typescript
npm install
cp .env.example .env   # optional — the demo runs offline without a key
npm start

The project is a standard ESM TypeScript package. The layout separates the pure logic (planning, auditing, cost math) from the thin I/O layer (the client and the demo entry point), which is what lets the whole thing be tested without a network:

prompt-caching-cost-optimizer-typescript/
├── src/
│   ├── models.ts            # pricing, multipliers, minimum cacheable prefix
│   ├── types.ts             # shared types for plans, blocks, and usage
│   ├── cache-planner.ts     # places the cache boundary correctly
│   ├── invalidator-audit.ts # finds silent cache invalidators
│   ├── savings.ts           # turns usage numbers into dollars
│   ├── client.ts            # composes the above around the Anthropic SDK
│   └── index.ts             # runnable demo (offline or live)
└── tests/
    ├── cache-planner.test.ts
    ├── invalidator-audit.test.ts
    ├── savings.test.ts
    └── client.test.ts

How prompt caching actually works

Before writing a line of the library, you need the mental model, because every design decision falls out of one invariant.

Prompt caching is a prefix match. The provider hashes the exact bytes of the rendered prompt up to each cache breakpoint. On the next request, if that prefix is byte-for-byte identical, the matching span is served from cache. Any change anywhere in the prefix — a single character — invalidates the cache from that point forward. The render order is fixed: tools first, then the system prompt, then the messages. A breakpoint on the last system block therefore caches the tools and the system prompt together.

That single fact drives the whole design. Stable content must physically come first, before any breakpoint. Volatile content — the user's actual question, a per-request timestamp, a session identifier — must come after the last breakpoint, or it poisons everything ahead of it.

The economics are what make the effort worthwhile. Reading cached tokens costs roughly one tenth of the base input price. Writing them to the cache costs more than an uncached token on the request that does the writing — about 1.25 times the base price for the default five-minute cache, or twice the base price for the one-hour cache. That write premium is the catch most cost models miss: the first request is more expensive, not less, and caching only pays off once you get enough reads to amortize that write.

There is one more constraint that trips people up: a prefix has to clear a minimum size before the provider will cache it at all. Below that threshold you get no cache write, no error, and no warning — just full-price tokens forever. The threshold is model-dependent, and a prompt that caches on one model will silently refuse to cache on another.

The library encodes all four of these facts — read multiplier, write multipliers, base pricing, and minimum prefix — so that neither the planner nor the cost calculator has to guess.

What a change actually invalidates

The prefix-match rule sounds absolute — change one byte, lose everything after it — and it is, but there is a useful nuance underneath it. The cache is layered in tiers that follow the render order, and a change only invalidates its own tier and the tiers that render after it. Changing a tool definition invalidates everything, because tools render first. Changing the system prompt invalidates the system and message caches but leaves the tools cache intact. Changing message content invalidates only the message cache. Toggling something like the thinking mode or the tool-choice hint sits in between.

The practical payoff is that you do not have to treat every request parameter as equally dangerous. You can flip tool_choice per request, or turn thinking on and off, without losing the expensive tools-and-system cache — only tool definitions and the model itself force a full rebuild. That is worth knowing when you design an agent that changes behavior mid-run: encode the behavior change as message content or a cheap toggle, and the fat cached prefix survives.

There is a corollary that catches teams running side computations. If a summarizer, a compaction pass, or a sub-agent spins up a separate request and rebuilds the system prompt, tools, or model with even a trivial difference, that fork misses the parent's cache entirely and pays cold write prices. The fix is mechanical: copy the parent's system, tools, and model verbatim into the fork, then append the fork-specific content at the end. planRequest makes this easy, because the stable inputs are explicit arguments you can pass through unchanged.

Module one: model facts

Start with the numbers, because everything else references them. The pricing table carries the input and output price per million tokens and the minimum cacheable prefix for each model, and the module exports the three multipliers as named constants so the cost math reads like the documentation rather than a pile of magic numbers.

// src/models.ts
export interface ModelPricing {
  inputPerMTok: number // USD per 1M input tokens
  outputPerMTok: number // USD per 1M output tokens
  minCacheableTokens: number // shorter prefixes silently will not cache
}

export const MODEL_PRICING: Record<string, ModelPricing> = {
  'claude-opus-4-8': {
    inputPerMTok: 5,
    outputPerMTok: 25,
    minCacheableTokens: 4096,
  },
  'claude-sonnet-5': {
    inputPerMTok: 3,
    outputPerMTok: 15,
    minCacheableTokens: 2048,
  },
  'claude-haiku-4-5': {
    inputPerMTok: 1,
    outputPerMTok: 5,
    minCacheableTokens: 4096,
  },
}

// Multipliers applied to the base input price.
export const CACHE_WRITE_MULTIPLIER_5M = 1.25
export const CACHE_WRITE_MULTIPLIER_1H = 2
export const CACHE_READ_MULTIPLIER = 0.1

export function getPricing(model: string): ModelPricing {
  const pricing = MODEL_PRICING[model]
  if (!pricing) {
    throw new Error(
      `Unknown model "${model}". Add it to MODEL_PRICING with its current pricing and minimum cacheable prefix.`
    )
  }
  return pricing
}

Two design notes worth internalizing. First, getPricing throws loudly on an unknown model rather than defaulting to some average, because a silent default would produce a savings report that is quietly wrong — the worst kind of wrong for a number that ends up in a cost review. Second, keep this table as the single source of truth. The moment pricing lives in two places, one of them is stale.

Module two: planning the request

This is the heart of the library. planRequest takes the four inputs — a stable system prompt, an optional deterministic tool list, an optional large context blob, and the volatile question — and produces a request whose cache boundary is in exactly the right place: on the last stable block, and never on the question.

// src/cache-planner.ts (excerpt)
export function estimateTokensDefault(text: string): number {
  // A deliberately rough heuristic. Use count_tokens for real accounting;
  // this is enough to decide whether a prefix clears the minimum.
  return Math.ceil(text.length / 4)
}

export function planRequest(input: PlanInput): CachePlan {
  const estimate = input.estimateTokens ?? estimateTokensDefault
  const pricing = getPricing(input.model)

  // Stable blocks render in prefix order: tools, then system, then context.
  // The breakpoint goes on the LAST stable block so everything before it caches.
  const system: CacheableTextBlock[] = [{ type: 'text', text: input.system }]
  if (input.context) {
    system.push({ type: 'text', text: input.context })
  }
  const lastStable = system[system.length - 1]
  lastStable.cache_control =
    input.ttl === '1h'
      ? { type: 'ephemeral', ttl: '1h' }
      : { type: 'ephemeral' }

  // The question is volatile: it changes every request, so it must sit AFTER
  // the breakpoint with no cache_control of its own.
  const messages = [
    {
      role: 'user' as const,
      content: [{ type: 'text' as const, text: input.question }],
    },
  ]

  const tools = input.tools ?? []
  const stableText =
    stableStringify(tools) + input.system + (input.context ?? '')
  const estimatedStablePrefixTokens = estimate(stableText)

  const warnings: string[] = []
  if (estimatedStablePrefixTokens < pricing.minCacheableTokens) {
    warnings.push(
      `Stable prefix is about ${estimatedStablePrefixTokens} tokens, below the ` +
        `${pricing.minCacheableTokens}-token minimum for ${input.model}; it will not cache.`
    )
  }

  return {
    model: input.model,
    system,
    tools,
    messages,
    breakpoints: system.filter(b => b.cache_control).length,
    estimatedStablePrefixTokens,
    warnings,
  }
}

The subtlety here is entirely about placement. The default five-minute cache uses a bare ephemeral marker; the one-hour cache adds the ttl field. Either way the marker lands on the last stable block — the context if there is one, otherwise the system prompt — so that the tools and the system prompt fall inside the cached span. The question never gets a marker, because caching a value that changes every request would mean writing a fresh cache entry each time and reading none of them: pure write premium, zero payoff.

The token estimate exists for one reason: to warn you when your stable prefix is too small to cache before you ship it. Below the minimum, the provider gives you no signal at all, so the planner gives you one. The estimate is deliberately crude — four characters to a token — because it only has to answer a yes-or-no question about a threshold, not produce an invoice. For real accounting, the provider's token-counting endpoint is authoritative.

Module three: auditing for silent invalidators

The planner puts the boundary in the right place, but a correctly-placed boundary still caches nothing if the bytes inside it change between requests. This is where most real-world caching quietly fails. The audit module exists to catch the culprits before they cost you.

There are two failure modes. The first is content that changes every request hiding inside the stable prefix: a Date.now() rendered into the system prompt header, a request UUID, a Math.random() seed, a wall-clock timestamp. Each one changes the prefix bytes on every call, so the cache is written fresh and never read.

// src/invalidator-audit.ts (excerpt)
const RULES: Array<{ rule: string; pattern: RegExp; hint: string }> = [
  {
    rule: 'iso-timestamp',
    pattern: /\d{4}-\d{2}-\d{2}T\d{2}:\d{2}/g,
    hint: 'A timestamp in the prefix changes every request. Move it after the cache boundary.',
  },
  {
    rule: 'uuid',
    pattern: /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi,
    hint: 'A per-request id busts the cache. Keep ids out of the cached prefix.',
  },
  {
    rule: 'date-now',
    pattern: /Date\.now\(/g,
    hint: 'A rendered clock value differs every call. Inject time after the boundary.',
  },
  {
    rule: 'math-random',
    pattern: /Math\.random\(/g,
    hint: 'Randomness in the prefix guarantees a cache miss every request.',
  },
]

export function auditForInvalidators(text: string): Finding[] {
  const findings: Finding[] = []
  for (const { rule, pattern, hint } of RULES) {
    for (const match of text.matchAll(pattern)) {
      findings.push({ rule, match: match[0], index: match.index ?? 0, hint })
    }
  }
  return findings
}

The second failure mode is subtler: non-deterministic serialization. Tool definitions are objects, and JSON.stringify preserves whatever key order the object happened to be built with. Build your tool list from an object whose keys land in a different order on the next process boot — a real risk when tools are assembled from a map or merged from config — and the serialized bytes differ even though the tools are logically identical. The prefix hash changes; the cache misses. The fix is a deterministic serializer that sorts object keys recursively.

// src/invalidator-audit.ts (excerpt)
export function stableStringify(value: unknown): string {
  if (value === null || typeof value !== 'object') {
    return JSON.stringify(value)
  }
  if (Array.isArray(value)) {
    return '[' + value.map(stableStringify).join(',') + ']'
  }
  const keys = Object.keys(value as Record<string, unknown>).sort()
  const entries = keys.map(
    k =>
      JSON.stringify(k) +
      ':' +
      stableStringify((value as Record<string, unknown>)[k])
  )
  return '{' + entries.join(',') + '}'
}

Arrays keep their order — order is meaningful for a tool list the model reads in sequence — but object keys are sorted, so two logically identical tool sets always produce identical bytes. auditTools runs both halves together: it returns the canonical serialization and any invalidator findings inside it.

Run this audit in a test against your real system prompt and tool list, and it becomes a regression guard: the day someone drops a timestamp into the preamble, a red test tells you before the cache read rate quietly craters in production.

Module four: measuring the savings

The whole point of caching is the money, and the money is knowable exactly, because every response comes back with a usage object that tells you precisely how the prompt tokens were billed. Three fields matter: input_tokens are the tokens billed at full price, cache_creation_input_tokens are the tokens written to the cache at the write multiplier, and cache_read_input_tokens are the tokens served from the cache at one tenth the price.

// src/savings.ts
export function computeSavings(
  model: string,
  usage: UsageLike,
  ttl: Ttl = '5m'
): SavingsReport {
  const pricing = getPricing(model)
  const inputPrice = pricing.inputPerMTok / 1_000_000
  const writeMult =
    ttl === '1h' ? CACHE_WRITE_MULTIPLIER_1H : CACHE_WRITE_MULTIPLIER_5M

  const uncached = usage.input_tokens
  const created = usage.cache_creation_input_tokens ?? 0
  const read = usage.cache_read_input_tokens ?? 0
  const totalPromptTokens = uncached + created + read

  const actualInputCostUsd =
    uncached * inputPrice +
    created * inputPrice * writeMult +
    read * inputPrice * CACHE_READ_MULTIPLIER
  const uncachedInputCostUsd = totalPromptTokens * inputPrice
  const savedUsd = uncachedInputCostUsd - actualInputCostUsd

  return {
    model,
    cacheHitRate: totalPromptTokens > 0 ? read / totalPromptTokens : 0,
    actualInputCostUsd,
    uncachedInputCostUsd,
    savedUsd,
    savedPct: uncachedInputCostUsd > 0 ? savedUsd / uncachedInputCostUsd : 0,
  }
}

The report compares two numbers: what you actually paid, and what you would have paid if every prompt token had been billed at full price. The difference is the savings, and the cache hit rate — cached reads over total prompt tokens — tells you how much of your prefix is actually being reused.

To see the shape of it, take a realistic coding assistant: a stable prefix of about 20,000 tokens (a system prompt, a tool catalog, and a retrieved knowledge base) and a fresh question of about 500 tokens on each call, running on Opus 4.8.

On a warm request, 20,000 of the 20,500 prompt tokens are served from cache at a tenth of the price. That is a 97.6 percent cache hit rate and an input cost of about 1.25 cents, against the 10.25 cents you would have paid with no caching — roughly an 88 percent reduction on the input side of that request.

The cold request tells the other half of the story, and it is the half cost models forget. On the very first call the 20,000-token prefix is written, not read, at the 1.25 write premium — so that request costs about 12.75 cents, more than the 10.25 cents it would have cost with no caching at all. Caching is a bet that you will get enough warm reads to pay back that write.

The bet pays off fast. Over a hundred requests — one cold write followed by ninety-nine warm reads inside the cache window — the cached path costs about 1.37 dollars against 10.25 dollars uncached, a roughly 87 percent reduction that swamps the one-time write premium after just a couple of reads.

The break-even is the useful number to carry around. With the five-minute cache and its 1.25 write premium, you are ahead after the second request; with the one-hour cache and its heavier 2 write premium, you need a third request before the cache has paid for itself. That is the entire decision behind which cache lifetime to choose: pick the one-hour window only when your traffic has gaps longer than five minutes but you will still come back inside the hour.

Module five: wiring it together

The client is deliberately thin. It takes anything shaped like the Anthropic SDK — an object with a messages.create method — plans the request, sends it, and computes the savings from whatever usage comes back. Accepting the client by injection rather than constructing one internally is the single decision that makes the whole library testable offline.

// src/client.ts (excerpt)
export class CachingClient {
  constructor(private readonly opts: CachingClientOptions) {}

  async ask(input: Omit<PlanInput, 'model'> & { model?: string }) {
    const model = input.model ?? this.opts.model ?? 'claude-opus-4-8'
    const plan = planRequest({ ...input, model })

    const response = await this.opts.anthropic.messages.create({
      model,
      max_tokens: this.opts.maxTokens ?? 1024,
      system: plan.system,
      tools: plan.tools,
      messages: plan.messages,
    })

    const savings = computeSavings(model, response.usage, input.ttl)
    return { response, plan, savings }
  }
}

Because MessagesCreator is a structural interface — just an object with messages.create — a real @anthropic-ai/sdk client satisfies it in production, and a hand-written fake satisfies it in tests. Neither the planner nor the cost math ever touches the network, so the interesting logic is verifiable in milliseconds.

Testing the library

The test suite is where the design pays off, because the parts worth testing are pure functions. Four files cover the four modules, and none of them make a network call.

The planner tests assert the invariant directly: the cache marker lands on the last stable block, the question block carries no marker, the one-hour lifetime emits the ttl field, and a stable prefix below the model minimum produces a warning while a large one does not.

// tests/cache-planner.test.ts (excerpt)
it('places the cache boundary on the last stable block, never on the question', () => {
  const plan = planRequest({
    model: 'claude-opus-4-8',
    system: 'x'.repeat(80_000), // ~20k tokens, well over the minimum
    question: 'What changed in the auth module?',
  })

  const lastSystem = plan.system[plan.system.length - 1]
  expect(lastSystem.cache_control).toEqual({ type: 'ephemeral' })
  expect(plan.messages[0].content[0].cache_control).toBeUndefined()
  expect(plan.breakpoints).toBe(1)
  expect(plan.warnings).toHaveLength(0)
})

it('warns when the stable prefix is below the model minimum', () => {
  const plan = planRequest({
    model: 'claude-opus-4-8',
    system: 'too short to cache',
    question: 'hello',
  })
  expect(plan.warnings[0]).toMatch(/will not cache/)
})

The audit tests confirm that stableStringify produces identical bytes for two objects whose keys were built in different orders — the property that actually protects the cache — and that each invalidator rule fires on a positive case and stays quiet on a clean prompt.

// tests/invalidator-audit.test.ts (excerpt)
it('serializes identically regardless of key order', () => {
  const a = {
    name: 'search',
    description: 'find',
    input_schema: { type: 'object' },
  }
  const b = {
    input_schema: { type: 'object' },
    description: 'find',
    name: 'search',
  }
  expect(stableStringify(a)).toBe(stableStringify(b))
})

it('flags a Date.now call and a timestamp in the prefix', () => {
  const findings = auditForInvalidators(
    'Current time is Date.now() as of 2026-07-13T09:30'
  )
  const rules = findings.map(f => f.rule)
  expect(rules).toContain('date-now')
  expect(rules).toContain('iso-timestamp')
})

The savings tests pin the cost math to worked examples: a usage record that is mostly cache reads yields a high hit rate and a large savings percentage, an all-uncached record yields zero savings and a zero hit rate, and a one-hour lifetime charges strictly more on the write than a five-minute one for the same token counts. The client test uses a fake MessagesCreator that records the request body it was handed, then asserts the body carries cache_control on the last system block and that the returned report matches computeSavings on the canned usage. That is the full contract, verified without a key.

Running the demo, with or without a key

The entry point is written so npm start does something useful in both worlds. When ANTHROPIC_API_KEY is set, it constructs a real client, sends the same large stable context twice with two different questions, and prints the savings report for each — so you watch the first call write the cache and the second read it, the hit rate jumping from near zero to the high nineties between the two.

When there is no key, the demo does not fail and it does not fake a network call. Instead it exercises the pure logic directly: it plans a request against a large context, prints any invalidator findings and planner warnings, and runs computeSavings against a hand-built usage record that mimics a warm read. That is enough to see the whole pipeline — plan, audit, measure — produce real output on a laptop with no credentials, which is exactly what you want when you are first evaluating whether caching is worth wiring into your own stack.

// src/index.ts (offline path, excerpt)
const plan = planRequest({
  model: 'claude-opus-4-8',
  system: KNOWLEDGE_BASE, // a large, stable context blob
  question: 'Summarize the retry policy.',
})
console.log('warnings:', plan.warnings)
console.log('audit:', auditForInvalidators(KNOWLEDGE_BASE))

// A simulated warm request: almost all tokens served from cache.
const savings = computeSavings('claude-opus-4-8', {
  input_tokens: 500,
  cache_creation_input_tokens: 0,
  cache_read_input_tokens: 20_000,
  output_tokens: 800,
})
console.log(`hit rate ${(savings.cacheHitRate * 100).toFixed(1)}%`)
console.log(
  `saved $${savings.savedUsd.toFixed(4)} (${(savings.savedPct * 100).toFixed(1)}%)`
)

Keeping the demo runnable offline is not a gimmick. It is the same property the tests rely on — pure logic separated from I/O — expressed as a thing you can run by hand. If the demo needed a key, so would every test, and the feedback loop would slow from milliseconds to seconds and from free to metered.

Patterns worth carrying beyond this project

The library is small, but the disciplines behind it are the ones that separate a cache that works from one that quietly does not.

Freeze the system prompt. The most common way to break caching is to interpolate live values — the current date, the user's name, the active mode — into the system prompt, which sits at the very front of the prefix and invalidates everything after it. Keep the system prompt byte-stable, and inject dynamic context later in the message list, where a change invalidates only the turns after it. On the newest models you can deliver trusted mid-session instructions as a system-role message inside the messages array rather than by editing the top-level system prompt, which keeps the cached prefix intact.

Serialize tools deterministically and never change them mid-conversation. Tools render at position zero, so adding, removing, or reordering a tool invalidates the entire cache. Sort your tool JSON, and if you need modes, encode the mode as message content rather than swapping the tool set.

Mind the lookback window. Each breakpoint walks backward only a limited number of content blocks to find a prior cache entry. In a long agentic loop that appends many tool-call and tool-result blocks in a single turn, a later breakpoint can fail to find the previous one and silently miss. Place an intermediate breakpoint every dozen-or-so blocks in long turns.

Pre-warm only when it pays. Firing a zero-output request at startup writes the cache so the first real request reads instead of writes, which is worth it when first-request latency is user-visible and there is a quiet moment before traffic. Under continuous traffic the first real request warms the cache for free, and a separate warm call is just an extra write.

Sequence your fan-out. A cache entry only becomes readable after the first response that wrote it begins streaming. Fire a hundred identical-prefix requests in parallel from a cold cache and all hundred pay the write premium — none can read what the others are still writing. When you fan out over a shared prefix, send one request first, wait for its first streamed token, then release the rest; they read the cache the first one just warmed, and your bill reflects one write and ninety-nine reads instead of a hundred writes.

Do not cache what does not repeat. If the first thousand tokens of your prompt differ on every request, there is no reusable prefix, and adding a marker only pays the write premium with zero reads. Caching is for the fat, stable preamble — measure first, and if the prefix is not shared, leave it off.

Where this fits

Prompt caching is one lever in a larger cost story. It attacks the repeated-input line item specifically; model routing attacks the wrong-model-for-the-job line item, and consumption-aware billing attacks the surprise-at-month-end problem that shows up when vendors move from seats to consumption pricing. The market pressure behind all of this is the same one our prediction that the agentic frontier token-pricing floor holds through Q4 2026 tracks: when the per-token floor stops falling, the only remaining lever is sending fewer tokens at full price, and caching is the cleanest way to do that. For the broader picture of why the industry is repricing token consumption at all, our news analysis on the hidden cost of AI token consumption pricing lays out the demand side.

The techniques here also compose cleanly with the request-validation discipline from the type-safe LLM tool-calling layer tutorial: the same deterministic tool serialization that protects your cache also gives you the stable schema your validator keys against. Caching and validation are two faces of the same idea — treat the boundary between your code and the model as a place that deserves engineering, not guesswork.

Conclusion

Prompt caching is not exotic. It is a prefix match, a boundary you place on purpose, and a usage object that tells you exactly what you saved. The reason it so often fails to save anything is not the mechanism — it is the invisible mistakes: a boundary in the wrong place, a timestamp in the prefix, a tool list that serializes differently each boot, and no measurement to catch any of it. The library you just built closes all four gaps: it places the boundary correctly, audits the prefix, serializes deterministically, and reports the savings in dollars, all verifiable offline.

Clone the complete project from the CrashBytes ByteSizedExamples repository, run the test suite without a key, and then point the client at your own system prompt and tool list. Watch the cache hit rate in the report climb toward the high nineties — and watch the input line on your bill do the opposite.