Build a Type-Safe LLM Tool-Calling Layer in TypeScript: Zod Validation and Auto-Repair

Every tutorial about LLM tool calling shows you the happy path. You define a function, you hand its JSON schema to the model, the model replies with a neat little object, and you call the function. It works in the demo. It works in the screenshot. Then you ship it, and three days later something calls create_invoice with amount: "one hundred", or currency: "dollars", or a customerId field that simply is not there, and your handler — which was typed as if the arguments were already clean — does something you did not intend with input you never validated.

This is the part the demos skip. When a model emits a tool or function call, the arguments do not arrive as a typed object. They arrive as a string of JSON that the model wrote, and a model is a text generator, not a schema-obeying API. The JSON might be malformed. It might be missing required fields. It might have the right field names with the wrong types — a number where you wanted a string, a string where you wanted a boolean. It might invent an enum value that looks plausible and does not exist. It might wrap the whole thing in "Sure! Here are the arguments:" and a Markdown fence. Feeding that directly into a typed handler is the single most common way production agent code goes wrong, and it fails in two different and equally serious ways: correctness and security.

This tutorial builds the layer that sits between the model's raw output and your handlers. It is provider-agnostic, it has zero runtime dependencies beyond Zod, and it does something most tool-calling code does not: when validation fails, it repairs itself. It feeds the specific validation errors back to the model, asks for corrected arguments, and tries again — in a bounded loop with a hard cap so it can never spin forever. By the end you will have a tool registry, a validator that turns Zod errors into model-readable feedback, a self-healing repair loop, a safe typed dispatcher, and metrics that tell you how often the model gets it right on the first try.

The full runnable project is at github.com/CrashBytes/ByteSizedExamples under typesafe-llm-tool-calling-zod-typescript. Clone it, run npm install, npm run typecheck, and npm test, and follow along. Every test is deterministic and offline — the only "model" in the suite is a function that returns canned strings, so the whole thing runs in a quarter of a second with no API key and no network.

Why unchecked tool arguments are a real problem

Before any code, it is worth being precise about what goes wrong, because the two failure modes want different fixes and both matter.

The correctness failure is the obvious one. A model asked for structured output is right most of the time and wrong often enough to hurt. If a single tool call is well-formed ninety-five percent of the time, then across a thousand calls you have fifty broken ones, and "broken" here does not mean "throws a clean error." It means amount came back as the string "90" instead of the number 90, your handler did amount * 1.08 to add tax, JavaScript happily coerced the string, and you wrote a subtly wrong number into a database. The type system told you amount was a number. The runtime value was a string. That gap — between what the types promise and what the model actually sent — is where the bugs live, and it is invisible until it is a support ticket.

The security failure is the one that should keep you up at night. Tool calls are, by definition, the model reaching out to touch the real world: writing files, sending emails, querying databases, moving money, calling other APIs. The arguments to those actions are attacker-influenceable in any system where a user can put text in front of the model — which is every chatbot, every RAG system, every agent that reads a web page or an email. A prompt-injected instruction can try to steer a tool call toward a destructive argument. An over-permissive schema — one that accepts a raw string where it should accept one of three enum values, or a number with no bounds where it should be a positive amount under some ceiling — is an open door. Validating tool arguments is not a nicety; it is the boundary between the model's suggestions and your system's actions, and boundaries are where you put your controls.

The fix for both is the same discipline: treat every tool-call argument as untrusted input, validate it against a strict schema before it reaches a handler, and make it structurally impossible for unvalidated data to be executed. That is what we are building.

What we are actually building

The layer has five responsibilities, and the whole tutorial is an implementation of these five bullets, built one file at a time:

• Define each tool as a Zod schema plus a typed handler, held in a registry that doubles as an allow-list of what the model is permitted to call.
• Validate the model's raw argument text against the tool's schema, turning any ZodError into human-readable, field-level feedback.
• Repair on failure: feed those specific errors back to the model, ask for corrected arguments, and re-validate — in a bounded loop with a hard cap.
• Dispatch the validated, fully-typed arguments to the handler, and never reach the handler on a failed validation.
• Measure how often the model is right on the first pass, how often repair saved the call, and how often it failed anyway.

Everything is provider-agnostic. The only place the layer touches a language model is a one-method interface, so you can wire in Anthropic, OpenAI, a local model, or — in every test — a deterministic mock that never touches the network. This is the same dependency-inversion discipline behind a resilient multi-provider LLM client: keep the provider behind a seam, and the whole system becomes testable and swappable.

Part 1: the tool registry

Start with the data model, because everything else operates on it. A tool is a name, a description, a Zod schema, and a handler whose argument type is inferred from that schema. That last part is the whole point: the handler cannot read a field the schema does not guarantee, because TypeScript infers the handler's parameter type directly from the schema you attached. Create src/toolRegistry.ts:

import type { z } from 'zod'

/**
 * A tool the model is allowed to call. The Zod `schema` is the single source of
 * truth for what valid arguments look like, and `handler` receives arguments
 * whose type is *inferred from that schema* — so the handler body is guaranteed
 * to see fully-typed, already-validated input. There is no way to call the
 * handler with unchecked args, because the dispatcher only reaches it after a
 * successful `schema.safeParse`.
 */
export interface ToolDefinition<Schema extends z.ZodTypeAny, Result> {
  readonly name: string
  /** Short, model-facing description of what the tool does and expects. */
  readonly description: string
  readonly schema: Schema
  readonly handler: (args: z.infer<Schema>) => Promise<Result> | Result
}

/**
 * Identity helper that pins the generic parameters so `handler`'s argument is
 * inferred from `schema` at the call site. Prefer this over building the object
 * literal by hand — it is what gives you a type error the moment a handler
 * reads a field the schema does not guarantee.
 */
export function defineTool<Schema extends z.ZodTypeAny, Result>(
  def: ToolDefinition<Schema, Result>
): ToolDefinition<Schema, Result> {
  return def
}

The defineTool helper looks like it does nothing — it returns its argument unchanged. What it does is pin the generics at the call site. When you write defineTool({ schema: z.object({ amount: z.number() }), handler: args => ... }), TypeScript infers Schema from the schema you passed and then types args as z.infer<Schema>, which is { amount: number }. If your handler tries to read args.customerId and the schema does not define customerId, you get a compile error, right there, before the code ever runs. The schema and the handler can never drift apart, because one is derived from the other. This is the difference between "I documented what the arguments should be" and "the compiler enforces what the arguments are."

Now the registry itself — a keyed catalogue that doubles as the security allow-list:

/** A tool whose specific schema/result types have been erased for storage. */
export type AnyToolDefinition = ToolDefinition<z.ZodTypeAny, unknown>

/**
 * An in-memory catalogue of callable tools keyed by name. The registry is the
 * allow-list: if a model asks for a tool that was never registered, dispatch
 * rejects the call as `unknown_tool` instead of guessing. Registering the same
 * name twice throws, because a silently shadowed tool is a debugging nightmare.
 */
export class ToolRegistry {
  private readonly tools = new Map<string, AnyToolDefinition>()

  register<Schema extends z.ZodTypeAny, Result>(
    tool: ToolDefinition<Schema, Result>
  ): this {
    if (this.tools.has(tool.name)) {
      throw new Error(`tool "${tool.name}" is already registered`)
    }
    this.tools.set(tool.name, tool as unknown as AnyToolDefinition)
    return this
  }

  get(name: string): AnyToolDefinition | undefined {
    return this.tools.get(name)
  }

  has(name: string): boolean {
    return this.tools.has(name)
  }

  names(): string[] {
    return [...this.tools.keys()]
  }
}

Two design choices deserve a word. First, the registry is an allow-list, not a lookup table. The tool name in a tool call comes from the model, which means it is untrusted: a model can hallucinate a tool that does not exist, and a prompt-injected model can try to name a tool it was never given. Routing every call through get(name) and rejecting anything that returns undefined means the set of callable actions is exactly the set you registered, no more. Second, register throws on a duplicate name. A registry that silently lets a second create_invoice shadow the first is a landmine — you will spend an afternoon wondering why your edits to a handler have no effect. Fail loud at registration time, when the stack trace points at the mistake.

The type erasure — storing everything as AnyToolDefinition with a cast — is the unavoidable price of dispatching by a runtime string. When you look a tool up by a name you only know at runtime, TypeScript cannot know which specific schema you got back, so the result type collapses to unknown on that path. We will recover full typing on the other path, dispatchTool, where the tool is known statically.

Part 2: pulling JSON out of noisy model text

Before we can validate arguments, we have to find them. Some providers hand you a clean arguments string on a structured tool-call object; others, and every case where you are parsing tool calls out of free-form completion text, give you the JSON wrapped in conversational packaging. Models love to say "Sure! Here are the arguments:" before the payload and "Let me know if you'd like changes!" after it, often with a Markdown code fence around the middle. We need a locator that tolerates all of that. Create src/validate.ts and start with extraction:

import type { z } from 'zod'

/**
 * Pull the first JSON object or array out of a (possibly fenced, possibly
 * prose-wrapped) model completion. Models routinely wrap tool arguments in
 * "Sure! Here are the arguments:" prose and a ```json fence; this locates the
 * actual value so `JSON.parse` gets clean input. It is a locator, not a parser.
 */
export function extractJson(text: string): string {
  const body = stripCodeFence(text)

  const start = firstIndexOf(body, '{', '[')
  if (start === -1) {
    throw new Error('no JSON value found in tool arguments')
  }

  const open = body[start]
  const close = open === '{' ? '}' : ']'
  const end = body.lastIndexOf(close)
  if (end <= start) {
    throw new Error('unterminated JSON value in tool arguments')
  }

  return body.slice(start, end + 1)
}

The function strips a Markdown fence if one is present, finds the first opening brace or bracket and the last matching closer, and returns the slice between them. It is deliberately not a JSON parser — JSON.parse is the parser. This is a locator whose only job is to peel away the conversational wrapper so the real parser gets clean input. When the text contains no JSON value at all, it throws, which — as we will see — is exactly what we want, because a thrown extraction becomes a validation failure and triggers a repair.

The helpers are plain string scanning, and the reason is worth a note:

/**
 * Return the contents of the first ``` fenced block, or the whole string when
 * there is no fence. Plain string scanning, deliberately not a regex: a lazy
 * `[\s\S]*?` between two fences is exactly the backtracking shape that trips
 * ReDoS scanners, and there is no need for it here.
 */
function stripCodeFence(text: string): string {
  const open = text.indexOf('```')
  if (open === -1) {
    return text
  }
  // Skip the optional language tag (e.g. ```json) on the opening-fence line.
  const lineEnd = text.indexOf('\n', open)
  const contentStart = lineEnd === -1 ? open + 3 : lineEnd + 1
  const close = text.indexOf('```', contentStart)
  return close === -1
    ? text.slice(contentStart)
    : text.slice(contentStart, close)
}

/** Index of whichever of `a`/`b` appears first, or -1 if neither is present. */
function firstIndexOf(text: string, a: string, b: string): number {
  const ia = text.indexOf(a)
  const ib = text.indexOf(b)
  if (ia === -1) return ib
  if (ib === -1) return ia
  return Math.min(ia, ib)
}

A regex like /```(?:json)?\s*([\s\S]*?)```/ would work on the happy path and introduce a catastrophic-backtracking risk on adversarial input. Since a model's output is, in the security framing above, attacker-influenceable, the last thing you want on your parsing path is a pattern a security scanner will flag as a denial-of-service vector. Plain indexOf scanning is linear, boring, and safe.

Part 3: schema-to-error feedback

Now the heart of validation. We need to take raw argument text, parse it, check it against a Zod schema, and — crucially — when it fails, produce feedback the model can act on. A generic "validation failed" is useless to a model; telling it exactly which field was wrong and why is what makes the repair loop converge in one turn instead of five. Add the error formatter to src/validate.ts:

/**
 * Turn a `ZodError` into numbered, field-path-prefixed feedback the model can
 * act on. The path matters: telling the model `amount: expected number,
 * received string` is far more actionable than a generic "validation failed",
 * and it is what makes the repair loop converge in one turn instead of five.
 */
export function formatZodError(error: z.ZodError): string {
  return error.issues
    .map(issue => {
      const path = issue.path.length > 0 ? issue.path.join('.') : '(root)'
      return `- ${path}: ${issue.message}`
    })
    .join('\n')
}

Zod's error.issues is an array of structured problems, each with a path (where in the object the problem is) and a message (what is wrong). Joining the path with dots and prefixing the message gives you lines like - amount: Number must be greater than 0 and - currency: Invalid enum value. Expected 'USD' | 'EUR' | 'GBP', received 'dollars'. That is a correction the model can make in one shot, because it names the field and states the constraint. This is the same principle behind a good LLM eval harness: the quality of the feedback determines the quality of the next attempt.

Now the function that ties parsing and validation together:

/** The result of validating raw argument text against a tool's schema. */
export type ValidationResult<T> =
  | { readonly ok: true; readonly value: T }
  | { readonly ok: false; readonly errors: string }

/**
 * Parse raw tool-call argument text and validate it against `schema`. Two
 * distinct failure modes collapse into one shape here: text that is not valid
 * JSON at all, and JSON that parses but violates the schema. Both come back as
 * `{ ok: false, errors }` with human-readable feedback, so the repair loop
 * treats "you sent me garbage" and "you sent me the wrong shape" identically.
 */
export function parseAndValidate<Schema extends z.ZodTypeAny>(
  schema: Schema,
  rawArguments: string
): ValidationResult<z.infer<Schema>> {
  let json: unknown
  try {
    json = JSON.parse(extractJson(rawArguments))
  } catch (error) {
    const message = error instanceof Error ? error.message : String(error)
    return {
      ok: false,
      errors: `- (root): arguments were not valid JSON (${message})`,
    }
  }

  const parsed = schema.safeParse(json)
  if (parsed.success) {
    return { ok: true, value: parsed.data }
  }
  return { ok: false, errors: formatZodError(parsed.error) }
}

The design decision here is that two different failures produce one shape. Argument text can fail in two ways: it is not valid JSON at all (a syntax error, an unterminated string, no JSON present), or it parses into a perfectly good JavaScript value that happens to violate the schema (wrong types, missing fields, bad enum). These feel like different problems, and a lot of code handles them separately, but from the repair loop's point of view they are identical: the model produced arguments we cannot use, and we need to tell it why and ask again. Collapsing both into { ok: false, errors } means the loop has exactly one branch to reason about. Note also the use of safeParse rather than parse: we never want validation to throw here, because a thrown error would break out of our controlled loop. safeParse returns a discriminated union we can branch on cleanly.

Here is what validation looks like in practice, from the test suite:

import { describe, expect, it } from 'vitest'
import { z } from 'zod'
import {
  extractJson,
  formatZodError,
  parseAndValidate,
} from '../src/validate.js'

describe('parseAndValidate', () => {
  const schema = z.object({
    city: z.string(),
    units: z.enum(['celsius', 'fahrenheit']),
  })

  it('accepts valid JSON that matches the schema', () => {
    const result = parseAndValidate(
      schema,
      '{"city":"Austin","units":"celsius"}'
    )
    expect(result.ok).toBe(true)
    if (result.ok)
      expect(result.value).toEqual({ city: 'Austin', units: 'celsius' })
  })

  it('reports invalid JSON as a root-level error', () => {
    const result = parseAndValidate(schema, '{"city":"Austin", units:}')
    expect(result.ok).toBe(false)
    if (!result.ok) expect(result.errors).toContain('not valid JSON')
  })

  it('reports a schema violation with the offending field', () => {
    const result = parseAndValidate(
      schema,
      '{"city":"Austin","units":"kelvin"}'
    )
    expect(result.ok).toBe(false)
    if (!result.ok) expect(result.errors).toContain('units:')
  })
})

Three cases, three shapes: a clean pass, a JSON syntax error surfaced as a root problem, and a schema violation that names units as the offender. The middle case — units: with a trailing empty value — is genuinely malformed JSON, and it comes back as a root error rather than crashing. The third case parses fine but kelvin is not in the enum, so it comes back naming the field. That is the exact feedback the model needs to fix it.

Part 4: the bounded auto-repair loop

This is the piece that makes the layer self-healing, and it is also the piece most likely to hurt you if you build it carelessly, because a repair loop is a loop that calls a language model, and a loop that calls a language model can burn money and time without bound if you let it. The entire discipline here is the word bounded.

First, the seam to the model. When validation fails, we need to ask the model for corrected arguments, and we want to do that without the core importing a provider SDK. Define the interface in src/types.ts:

/**
 * A raw tool/function call as emitted by an LLM: the tool name plus the
 * arguments exactly as the model wrote them. `rawArguments` is untrusted text —
 * it may be malformed JSON, be missing fields, carry wrong types, or invent
 * enum values. Nothing here has been validated yet.
 */
export interface ToolCall {
  readonly toolName: string
  /** The arguments exactly as the model emitted them — untrusted JSON text. */
  readonly rawArguments: string
}

/** Everything the model needs to correct a rejected tool call on a repair turn. */
export interface RepairRequest {
  readonly toolName: string
  /** A short description of the schema the args must satisfy. */
  readonly schemaDescription: string
  /** The argument text that just failed validation. */
  readonly previousArguments: string
  /** Human-readable, field-level validation errors from Zod. */
  readonly errors: string
  /** 1-based index of the repair turn (attempt 2 is the first repair). */
  readonly attempt: number
}

/**
 * The single seam between this layer and a real language model. When a tool
 * call fails validation, we hand the model the specific errors and ask for
 * corrected arguments. A deterministic mock implements this with canned
 * strings; production wraps a provider SDK. The core never imports an SDK.
 */
export interface ModelClient {
  repairToolCall(request: RepairRequest): Promise<string>
}

The ModelClient interface has exactly one method, and it takes a RepairRequest carrying everything the model needs to fix its mistake: the tool name, a description of the schema, the previous arguments that failed, and the specific validation errors. In production this method wraps your provider SDK; in every test it is a mock returning a canned string. This is the same one-interface discipline that makes the rest of the AI stack testable, from a multi-provider client to a parallel subagent orchestrator: one narrow seam, mock on one side, real provider on the other.

We also need result types. A resolution either succeeds with typed args and an attempt count, or fails with a reason:

/** Why a tool call could not be resolved to valid, typed arguments. */
export type FailureReason = 'unknown_tool' | 'exhausted_attempts'

/** Discriminated outcome of resolving raw arguments to a validated value. */
export type ResolveResult<T> =
  | {
      readonly ok: true
      readonly toolName: string
      readonly args: T
      /** Attempts used. 1 means valid on the first pass, no repair needed. */
      readonly attempts: number
    }
  | {
      readonly ok: false
      readonly toolName: string
      readonly reason: FailureReason
      readonly attempts: number
      /** The last validation-error feedback, kept for logging. */
      readonly lastErrors: string
    }

Now the loop itself. Create src/repair.ts:

import type { z } from 'zod'
import type { ToolDefinition } from './toolRegistry.js'
import type { ModelClient, ResolveResult, ToolCall } from './types.js'
import { parseAndValidate } from './validate.js'

const DEFAULT_MAX_ATTEMPTS = 3

export interface RepairOptions {
  /**
   * Total attempts allowed, including the first pass. Must be at least 1. A
   * value of 1 disables repair entirely (validate once, then give up). Default
   * is 3: the first pass plus two repair turns.
   */
  readonly maxAttempts?: number
  /** Optional schema description fed to the model on repair. Defaults to the tool's own description. */
  readonly schemaDescription?: string
}

/**
 * Resolve a raw tool call to validated, fully-typed arguments, running a
 * BOUNDED auto-repair loop on failure.
 *
 * The loop is the heart of the layer. On each attempt we parse and validate the
 * current argument text. On success we return the typed value and the attempt
 * count. On failure we feed the *specific* Zod errors back to the model and ask
 * it to re-emit corrected arguments — but only while attempts remain. The hard
 * cap is what separates a self-healing layer from an infinite, token-burning
 * loop: a model that cannot satisfy the schema will fail the same way forever,
 * so we stop and settle as `exhausted_attempts` rather than asking again.
 */
export async function resolveArguments<Schema extends z.ZodTypeAny, Result>(
  tool: ToolDefinition<Schema, Result>,
  call: ToolCall,
  model: ModelClient,
  options: RepairOptions = {}
): Promise<ResolveResult<z.infer<Schema>>> {
  const maxAttempts = options.maxAttempts ?? DEFAULT_MAX_ATTEMPTS
  if (maxAttempts < 1) {
    throw new Error('maxAttempts must be at least 1')
  }
  const schemaDescription = options.schemaDescription ?? tool.description

  let current = call.rawArguments
  let lastErrors = ''

  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
    const result = parseAndValidate(tool.schema, current)
    if (result.ok) {
      return {
        ok: true,
        toolName: tool.name,
        args: result.value,
        attempts: attempt,
      }
    }
    lastErrors = result.errors

    // Do not ask for a repair we have no attempt left to validate — that would
    // burn a model call whose result we could never use.
    if (attempt >= maxAttempts) {
      break
    }

    current = await model.repairToolCall({
      toolName: tool.name,
      schemaDescription,
      previousArguments: current,
      errors: lastErrors,
      attempt: attempt + 1,
    })
  }

  return {
    ok: false,
    toolName: tool.name,
    reason: 'exhausted_attempts',
    attempts: maxAttempts,
    lastErrors,
  }
}

Read the loop carefully, because every line is load-bearing. We start with the raw arguments from the tool call. On each iteration we run parseAndValidate. If it succeeds, we return immediately with the typed value and the current attempt number — so attempts === 1 means the model got it right with no repair at all, which is the metric you most want to track. If validation fails, we save the errors and then check: is this the last allowed attempt? If it is, we break out and settle as failed. We do not call the model for a repair we would have no attempt left to validate — that would spend a model call whose output we could never use. Only if attempts remain do we call model.repairToolCall, handing it the specific errors, and loop back with the corrected text.

The guard at the top — the check that throws when maxAttempts is less than 1 — is deliberate. A maxAttempts of zero would mean "never even validate," which is never what anyone intends; it is a caller bug, so we surface it loudly rather than silently doing nothing.

The single most important property of this loop is that it terminates. The counter goes up by one every iteration and the loop exits when it reaches maxAttempts, full stop. There is no path where a stubborn model keeps the loop alive. If the model returns the same broken arguments on every turn — which happens, and we will test exactly that — the loop still runs at most maxAttempts times and then gives up with exhausted_attempts and the last set of errors attached. That bound is the difference between a self-healing layer and a runaway bill.

The arithmetic of a bounded repair loop is forgiving in a way that is easy to underestimate. Suppose the model produces valid arguments eighty percent of the time on any given attempt, and suppose failures are roughly independent because each repair turn shows the model a different, more specific error. One attempt gets you eighty percent. Two attempts — the first pass plus one repair — get you to ninety-six percent, because the only way to fail twice is to fail two independent tries. Three attempts push past ninety-nine percent. You buy a dramatic reliability improvement for a worst case of exactly three model calls, and the vast majority of calls that succeed on the first pass cost you nothing extra at all.

There is a caveat the chart hides, and it is the same one that governs all retries: repair only helps when the failures are independent. If the model fails because the prompt is fundamentally confusing, or because the schema is asking for something impossible, then all three attempts fail the same way and you have spent triple the tokens to learn what one attempt told you. Repair is insurance against the model's ordinary noisiness, not a substitute for a schema and a prompt that make sense. When your metrics show a tool with a high repair rate that never converges, that is not a repair-loop problem — it is a signal that the tool's schema or description needs work.

Part 5: safe typed dispatch

With resolution in hand, dispatch is short, but it is where the type safety pays off. Create src/dispatch.ts:

import type { z } from 'zod'
import { resolveArguments, type RepairOptions } from './repair.js'
import type {
  AnyToolDefinition,
  ToolDefinition,
  ToolRegistry,
} from './toolRegistry.js'
import type {
  DispatchResult,
  MetricsCollector,
  ModelClient,
  ToolCall,
} from './types.js'

export interface DispatchOptions extends RepairOptions {
  /** Optional collector; when present, every dispatch records its outcome. */
  readonly metrics?: MetricsCollector
}

/**
 * Safely dispatch a raw tool call to a single, statically-known tool. This is
 * the fully-typed path: `resolveArguments` guarantees `handler` only ever runs
 * on arguments that satisfied the schema, so inside the handler `args` has the
 * exact inferred type and no `any` leaks through. A failed resolution never
 * touches the handler — it returns an `ok: false` result the caller must handle.
 */
export async function dispatchTool<Schema extends z.ZodTypeAny, Result>(
  tool: ToolDefinition<Schema, Result>,
  call: ToolCall,
  model: ModelClient,
  options: DispatchOptions = {}
): Promise<DispatchResult<Result>> {
  const resolved = await resolveArguments(tool, call, model, options)

  if (!resolved.ok) {
    options.metrics?.record('failed', resolved.attempts)
    return {
      ok: false,
      toolName: tool.name,
      attempts: resolved.attempts,
      reason: resolved.reason,
      errors: resolved.lastErrors,
    }
  }

  options.metrics?.record(
    resolved.attempts === 1 ? 'first_pass' : 'repaired',
    resolved.attempts
  )
  const result = await tool.handler(resolved.args)
  return { ok: true, toolName: tool.name, attempts: resolved.attempts, result }
}

The critical property is structural: the handler is only reachable after a successful resolution. Look at the control flow — if resolved.ok is false, we return early, and the handler line never executes. There is no branch where unvalidated arguments reach tool.handler. And because resolveArguments returns args typed as z.infer<Schema>, the handler receives exactly the type it was written against. This is not a convention you have to remember to follow; it is enforced by the shape of the code. The unsafe path does not exist.

The registry-driven path handles the untrusted-tool-name case:

/**
 * Dispatch a raw tool call through a registry. The tool name comes from the
 * model and is therefore untrusted: if it names a tool that was never
 * registered, we reject with `unknown_tool` instead of throwing or guessing.
 * Result types are erased to `unknown` on this path — that is the price of
 * dispatching by a runtime string; use `dispatchTool` when you know the tool.
 */
export async function dispatchCall(
  registry: ToolRegistry,
  call: ToolCall,
  model: ModelClient,
  options: DispatchOptions = {}
): Promise<DispatchResult<unknown>> {
  const tool: AnyToolDefinition | undefined = registry.get(call.toolName)
  if (!tool) {
    options.metrics?.record('failed', 0)
    return {
      ok: false,
      toolName: call.toolName,
      attempts: 0,
      reason: 'unknown_tool',
      errors: `no tool named "${call.toolName}" is registered`,
    }
  }
  return dispatchTool(tool, call, model, options)
}

dispatchCall is the front door for real tool calls, where the name arrives from the model. It looks the tool up in the registry's allow-list; a name that is not registered — a hallucinated tool, an injected one — returns unknown_tool without ever calling the model or a handler. When the tool is found, it delegates to dispatchTool for the full validated flow. The result type is unknown here because we cannot know at compile time which tool a runtime string names; if you are dispatching a tool you know statically, call dispatchTool directly and keep the precise result type.

Here is the security-critical behavior under test:

describe('dispatchCall (registry)', () => {
  const registry = new ToolRegistry().register(invoiceTool)

  it('rejects an unknown tool name without touching the model', async () => {
    const call: ToolCall = { toolName: 'launch_missiles', rawArguments: '{}' }
    const out = await dispatchCall(registry, call, neverCalled, {})
    expect(out.ok).toBe(false)
    if (!out.ok) expect(out.reason).toBe('unknown_tool')
  })
})

A model that hallucinates launch_missiles gets a clean rejection, and the neverCalled mock model confirms we did not even attempt a repair — there is nothing to repair when the tool does not exist.

Part 6: measuring first-pass rate and repair rate

You cannot manage what you do not measure, and the single most useful number in a tool-calling system is the first-pass-valid rate: how often the model gets the arguments right with no repair at all. A falling first-pass rate is an early warning that a prompt regressed, a schema tightened, or a model version changed under you. Add the collector to src/types.ts:

/** How a single dispatched call ended up, for metrics. */
export type CallOutcome = 'first_pass' | 'repaired' | 'failed'

/** An immutable read of the collector's counters plus derived rates. */
export interface MetricsSnapshot {
  readonly totalCalls: number
  readonly firstPassValid: number
  readonly repaired: number
  readonly failed: number
  readonly totalAttempts: number
  /** firstPassValid / totalCalls, in the range 0..1 (0 when no calls yet). */
  readonly firstPassValidRate: number
  /** repaired / totalCalls: calls that needed at least one repair but recovered. */
  readonly repairRate: number
  /** failed / totalCalls: calls that exhausted their attempts. */
  readonly failureRate: number
  /** Mean attempts per call — the direct cost of the repair loop. */
  readonly averageAttempts: number
}

/**
 * Counts the health of the tool-calling layer. `firstPassValidRate` is the
 * signal that matters most: it tells you how often the model gets the arguments
 * right without any repair, which is the cheapest and fastest path. A falling
 * first-pass rate is an early warning that a prompt, a schema, or a model
 * version has regressed.
 */
export class MetricsCollector {
  private totalCalls = 0
  private firstPassValid = 0
  private repaired = 0
  private failed = 0
  private totalAttempts = 0

  record(outcome: CallOutcome, attempts: number): void {
    this.totalCalls += 1
    this.totalAttempts += attempts
    if (outcome === 'first_pass') {
      this.firstPassValid += 1
    } else if (outcome === 'repaired') {
      this.repaired += 1
    } else {
      this.failed += 1
    }
  }

  snapshot(): MetricsSnapshot {
    const calls = this.totalCalls
    const safe = (n: number) => (calls === 0 ? 0 : n / calls)
    return {
      totalCalls: calls,
      firstPassValid: this.firstPassValid,
      repaired: this.repaired,
      failed: this.failed,
      totalAttempts: this.totalAttempts,
      firstPassValidRate: safe(this.firstPassValid),
      repairRate: safe(this.repaired),
      failureRate: safe(this.failed),
      averageAttempts: safe(this.totalAttempts),
    }
  }
}

The collector is deliberately dumb: it counts outcomes and attempts and computes rates on demand. The safe helper guards against dividing by zero before any call has landed. The three rates tell a complete story. A high firstPassValidRate means your schemas and prompts are well-matched to the model. A high repairRate with a low failureRate means the model stumbles but the loop rescues it — fine, but each repair is an extra round trip, so watch it. A non-trivial failureRate means calls are exhausting their attempts, which points at a schema or prompt problem the repair loop cannot paper over. This is exactly the kind of signal you would export as a span attribute if you were instrumenting the agent with OpenTelemetry: first-pass rate per tool is a metric worth a dashboard.

The metrics are pinned down by a test that exercises all three outcomes:

it('tracks first-pass, repaired, and failed calls with correct rates', async () => {
  const metrics = new MetricsCollector()

  // First-pass valid.
  await dispatchTool(
    tool,
    {
      toolName: 'set_flag',
      rawArguments: '{"name":"a","value":true}',
    } satisfies ToolCall,
    new MockModelClient(['{}']),
    { metrics }
  )

  // Repaired: bad on the first pass, fixed on the repair turn.
  await dispatchTool(
    tool,
    {
      toolName: 'set_flag',
      rawArguments: '{"name":"b","value":"yes"}',
    } satisfies ToolCall,
    new MockModelClient(['{"name":"b","value":true}']),
    { metrics }
  )

  // Failed: never validates.
  await dispatchTool(
    tool,
    {
      toolName: 'set_flag',
      rawArguments: '{"name":"c","value":"nope"}',
    } satisfies ToolCall,
    new MockModelClient(['{"name":"c","value":"still-bad"}']),
    { metrics, maxAttempts: 2 }
  )

  const snap = metrics.snapshot()
  expect(snap.totalCalls).toBe(3)
  expect(snap.firstPassValid).toBe(1)
  expect(snap.repaired).toBe(1)
  expect(snap.failed).toBe(1)
  expect(snap.firstPassValidRate).toBeCloseTo(1 / 3, 5)
})

One of each outcome, and the rates come out to a third apiece. The satisfies ToolCall annotation is a small nicety — it checks the object literal against the type without widening it, so a typo in a field name is a compile error in the test itself.

Part 7: a deterministic mock and the full flow

Every test above uses MockModelClient, and it is worth seeing, because its design is what makes the whole suite deterministic and offline. Create src/mockModelClient.ts:

import type { ModelClient, RepairRequest } from './types.js'

/**
 * A deterministic `ModelClient` for tests and the example — NO network, NO API
 * key. You hand it a queue of repair responses; each call to `repairToolCall`
 * returns the next one, then sticks on the last. That last-response-repeats
 * behaviour lets a test simulate a model that never corrects itself (to drive
 * the `exhausted_attempts` path) simply by queueing a still-broken response.
 *
 * Every request is recorded on `requests`, so tests can assert that the model
 * was actually shown the field-level Zod errors it was supposed to fix.
 */
export class MockModelClient implements ModelClient {
  private index = 0
  readonly requests: RepairRequest[] = []

  constructor(private readonly responses: readonly string[]) {
    if (responses.length === 0) {
      throw new Error('MockModelClient needs at least one queued response')
    }
  }

  async repairToolCall(request: RepairRequest): Promise<string> {
    this.requests.push(request)
    const i = Math.min(this.index, this.responses.length - 1)
    this.index += 1
    return this.responses[i]
  }
}

The mock returns queued responses in order and then sticks on the last one, which is a small trick that pays off. To test a successful repair, you queue one corrected response. To test the exhausted_attempts path, you queue a still-broken response, and because the mock repeats it, every repair turn gets the same bad arguments and the loop runs to its cap. The requests array records every RepairRequest, so a test can assert the model was actually shown the right errors — proving the feedback loop is wired correctly, not just that the counts line up.

Now wire the whole thing together in src/example.ts with two real tools:

import { z } from 'zod'
import { defineTool, ToolRegistry } from './toolRegistry.js'
import { dispatchCall } from './dispatch.js'
import { MetricsCollector, type ToolCall } from './types.js'
import { MockModelClient } from './mockModelClient.js'

const getWeather = defineTool({
  name: 'get_weather',
  description:
    'Look up the current weather for a city. { city: string, units: "celsius" | "fahrenheit" }',
  schema: z.object({
    city: z.string().min(1),
    units: z.enum(['celsius', 'fahrenheit']),
  }),
  handler: args =>
    `Weather for ${args.city} in ${args.units}: 22 degrees, clear.`,
})

const createInvoice = defineTool({
  name: 'create_invoice',
  description:
    'Create an invoice. { customerId: string, amount: number (>0), currency: "USD" | "EUR" | "GBP" }',
  schema: z.object({
    customerId: z.string().min(1),
    amount: z.number().positive(),
    currency: z.enum(['USD', 'EUR', 'GBP']),
  }),
  handler: args =>
    `Invoice for ${args.customerId}: ${args.amount.toFixed(2)} ${args.currency} created.`,
})

const registry = new ToolRegistry().register(getWeather).register(createInvoice)

Notice args.amount.toFixed(2) in the invoice handler. That method only exists on a number, and it type-checks because amount is inferred as number from z.number().positive(). If the model sent "120.5" as a string — which is exactly what it does in the example below — the handler never runs on it, because validation rejects the string before dispatch reaches the handler. The .toFixed call is safe precisely because the layer guarantees the handler only sees a real number. Here is the driver, running one clean call, one that repairs, and one that fails:

async function main(): Promise<void> {
  const metrics = new MetricsCollector()

  // 1) A clean first-pass call: the model got it right, no repair needed.
  const weatherCall: ToolCall = {
    toolName: 'get_weather',
    rawArguments: '{"city":"Austin","units":"celsius"}',
  }
  const weather = await dispatchCall(
    registry,
    weatherCall,
    new MockModelClient(['{}']),
    {
      metrics,
    }
  )
  console.log('get_weather:', weather)

  // 2) A malformed call that repairs. The model first emits a wrong enum value
  //    and a stringified amount; shown the errors, it re-emits valid arguments.
  const invoiceCall: ToolCall = {
    toolName: 'create_invoice',
    rawArguments:
      'Sure! Here are the arguments:\n```json\n{"customerId":"cus_42","amount":"120.5","currency":"dollars"}\n```',
  }
  const repairModel = new MockModelClient([
    '{"customerId":"cus_42","amount":120.5,"currency":"USD"}',
  ])
  const invoice = await dispatchCall(registry, invoiceCall, repairModel, {
    metrics,
  })
  console.log('create_invoice:', invoice)

  // 3) A call the model can never fix: it keeps returning a negative amount.
  const badCall: ToolCall = {
    toolName: 'create_invoice',
    rawArguments: '{"customerId":"cus_9","amount":-5,"currency":"USD"}',
  }
  const stubborn = new MockModelClient([
    '{"customerId":"cus_9","amount":-5,"currency":"USD"}',
  ])
  const failed = await dispatchCall(registry, badCall, stubborn, {
    metrics,
    maxAttempts: 3,
  })
  console.log('create_invoice (unfixable):', failed)

  console.log('\nmetrics:', metrics.snapshot())
}

The second call is the whole tutorial in one place. The raw arguments are wrapped in prose and a code fence, the amount is a string, and the currency is the hallucinated dollars. extractJson peels off the wrapper, parseAndValidate rejects the string amount and the bad enum, formatZodError turns those into feedback, the repair loop hands them to the mock model, the mock returns corrected arguments, validation passes on the second attempt, and the handler runs. Running npm run example prints exactly this:

get_weather: { ok: true, toolName: 'get_weather', attempts: 1,
  result: 'Weather for Austin in celsius: 22 degrees, clear.' }
create_invoice: { ok: true, toolName: 'create_invoice', attempts: 2,
  result: 'Invoice for cus_42: 120.50 USD created.' }
create_invoice (unfixable): { ok: false, toolName: 'create_invoice',
  attempts: 3, reason: 'exhausted_attempts',
  errors: '- amount: Number must be greater than 0' }
metrics: { totalCalls: 3, firstPassValid: 1, repaired: 1, failed: 1,
  totalAttempts: 6, firstPassValidRate: 0.333..., repairRate: 0.333...,
  failureRate: 0.333..., averageAttempts: 2 }

The weather call succeeds on attempt one, the invoice repairs to attempt two, and the unfixable call runs its full three-attempt budget before giving up with the last error attached. The metrics summarize all three at a glance.

Pitfalls to avoid

The design above is shaped by three failure modes that are easy to walk into.

Infinite repair loops. This is the cardinal sin, and the whole maxAttempts mechanism exists to prevent it. If your repair loop has no hard cap, a model that cannot satisfy a schema will loop forever, and because each turn is a paid model call, "forever" is also "expensive." The bound must be a small constant — three is a good default — and the loop must count up and exit unconditionally when it hits the cap, with no clever "just one more try if it looks close" escape hatch. The test that queues a permanently-broken response and asserts the loop stops at three is not optional; it is the test that proves you cannot bankrupt yourself.

Over-permissive schemas. A schema that accepts too much is a validation layer that validates nothing. z.string() for a field that should be one of three enum values means a hallucinated value sails straight through. z.number() for an amount that must be positive and bounded means a negative or absurd number is "valid." The tighter the schema, the more the layer protects you — and, as a bonus, the better the repair feedback, because a specific constraint produces a specific error the model can fix. Use z.enum for closed sets, .positive(), .min(), .max(), .int(), .url(), .email(), and every other refinement Zod gives you. The schema is your security boundary; make it strict. This is the same principle you would apply when signing and verifying agent commit provenance: the value of a check is exactly the strictness of what it rejects.

Non-determinism. A model is not a pure function; the same prompt can produce different arguments on different calls. This has two consequences. First, never write tests against a live model — they will flake, and a flaky test suite is a suite people stop trusting. Route everything through the ModelClient interface and mock it, so your tests are deterministic and fast. Second, in production, budget for the model to occasionally fail even a call it usually gets right; that is what the repair loop and the metrics are for. Watch the first-pass rate over time, and when it drifts, investigate the prompt and the schema rather than raising maxAttempts and hoping.

Wiring a real provider

The mock is what makes the tests fast; at some point you want real tokens. Because the model lives behind one interface, wiring a real provider is a single adapter. Here is ModelClient backed by Anthropic's SDK — no changes to the registry, the validator, the repair loop, or your tool definitions:

import Anthropic from '@anthropic-ai/sdk'
import type {
  ModelClient,
  RepairRequest,
} from 'typesafe-llm-tool-calling-zod-typescript'

const client = new Anthropic()

export const anthropicRepair: ModelClient = {
  async repairToolCall(req: RepairRequest): Promise<string> {
    const res = await client.messages.create({
      model: 'claude-opus-4-8',
      max_tokens: 1024,
      messages: [
        {
          role: 'user',
          content:
            `The arguments you produced for the tool "${req.toolName}" failed ` +
            `validation.\n\nSchema: ${req.schemaDescription}\n\n` +
            `Your previous arguments:\n${req.previousArguments}\n\n` +
            `Validation errors:\n${req.errors}\n\n` +
            `Reply with ONLY corrected JSON arguments. No prose, no code fence.`,
        },
      ],
    })
    return res.content.map(b => (b.type === 'text' ? b.text : '')).join('')
  },
}

The repair prompt is doing real work: it names the tool, restates the schema, shows the model its own failed attempt, and lists the exact errors, then asks for corrected JSON and nothing else. That structure is why the loop tends to converge in a single repair — the model is not guessing what went wrong, it is being told. Swapping providers means writing a different repairToolCall, and nothing else in the layer changes.

Running it

The companion project uses Vitest and TypeScript with nodenext module resolution, which is why every internal import carries a .js extension even though the files are .ts. From the project directory:

npm install
npm run typecheck   # tsc --noEmit
npm test            # vitest run
npm run example     # tsx src/example.ts

You should see all four test files green — the registry, the validator, the dispatcher and repair loop, and the metrics — twenty tests in total, running in well under a second because nothing touches the network. The only "model" in the suite is MockModelClient returning strings.

Where to take it next

What you have is the load-bearing core, and it is deliberately small. A few directions to extend it, in rough order of how often you will reach for them.

First, structured logging of every RepairRequest and outcome. The RepairRequest already carries the tool name, the failed arguments, and the errors; emit a log line per repair and you have an audit trail of exactly which tools stumble and why. That is the raw data behind a "which schema needs work" dashboard.

Second, per-tool maxAttempts. A cheap, forgiving tool might warrant three repair turns; an expensive one might warrant one. Thread the budget through the tool definition and let each tool tune its own cap.

Third, a repair-prompt template per tool. The generic prompt in the adapter above works, but some tools benefit from an example of correct arguments in the repair message. Add an optional repairHint to the tool definition and splice it into the prompt.

Fourth, parallel dispatch of independent tool calls. When a model emits several tool calls at once, you can resolve and dispatch them concurrently — which is exactly where a parallel subagent orchestrator plugs in, with each tool call becoming a bounded, retried unit of work.

None of these change the shape of the core. They hang off the seams it already exposes — the ModelClient interface, the ToolDefinition, the MetricsCollector — which is the sign the core was factored correctly. The value of the whole layer is a single, boring guarantee: no handler in your system ever runs on arguments that did not pass a strict schema, and when the model gets it wrong, the system tells the model exactly what was wrong and gives it a bounded chance to fix itself. That guarantee is the difference between a tool-calling demo and a tool-calling system you can put in front of users.

Further Reading

• Build a resilient multi-provider LLM client in TypeScript — the client that produces the completions your tool-calling layer validates.
• Build a parallel subagent orchestrator in TypeScript — fan out independent tool calls with bounded concurrency and retries.
• Build an LLM eval harness with an LLM judge in TypeScript — measure whether your prompts and schemas actually work before you ship them.
• Instrument an MCP agent with OpenTelemetry tracing — export first-pass rate and repair rate as span attributes on a real dashboard.
• Verifiable agent commit provenance in TypeScript — the same strict-boundary discipline applied to what your agent commits.