Parse Streaming JSON From an LLM: A Tolerant Partial-JSON Parser in TypeScript

Structured output is how most LLM features actually ship. You do not ask the model for prose and hope; you ask it for a JSON object with a fixed shape — a triage result, an extraction, a set of tool arguments — and you render the fields. The moment you also want that output to stream, so the interface feels alive instead of frozen behind a spinner, you hit a wall that has nothing to do with the model and everything to do with the language runtime: JSON.parse throws on anything that is not a complete document.

Watch what that means in practice. The model streams the object one token at a time. After the first few tokens the buffer holds {"category":"bil. That is a perfectly reasonable prefix of the final answer, and a human can already see where it is going. JSON.parse cannot. It throws. It throws on the next chunk too, and the one after that. For a small support-ticket object serialized to 210 characters, JSON.parse throws on 209 of the 210 prefixes — every single one except the last. Your progressive UI has exactly one moment when it can render anything, and it is the moment the stream ends, which is the moment you no longer needed streaming at all.

This tutorial builds the missing piece: a tolerant partial-JSON parser that takes any prefix of a JSON document and returns the best-effort value parsed so far, closing whatever structure is still open. Feed it {"category":"bil and it returns { category: "bil" }. Feed it the buffer one chunk longer and it returns one field more complete. It never throws, it has zero runtime dependencies, and it is covered by 49 offline tests you can run without an API key. The complete, runnable project lives in the CrashBytes ByteSizedExamples repository.

Why this problem is worth a library

The naive fixes all look tempting and all fail in ways that surface in production, not in the demo.

The first instinct is to wait. Buffer the whole stream, parse once at the end, render. This works and is what most teams ship first, but it throws away the entire point of streaming: the user stares at a spinner for the full generation time even though the first field was decided in the first few hundred milliseconds. On a long object — an extraction with a paragraph-length summary field in the middle — the fields after the long one are invisible until the long one finishes, which is the worst possible ordering.

The second instinct is to count braces. Track how many { and [ you have seen, append the matching number of } and ], and parse that. This handles the shallow happy path and breaks the instant a brace appears inside a string. The buffer {"note":"a } b is not an object with an extra close brace; the } is a literal character inside a string value, and a brace-counter will either close the object early or miscount the depth. Strings are where naive JSON handling goes to die, because every structural character — {, }, [, ], ,, :, " — is also a legal character inside a string.

The third instinct is to reach for a regex. There is no regular expression that parses JSON; JSON is a recursive grammar and regex is not, and the moment you have nested objects the approach collapses. People try anyway. It is always a mistake.

The honest solution is a small, purpose-built parser that understands JSON's grammar well enough to know where it is when the buffer runs out — inside a string, between a key and its colon, halfway through a number — and repairs the buffer accordingly. That is what we build, in three parts: a tokenizer that tracks string and escape state precisely, a tolerant parser that auto-closes open containers, and a thin accumulator that re-parses the growing buffer on every streamed chunk.

Those are not invented numbers — they are measured against the exact demo object in the companion project. The tolerant parser exposes the first field about a tenth of the way through the stream and has three of the seven fields readable by the one-third mark. JSON.parse sits flat on zero until the final byte and then jumps to all seven at once. The flat stretch in the partial curve between 30 and 60 percent is real and instructive: the summary field in the middle is a long string, and while it streams, no new top-level field can resolve — but, crucially, the three fields you already had stay stable and rendered the whole time.

What you will build

The library is three modules' worth of responsibility, and you implement them in order:

1. A tokenizer that scans the raw buffer left to right into JSON tokens, marking only the final token as incomplete when the buffer ends mid-token.
2. A tolerant parser that walks those tokens with an explicit stack of open objects and arrays, and auto-closes everything still open at the end.
3. A StreamingJsonAccumulator that holds the growing buffer and re-parses it on every push, so a UI can bind directly to the latest best-effort value.

Everything is synchronous and pure. There is no network, no async, no dependency on any particular model SDK. You drive it with strings, which means you can test every edge case deterministically and offline — and the 49 tests in the repo do exactly that.

We build in TypeScript with strict compiler settings, ESM modules, and Vitest for the test suite, mirroring the toolchain used across the other TypeScript LLM tutorials in the CrashBytes companion repo — for example the type-safe LLM tool-calling library built on Zod, which produces the very kind of structured output this parser consumes as it streams.

The shape of the data

Before any parsing, define what a token is. A JSON token is one of the structural punctuation marks, a string, a number, one of the three literals, or — the part that matters for streaming — an incomplete tail.

type Token =
  | { type: '{' | '}' | '[' | ']' | ':' | ','; complete: true }
  | { type: 'string'; value: string; complete: boolean }
  | { type: 'number'; value: number; complete: boolean }
  | { type: 'true' | 'false' | 'null'; value: boolean | null; complete: true }
  | { type: 'literal'; value: boolean | null; complete: false }
  | { type: 'garbage'; complete: false }

The complete flag is the whole design in one field. Because the tokenizer scans strictly left to right and stops at the end of the buffer, only the last token can ever be incomplete: everything before it was, by definition, followed by more characters, so it terminated. That single invariant — only the tail can be incomplete — collapses an intimidating problem into a tractable one. The parser handles the body of the token stream as ordinary complete JSON and applies its special tolerance to exactly one token: the tail.

A literal token is a partial true/false/null (say the buffer ended at tr), and garbage is a run of letters that is not a prefix of any literal. Both are, by construction, never complete — they only ever appear as that final tail token.

Part 1: a tokenizer that respects strings

The tokenizer is where correctness is won or lost, because strings are where the structural characters hide. Here is the scanner for a single string literal, which is the most important function in the library:

const HEX4 = /^[0-9a-fA-F]{4}$/

function scanString(
  s: string,
  start: number
): { raw: string; end: number; complete: boolean } {
  let i = start + 1
  let contentEnd = i // exclusive end of the safely-scanned content

  while (i < s.length) {
    const c = s[i]

    if (c === '\\') {
      const next = s[i + 1]
      if (next === undefined) break // lone trailing backslash: drop it
      if (next === 'u') {
        const hex = s.slice(i + 2, i + 6)
        if (!HEX4.test(hex)) break // incomplete \uXXXX at buffer end: drop it
        i += 6
        contentEnd = i
        continue
      }
      i += 2 // simple escape: \" is an escaped quote, NOT a terminator
      contentEnd = i
      continue
    }

    if (c === '"') {
      return { raw: s.slice(start, i + 1), end: i + 1, complete: true }
    }

    i += 1
    contentEnd = i
  }

  // Buffer ended before the closing quote: close it at the last safe point.
  return {
    raw: s.slice(start, contentEnd) + '"',
    end: s.length,
    complete: false,
  }
}

Three things in this function are doing real work. First, the backslash branch: when it sees \, it consumes two characters, so an escaped quote \" can never be mistaken for the string terminator. This is the single most common bug in hand-rolled JSON handling, and it is fixed here by construction. Second, the \u sub-branch checks that four hex digits actually follow; if the buffer ended in the middle of a \u00 escape, it stops before the broken escape and closes the string at the last valid character, so the repaired literal is always parseable. Third, contentEnd tracks the last position that formed valid string content, so when the buffer runs out mid-string the function returns s.slice(start, contentEnd) + '"' — the good part, re-closed. The returned raw is always a syntactically valid JSON string, which means the tokenizer can lean on the platform's own JSON.parse to decode escapes rather than re-implementing Unicode handling:

if (c === '"') {
  const r = scanString(s, i)
  // r.raw is always a valid JSON string literal, so this never throws.
  tokens.push({
    type: 'string',
    value: JSON.parse(r.raw) as string,
    complete: r.complete,
  })
  i = r.end
  continue
}

Reusing JSON.parse on a guaranteed-valid fragment is a deliberate choice. The library is tolerant at the grammar level, where the streaming problem lives, and strict at the lexical level, where the platform already has a correct, fast, battle-tested implementation. You do not want to be the person who re-implemented surrogate-pair decoding and got it subtly wrong.

Part 2: numbers and literals — the ambiguity rule

Numbers force a decision that goes to the heart of what "best effort" means. A number is complete only if it matches the JSON number grammar:

const VALID_NUMBER = /^-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?$/
const NUMBER_CHAR = /[-+0-9.eE]/

function scanNumber(s: string, start: number) {
  let i = start
  while (i < s.length && NUMBER_CHAR.test(s[i]!)) i += 1
  const raw = s.slice(start, i)
  return { raw, end: i, complete: VALID_NUMBER.test(raw) }
}

If the buffer ends at 1., or 1e, or a lone -, that number is incomplete and — this is the design decision — it is dropped along with its key. The reason is that a partial number is genuinely ambiguous in a way a partial string is not. The buffer {"amount":1. could be resolving to 1.0 or 1.5 or 1.999; there is no honest "so far" value. Showing 1 would be inventing data. So the parser declines to guess, drops the field, and lets it appear a chunk later when the number is unambiguous. A partial string, by contrast, has a truthful prefix — "hel" really is the first three characters of whatever is coming — so it is kept.

Literals get the opposite treatment, and for a precise reason:

const LITERALS = ['true', 'false', 'null'] as const

function scanLiteral(s: string, start: number) {
  let i = start
  while (i < s.length && /[a-z]/.test(s[i]!)) i += 1
  const raw = s.slice(start, i)

  if (raw === 'true' || raw === 'false' || raw === 'null') {
    return {
      token: { type: raw, value: literalValue(raw), complete: true },
      end: i,
    }
  }
  const match = LITERALS.find(l => raw.length > 0 && l.startsWith(raw))
  if (match) {
    // Unambiguous prefix -> resolve it. t->true, f->false, n->null.
    return {
      token: { type: 'literal', value: literalValue(match), complete: false },
      end: i,
    }
  }
  return { token: { type: 'garbage', complete: false }, end: i }
}

The three JSON literals start with three different letters — t, f, n — so any non-empty prefix identifies exactly one of them. A buffer ending in tr can only be heading toward true; there is no other legal continuation. That makes the resolution safe, so {"done":tr yields { done: true }. This is the mirror image of the number rule: literals are resolved because their prefix is unambiguous, numbers are dropped because theirs is not. Getting these two opposite-looking rules from the same underlying principle — keep what is truthful, drop what is a guess — is the tutorial's central lesson.

Part 3: the tolerant, auto-closing parser

With tokens in hand, the parser walks them using an explicit stack of frames. An object frame remembers the object being built, the current key, and what it expects to see next; an array frame remembers the array and its state.

type ObjectFrame = {
  type: 'object'
  obj: Record<string, unknown>
  currentKey: string | undefined
  state: 'key' | 'colon' | 'value' | 'comma'
}
type ArrayFrame = { type: 'array'; arr: unknown[]; state: 'value' | 'comma' }
type Frame = ObjectFrame | ArrayFrame

Tracking the object's state — is it expecting a key, a colon, a value, or a comma next? — is what lets the parser make the right call about the tail token. The same incomplete string means different things in different positions: {"na is a partial key and gets dropped, while {"name":"Ad is a partial value and gets kept. The state machine is how the parser knows which it is looking at.

The main loop processes every token except the incomplete tail as ordinary, complete JSON, pushing frames on {/[, popping and attaching them on }/], and routing strings to either the key slot or a value depending on the frame's state:

case 'string': {
  const top = stack[stack.length - 1]
  if (top && top.type === 'object' && top.state === 'key') {
    top.currentKey = tok.value
    top.state = 'colon'
  } else {
    attachValue(tok.value)
  }
  break
}

After the main loop, the parser handles the one incomplete tail token according to the frame it lands in — keeping partial values, dropping partial keys — and then does the thing that makes the whole approach work: it auto-closes every container still on the stack, innermost first.

// Incorporate the trailing incomplete token according to grammar position.
if (tail) {
  const top = stack[stack.length - 1]
  if (
    top?.type === 'object' &&
    top.state === 'value' &&
    (tail.type === 'string' || tail.type === 'literal')
  ) {
    top.obj[top.currentKey as string] = tail.value
    top.state = 'comma'
    top.currentKey = undefined
  } else if (
    top?.type === 'array' &&
    top.state === 'value' &&
    (tail.type === 'string' || tail.type === 'literal')
  ) {
    top.arr.push(tail.value)
  }
  // partial key / number / garbage -> dropped by omission
}

// Auto-close every still-open container from the innermost outward.
while (stack.length > 0) {
  const frame = stack.pop()!
  attachClosed(frame.type === 'array' ? frame.arr : frame.obj)
}

return hasRoot ? root : undefined

Because the frames hold real, progressively-filled objects and arrays, closing them is just a matter of attaching each finished container to its parent. A deeply nested partial like {"user":{"profile":{"name":"Ada closes from the inside out — the innermost object, then the middle, then the root — and produces { user: { profile: { name: "Ada" } } }. Arbitrary nesting comes for free because the stack is doing the bookkeeping.

The public entry point wraps this in a belt-and-suspenders guard so the never-throw guarantee holds even against inputs the author did not foresee:

export function parsePartialJson(buffer: string): unknown {
  try {
    return parseInternal(buffer)
  } catch {
    return undefined
  }
}

A parser you call on every streamed chunk must never throw, because a throw in that path is a crash in the render loop. The internal parser is written so it cannot throw on any prefix of valid JSON; the outer try/catch is there so that even a genuinely malformed model output degrades to undefined instead of taking down the component. Resilience at the boundary is a theme across these tutorials — the same instinct drives the retry and failover logic in the resilient multi-provider LLM client.

Part 4: the streaming accumulator

The parser is pure and stateless. Streaming needs a thin stateful wrapper that holds the growing buffer and re-parses on each chunk:

export class StreamingJsonAccumulator {
  #buffer = ''
  #value: unknown = undefined

  push(chunk: string): unknown {
    this.#buffer += chunk
    this.#value = parsePartialJson(this.#buffer)
    return this.#value
  }

  get buffer(): string {
    return this.#buffer
  }
  get value(): unknown {
    return this.#value
  }
  reset(): void {
    this.#buffer = ''
    this.#value = undefined
  }
}

That is the entire streaming interface. Every time a token arrives from the model, you push it and get back the best-effort object so far. Re-parsing the full buffer on every chunk is O(n) per chunk and O(n²) over a whole stream, which sounds alarming until you remember the actual sizes: structured LLM outputs are kilobytes, not megabytes, and re-parsing a few kilobytes on each of a few hundred chunks is microseconds of work next to the tens of milliseconds between tokens. Simplicity wins here; if you ever genuinely stream a megabyte object, an incremental parser is a worthwhile complication, but you will almost certainly never need it.

Wiring it to a real model stream is then trivial. Whatever your SDK's streaming primitive is — an async iterator of text deltas is the common shape — you feed each delta straight in:

const acc = new StreamingJsonAccumulator()

for await (const delta of model.streamStructuredOutput(prompt)) {
  const partial = acc.push(delta.text)
  render(partial) // bind your UI to the latest best-effort object
}

render gets called with an object that grows a field at a time. Your framework diffs it and updates only what changed, so the category badge appears, then the priority, then the summary types itself in — the interface behaves the way users already expect streaming to behave, because they have watched chat responses stream for years. The only reason structured output usually doesn't stream is that JSON.parse made it hard, and now it is not.

Part 5: proving it with tests

The test suite is where a parser like this earns trust, because the failure modes are all edge cases and edge cases are invisible until someone hits them. The companion project ships 49 Vitest cases; the most important one is not any single example but a property: for every prefix of a valid document, the parser does not throw.

import { describe, it, expect } from 'vitest'
import { parsePartialJson } from './partial-json'

describe('never throws on any prefix', () => {
  const docs = [
    '{"category":"billing","tags":["a","b"],"confidence":0.93,"escalate":true}',
    '[1,-2,3.5,true,false,null,"x",{"nested":[{"deep":true}]}]',
    '{"msg":"a \\"quoted\\" word and a \\n newline","n":-12.5e3}',
  ]
  for (const doc of docs) {
    for (let i = 1; i <= doc.length; i++) {
      const prefix = doc.slice(0, i)
      it(`prefix length ${i} of ${doc.length}`, () => {
        expect(() => parsePartialJson(prefix)).not.toThrow()
      })
    }
  }
})

This single generator produces hundreds of assertions, one per prefix length across several documents chosen to exercise escapes, negatives, exponents, and nesting. If any prefix ever throws — a broken escape, a miscounted depth, a number scanned wrong — one of these fails and names the exact prefix length, so you can slice the document there and see precisely what tripped it. It is the cheapest, highest-value test in the suite, and it is the one that lets you refactor the parser fearlessly.

The targeted cases then pin down the deliberate behaviors so a future edit cannot quietly change them:

it('closes and keeps a partial string value', () => {
  expect(parsePartialJson('{"msg":"hel')).toEqual({ msg: 'hel' })
})

it('drops a partial key', () => {
  expect(parsePartialJson('{"name":"Ada","ag')).toEqual({ name: 'Ada' })
})

it('resolves an unambiguous partial literal', () => {
  expect(parsePartialJson('{"done":tr')).toEqual({ done: true })
})

it('drops an ambiguous partial number', () => {
  expect(parsePartialJson('{"n":1.')).toEqual({})
})

it('never mistakes an escaped quote for a terminator', () => {
  expect(parsePartialJson('{"note":"a \\" b')).toEqual({ note: 'a " b' })
})

it('closes deep nesting from the inside out', () => {
  expect(parsePartialJson('{"u":{"p":{"name":"Ada')).toEqual({
    u: { p: { name: 'Ada' } },
  })
})

Each of these is a specification as much as a test. The behavior table earlier in this article is not documentation that can drift from the code; it is exactly what these assertions enforce, and if someone "fixes" the partial-number rule to show 1 instead of dropping the field, the fourth test goes red and the pull request has to justify the change on purpose.

The design decisions, stated plainly

A tolerant parser makes choices, and hiding them is how you get a library people cannot reason about. Here are the ones this library makes, and why.

Empty or whitespace-only input returns undefined, never throws. There is no value in an empty buffer, and the streaming caller wants a benign "nothing yet," not an exception on the first chunk.

Partial string values are kept; partial keys are dropped. A value prefix is truthful — the characters you have really are the start of the value. A key prefix is not usable: you cannot attach a value to half a key without inventing the other half, and a key that changes identity as more characters arrive would make the object's shape flicker. So a value streams in character by character and a key only appears once it is whole.

Unambiguous partial literals resolve; ambiguous partial numbers are dropped. Same principle from two directions: t/f/n each identify one literal, so resolving them is truthful, while 1. could become any of infinitely many numbers, so showing anything is a guess. Keep the truth, drop the guess.

Re-parse the whole buffer per chunk. O(n²) over a stream sounds worse than it is at kilobyte scale, and the simplicity buys you a stateless, trivially-testable core. Optimize only if a profiler, not an instinct, tells you to.

Reuse JSON.parse for lexical decoding. The library is tolerant about grammar and strict about lexing, delegating escape and Unicode handling to the platform on fragments it has already proven valid.

These are the same disciplines that show up whenever you build the plumbing under an agent rather than the agent itself — the unglamorous layer decides whether the glamorous layer feels good. It is the same reason a durable agent memory layer is worth building carefully: the interesting behavior on top is only as reliable as the boring machinery underneath.

Where this fits

Streaming structured output is quietly becoming the default way LLM features present themselves, because the model economics finally allow it — tokens are cheap and fast enough that the interface, not the inference, is the bottleneck users feel. A chat bubble that streams text is table stakes; a form that fills itself in, a table whose rows appear as they are extracted, a plan whose steps materialize one at a time — that is the current frontier of LLM UX, and every one of those interfaces needs to parse a JSON object that is not finished yet.

The parser in this tutorial is small on purpose. It is a few hundred lines, it has no dependencies, and it does exactly one thing: turn any prefix of a JSON document into the best value available so far, without ever throwing. That is the kind of component you write once, cover with tests, and never think about again — while it quietly makes every streaming feature you build on top of it feel instantaneous.

Clone the complete project from the CrashBytes ByteSizedExamples repository, run npm test to watch all 49 cases pass without a key, and then run npm run demo to see a support-ticket object stream in field by field. Point it at your own model's structured output next, and give your interface back the streaming it was always supposed to have.