Building a Production Async Agent Queue in TypeScript with Bun and Mistral Work Mode: A 2026 Tutorial

The synchronous coding agent — your IDE pauses, the spinner spins, the model returns in twelve seconds — is becoming the second-class citizen of agentic AI. The first-class citizen is the long-running agent: a job that runs for ten or sixty or two hundred minutes, with its own working tree, its own tool calls totaling in the thousands, and a result that arrives by webhook hours later. Mistral shipped Le Chat Work mode last week with explicit support for this shape. OpenAI's background mode has been generally available since March. Anthropic's long-running orchestration primitives were announced at the same time as the 1M-context push. Every frontier provider now has the surface — and the bill for sync-only agent infrastructure is starting to look antique.

This tutorial is the version of "build a coding agent" I have been writing for the last two years, but with the right shape for 2026. We will build a small TypeScript queue on Bun that submits jobs to Mistral Work mode (with an OpenAI background-mode fallback), tracks them in a tiny SQLite database, and exposes a /jobs endpoint and a five-line dashboard so you can watch them work. By the end you will have a shape that scales from one agent on your laptop to hundreds running overnight in a Cloudflare Container, with the same code.

If you only have time to read the architecture section and steal the queue shape, that is fine. The whole point of the new paradigm is that the queue is the product; the model is interchangeable.

The complete, runnable companion project — the assembled queue with tests, an offline demo, and the real Mistral Work-mode and OpenAI background-mode adapters — lives at CrashBytes/ByteSizedExamples/async-agent-queue-mistral-bun. Clone it and run bun install && bun test && bun run demo to watch the queue drain jobs end to end with no API keys and no network, or build it from scratch as you read.

Why async is the new shape

Sub-30-second sync calls were the right primitive for chat. They are the wrong primitive for agentic coding. The reason is not that sync is broken — it is that the cost-of-doing-work has finally outgrown the time-budget of a single HTTP request.

A modern agentic coding job — port a service from Express to Hono, audit a repo for a class of bug, write a thousand-line migration script with a working test suite — needs minutes of model time, not seconds. It needs hundreds of tool calls, sometimes thousands. It needs a real working tree. The agent's average wall-clock is now bumping up against five minutes for nontrivial work on GPT-5.5 and Claude Opus 4.7, and the tail goes well past an hour. Pricing changed in tandem: agentic foundation-model rates roughly doubled in the April reset that I covered in the agentic-foundation-model reset analysis, and the new tiers explicitly subsidize background work over sync.

Here is the wall-clock distribution for a real coding-agent workload — a nightly PR-review queue at a 200-engineer fintech I consulted with in April, across two weeks of production traffic. The model is GPT-5.5 with Claude Opus 4.7 as a fallback for tasks that fail tool-budget. Nothing exotic.

Half of all jobs run for more than two minutes. Fifteen percent run for more than fifteen minutes. Eighty-six jobs in two weeks ran for over an hour. If your infrastructure is shaped for a request to land in under thirty seconds you are paying for one of three failure modes: you are timing out the tail (silent data loss), you are pinning HTTP workers (a load-balancing nightmare), or you are reissuing the work (paying for the same tokens twice). The teams that have figured out async are paying for none of these.

The async shape solves all three: submit-and-disconnect; the provider runs the agent for as long as it needs; you get the result by polling a status endpoint or by webhook. Your HTTP layer is free to do what HTTP layers are good at — short, fast, cheap requests.

The new pricing tiers reward this directly. Mistral Work mode and OpenAI background mode both bill at roughly a 50% discount versus sync for the same output tokens, with the trade that latency is "best-effort, scheduled". For overnight workloads the latency cost is zero — you do not care if the result arrives at 03:42 or 04:15. For ad-hoc workloads where someone is watching a spinner, you stay on sync. The right move is not all-or-nothing; it is a provider-side capability you select per job.

The async shape also unlocks scale you cannot reach on sync. A single synchronous agent saturates one TCP connection, one HTTP worker, one stream parser, one set of tool-call buffers. A queue with N async slots can have N concurrent agents, none of them owning a connection. The biggest fintech teams I work with are running 50–200 agents concurrently overnight, on infrastructure that would melt if it had to keep that many sockets open.

Architecture in one diagram

Here is the whole thing, ASCII-flavored. We will build each box.

   [HTTP API] -- POST /jobs --> [SQLite jobs table]
        |                              |
        v                              v
   [Submit worker] ---- create ---> [Provider]   (Mistral Work / OpenAI bg)
        |                              |
        |   poll loop OR webhook       |
        |<-----------------------------|
        v
   [Reconcile worker] -> [SQLite jobs] -> status: succeeded | failed | over-budget
        |
        v
   [Result handler] -> apply patch / open PR / notify Slack

Five components. Two of them are workers. One is a database. The other two are HTTP. The whole thing fits in around 700 lines of TypeScript and is single-binary deployable on Bun.

The two design rules are non-negotiable.

Rule 1: the queue owns the truth. The provider's view of the job is authoritative for the agent's state — what tools it called, what tokens it spent, what artifacts it produced. The queue's view is authoritative for your state — whether you have applied the result, whether the job is billed back to a budget, whether the customer has been notified. Treat these as separate state machines that occasionally exchange messages, and you will sleep at night.

Rule 2: provider-agnostic at the seam, provider-specific in the leaves. We will define one AsyncAgentProvider interface and implement it twice (Mistral Work, OpenAI background). The leaf adapters know about provider-specific tool-call formats, status enums, and webhook signatures. The queue knows nothing about any of that.

Prerequisites

You need:

• Bun 1.2 or later. The bun runtime, not just the package manager.
• TypeScript 5.5+, but you will not need to configure it — Bun does that.
• A Mistral API key with Le Chat Work mode access. Sign up at console.mistral.ai. Work mode requires a Pro or Team plan as of May 2026.
• An OpenAI API key with background-mode access. Background mode is on the GPT-5.5 standard tier; you do not need a separate entitlement.
• A way to receive webhooks. For the tutorial we will use a Cloudflare tunnel; the patterns work identically on ngrok, Tailscale Funnel, or a real public hostname.
• About 90 minutes of reading time and 4 hours of build time if you want to ship the whole thing.

The tutorial's repo lives in your local working directory. There is no external repo to clone — this is a from-scratch build.

mkdir async-agent-queue && cd async-agent-queue
bun init -y
bun add hono @anthropic-ai/sdk openai @mistralai/mistralai zod
bun add -d @types/bun

The hono import is for the HTTP layer; we want something fast and tiny that runs identically on Bun, Workers, and Node. The three SDK packages are for the provider adapters; we install all three even though we only wire up Mistral and OpenAI in this tutorial — the Anthropic SDK is the easiest to stub for the third-leg case.

Step 1 — The job state machine

Before we touch a model, we define the job. Two pieces: the schema, and the allowed transitions.

// src/job.ts
import { z } from 'zod'

export const JobStatus = z.enum([
  'queued', // accepted by us, not yet sent to provider
  'submitted', // sent to provider, awaiting first signal
  'running', // provider has started work
  'succeeded', // provider returned a result we accepted
  'failed', // provider returned an error or we rejected the result
  'over_budget', // we killed it because it crossed a cost ceiling
  'cancelled', // user or operator killed it
])
export type JobStatus = z.infer<typeof JobStatus>

export const Job = z.object({
  id: z.string(), // our id, ULID
  providerJobId: z.string().nullable(), // the provider's id, set on submit
  provider: z.enum(['mistral-work', 'openai-bg']),
  model: z.string(),
  prompt: z.string(),
  toolset: z.array(z.string()),
  status: JobStatus,
  budgetUsd: z.number(), // hard ceiling
  spentUsd: z.number(), // last-known cost
  createdAt: z.number(),
  startedAt: z.number().nullable(),
  finishedAt: z.number().nullable(),
  result: z.any().nullable(),
  error: z.string().nullable(),
})
export type Job = z.infer<typeof Job>

// Allowed transitions. Anything else is a bug.
export const transitions: Record<JobStatus, JobStatus[]> = {
  queued: ['submitted', 'cancelled', 'failed'],
  submitted: ['running', 'failed', 'cancelled'],
  running: ['succeeded', 'failed', 'over_budget', 'cancelled'],
  succeeded: [],
  failed: [],
  over_budget: [],
  cancelled: [],
}

export function canTransition(from: JobStatus, to: JobStatus) {
  return transitions[from].includes(to)
}

The canTransition helper is the single chokepoint for state changes. Every mutation goes through it. Without this, a webhook arriving after a poll result will silently turn a succeeded job back into running, and you will spend a Saturday afternoon explaining to the team why the agents keep re-running their own work.

Step 2 — The SQLite store

Bun ships with a SQLite driver as fast as anything you can buy. We use it.

// src/store.ts
import { Database } from 'bun:sqlite'
import { Job } from './job'

const db = new Database('agent-queue.sqlite')
db.exec(`
  CREATE TABLE IF NOT EXISTS jobs (
    id TEXT PRIMARY KEY,
    provider_job_id TEXT,
    provider TEXT NOT NULL,
    model TEXT NOT NULL,
    prompt TEXT NOT NULL,
    toolset TEXT NOT NULL,
    status TEXT NOT NULL,
    budget_usd REAL NOT NULL,
    spent_usd REAL NOT NULL DEFAULT 0,
    created_at INTEGER NOT NULL,
    started_at INTEGER,
    finished_at INTEGER,
    result TEXT,
    error TEXT
  );
  CREATE INDEX IF NOT EXISTS jobs_status_idx ON jobs(status);
  CREATE INDEX IF NOT EXISTS jobs_provider_job_idx ON jobs(provider_job_id);
`)

export const store = {
  insert(j: Job) {
    db.run(
      `INSERT INTO jobs (id, provider_job_id, provider, model, prompt, toolset,
                         status, budget_usd, spent_usd, created_at, started_at,
                         finished_at, result, error)
       VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
      [
        j.id,
        j.providerJobId,
        j.provider,
        j.model,
        j.prompt,
        JSON.stringify(j.toolset),
        j.status,
        j.budgetUsd,
        j.spentUsd,
        j.createdAt,
        j.startedAt,
        j.finishedAt,
        j.result ? JSON.stringify(j.result) : null,
        j.error,
      ]
    )
  },
  get(id: string): Job | null {
    const row: any = db.query('SELECT * FROM jobs WHERE id = ?').get(id)
    return row ? rowToJob(row) : null
  },
  byProviderId(pid: string): Job | null {
    const row: any = db
      .query('SELECT * FROM jobs WHERE provider_job_id = ?')
      .get(pid)
    return row ? rowToJob(row) : null
  },
  inFlight(): Job[] {
    return db
      .query(`SELECT * FROM jobs WHERE status IN ('submitted','running')`)
      .all()
      .map(rowToJob as any)
  },
  update(j: Job) {
    db.run(
      `UPDATE jobs SET provider_job_id = ?, status = ?, spent_usd = ?,
                       started_at = ?, finished_at = ?, result = ?, error = ?
       WHERE id = ?`,
      [
        j.providerJobId,
        j.status,
        j.spentUsd,
        j.startedAt,
        j.finishedAt,
        j.result ? JSON.stringify(j.result) : null,
        j.error,
        j.id,
      ]
    )
  },
}

function rowToJob(row: any): Job {
  return Job.parse({
    id: row.id,
    providerJobId: row.provider_job_id,
    provider: row.provider,
    model: row.model,
    prompt: row.prompt,
    toolset: JSON.parse(row.toolset),
    status: row.status,
    budgetUsd: row.budget_usd,
    spentUsd: row.spent_usd,
    createdAt: row.created_at,
    startedAt: row.started_at,
    finishedAt: row.finished_at,
    result: row.result ? JSON.parse(row.result) : null,
    error: row.error,
  })
}

Two production hooks worth flagging. First, every read goes through Job.parse, which means a corrupted row throws at read time, not at use time. You will thank yourself the first time a schema migration leaves a column half-populated. Second, the inFlight() query is the input to the reconcile worker — it is the entire reason the index on status exists. Skip the index and you will lock the table when the queue grows past a few thousand jobs.

Step 3 — The provider abstraction

Two adapters; one interface. The interface has exactly four methods, because that is all the queue needs.

// src/provider.ts
export interface AsyncAgentProvider {
  name: 'mistral-work' | 'openai-bg'

  /** Submit a job. Returns the provider's id. */
  submit(args: SubmitArgs): Promise<{ providerJobId: string }>

  /** Look up the current state of the job. */
  poll(providerJobId: string): Promise<ProviderState>

  /** Cancel the job. Best-effort; some providers ignore this. */
  cancel(providerJobId: string): Promise<void>

  /** Verify a webhook signature. Returns the matching providerJobId or null. */
  verifyWebhook(
    headers: Headers,
    body: string
  ): { providerJobId: string } | null
}

export type SubmitArgs = {
  model: string
  prompt: string
  toolset: string[]
  webhookUrl?: string
  metadata?: Record<string, string>
}

export type ProviderState =
  | { kind: 'submitted' }
  | { kind: 'running'; spentUsd: number }
  | { kind: 'succeeded'; spentUsd: number; result: unknown }
  | { kind: 'failed'; spentUsd: number; error: string }

The ProviderState is a closed sum type. Adding a new state means changing a type in one place and getting compile errors at every call site — exactly the experience you want. The metadata field on submit is what we use to round-trip our own job id; both providers accept arbitrary metadata and echo it back on webhooks, which lets us avoid a separate id-mapping table.

The Mistral Work mode adapter

Mistral's Work mode API exposes async sessions. The shape is "create a session, get an id, poll or webhook for state, fetch artifacts". The SDK hides most of this; we still want to be careful about the cost field because Work mode bills differently than sync.

// src/providers/mistral.ts
import { Mistral } from '@mistralai/mistralai'
import type { AsyncAgentProvider } from '../provider'

const m = new Mistral({ apiKey: process.env.MISTRAL_API_KEY! })

export const mistralWork: AsyncAgentProvider = {
  name: 'mistral-work',

  async submit({ model, prompt, toolset, webhookUrl, metadata }) {
    const session = await m.workSessions.create({
      model,
      messages: [{ role: 'user', content: prompt }],
      tools: toolset.map(t => ({ type: 'function', function: { name: t } })),
      mode: 'async',
      webhook: webhookUrl ? { url: webhookUrl, events: ['*'] } : undefined,
      metadata,
    })
    return { providerJobId: session.id }
  },

  async poll(id) {
    const s = await m.workSessions.retrieve(id)
    const spent = s.usage?.totalCostUsd ?? 0
    if (s.status === 'pending') return { kind: 'submitted' }
    if (s.status === 'running') return { kind: 'running', spentUsd: spent }
    if (s.status === 'completed') {
      const out = await m.workSessions.artifacts(id)
      return { kind: 'succeeded', spentUsd: spent, result: out }
    }
    return {
      kind: 'failed',
      spentUsd: spent,
      error: s.failureReason ?? 'unknown',
    }
  },

  async cancel(id) {
    await m.workSessions.cancel(id).catch(() => {}) // best-effort
  },

  verifyWebhook(headers, body) {
    const sig = headers.get('mistral-webhook-signature')
    if (!sig || !verifyHmac(sig, body, process.env.MISTRAL_WEBHOOK_SECRET!)) {
      return null
    }
    const ev = JSON.parse(body)
    return { providerJobId: ev.session_id }
  },
}

function verifyHmac(sig: string, body: string, secret: string): boolean {
  const h = new Bun.CryptoHasher('sha256', secret)
  h.update(body)
  return h.digest('hex') === sig
}

The webhook signature check is not optional. Mistral, OpenAI, and Anthropic all sign webhook payloads with HMAC-SHA256; if you accept an unsigned webhook you have shipped a "trigger any state transition" RCE on your queue. Bun.CryptoHasher does the right thing and is constant-time on recent Bun versions; if you are on Node, use crypto.timingSafeEqual.

The OpenAI background-mode adapter

OpenAI's background mode lives on the Responses API with background: true. The shape is the same — submit, poll, webhook — with slightly different field names. The capability flag we want to know about: background mode does not currently support tool-call streaming, so the result you get back is the final-state artifact, not an event stream.

// src/providers/openai.ts
import OpenAI from 'openai'
import type { AsyncAgentProvider } from '../provider'

const o = new OpenAI({ apiKey: process.env.OPENAI_API_KEY! })

export const openaiBg: AsyncAgentProvider = {
  name: 'openai-bg',

  async submit({ model, prompt, toolset, webhookUrl, metadata }) {
    const r = await o.responses.create({
      model,
      input: prompt,
      tools: toolset.map(t => ({ type: 'function', name: t, parameters: {} })),
      background: true,
      webhook_url: webhookUrl,
      metadata,
    } as any)
    return { providerJobId: r.id }
  },

  async poll(id) {
    const r: any = await o.responses.retrieve(id)
    const spent = r.usage?.total_cost_usd ?? 0
    if (r.status === 'queued') return { kind: 'submitted' }
    if (r.status === 'in_progress') return { kind: 'running', spentUsd: spent }
    if (r.status === 'completed') {
      return { kind: 'succeeded', spentUsd: spent, result: r.output }
    }
    return {
      kind: 'failed',
      spentUsd: spent,
      error: r.error?.message ?? 'unknown',
    }
  },

  async cancel(id) {
    await (o as any).responses.cancel(id).catch(() => {})
  },

  verifyWebhook(headers, body) {
    const sig = headers.get('openai-signature')
    if (!sig || !verifyHmac(sig, body, process.env.OPENAI_WEBHOOK_SECRET!)) {
      return null
    }
    const ev = JSON.parse(body)
    return { providerJobId: ev.data.response_id }
  },
}

function verifyHmac(sig: string, body: string, secret: string): boolean {
  const h = new Bun.CryptoHasher('sha256', secret)
  h.update(body)
  return h.digest('hex') === sig
}

The thing I want you to notice: from the queue's perspective, these are the same provider. The only thing that differs is where you put the API keys.

Step 4 — The submit endpoint

We are ready to accept jobs.

// src/server.ts
import { Hono } from 'hono'
import { ulid } from 'ulid'
import { store } from './store'
import { mistralWork } from './providers/mistral'
import { openaiBg } from './providers/openai'
import type { Job } from './job'

const app = new Hono()
const providers = { 'mistral-work': mistralWork, 'openai-bg': openaiBg }

const WEBHOOK_BASE = process.env.WEBHOOK_BASE ?? 'https://localhost:8787'

app.post('/jobs', async c => {
  const body = await c.req.json()
  const provider = providers[body.provider as keyof typeof providers]
  if (!provider) return c.json({ error: 'unknown provider' }, 400)

  const id = ulid()
  const job: Job = {
    id,
    providerJobId: null,
    provider: provider.name,
    model: body.model,
    prompt: body.prompt,
    toolset: body.toolset ?? [],
    status: 'queued',
    budgetUsd: body.budgetUsd ?? 5,
    spentUsd: 0,
    createdAt: Date.now(),
    startedAt: null,
    finishedAt: null,
    result: null,
    error: null,
  }
  store.insert(job)

  // Submit to provider out-of-band so the HTTP response stays fast.
  queueMicrotask(async () => {
    try {
      const { providerJobId } = await provider.submit({
        model: job.model,
        prompt: job.prompt,
        toolset: job.toolset,
        webhookUrl: `${WEBHOOK_BASE}/webhooks/${provider.name}`,
        metadata: { queue_job_id: id },
      })
      store.update({ ...job, providerJobId, status: 'submitted' })
    } catch (e: any) {
      store.update({ ...job, status: 'failed', error: e.message })
    }
  })

  return c.json({ id, status: 'queued' }, 202)
})

app.get('/jobs/:id', c => {
  const job = store.get(c.req.param('id'))
  return job ? c.json(job) : c.json({ error: 'not found' }, 404)
})

export default app

Two patterns earn their place. First, queueMicrotask lets the submit happen out-of-band so the HTTP response is sub-millisecond — the user gets their id immediately and never waits on the provider's submission latency. Second, every job gets a webhookUrl. Even if you intend to use polling, having the webhook wired means you fall back to push without re-deploying when poll-rate becomes a cost concern (it always becomes a cost concern).

Step 5 — The reconcile worker

Two ways to know a job's state changed: poll the provider, or receive a webhook. We do both, because the webhook can fail (bad TLS day, queue saturated, public hostname rotated) and polling is the safety net.

// src/reconcile.ts
import { store } from './store'
import { mistralWork } from './providers/mistral'
import { openaiBg } from './providers/openai'
import { canTransition, type Job, type JobStatus } from './job'

const providers = { 'mistral-work': mistralWork, 'openai-bg': openaiBg }
const POLL_INTERVAL_MS = 30_000

export async function reconcileOnce() {
  for (const job of store.inFlight()) {
    if (!job.providerJobId) continue // still queued
    const provider = providers[job.provider]
    try {
      const state = await provider.poll(job.providerJobId)
      applyState(job, state)
    } catch (e: any) {
      console.warn(`poll failed for ${job.id}: ${e.message}`)
    }
  }
}

export function applyState(
  job: Job,
  state:
    | { kind: 'submitted' }
    | { kind: 'running'; spentUsd: number }
    | { kind: 'succeeded'; spentUsd: number; result: unknown }
    | { kind: 'failed'; spentUsd: number; error: string }
) {
  const next: Partial<Job> = {}

  if (state.kind === 'submitted') return
  if (state.kind === 'running') {
    next.status = 'running'
    next.spentUsd = state.spentUsd
    next.startedAt = job.startedAt ?? Date.now()
    if (state.spentUsd > job.budgetUsd) {
      next.status = 'over_budget'
      next.finishedAt = Date.now()
      providers[job.provider].cancel(job.providerJobId!).catch(() => {})
    }
  } else if (state.kind === 'succeeded') {
    next.status = 'succeeded'
    next.spentUsd = state.spentUsd
    next.result = state.result
    next.finishedAt = Date.now()
  } else {
    next.status = 'failed'
    next.spentUsd = state.spentUsd
    next.error = state.error
    next.finishedAt = Date.now()
  }

  if (next.status && !canTransition(job.status, next.status as JobStatus)) {
    return // no-op; either already past this state or a bug we want to ignore
  }
  store.update({ ...job, ...next } as Job)
}

// Run forever
setInterval(reconcileOnce, POLL_INTERVAL_MS)

The applyState function is doing four jobs at once: it is the state transition guard, the cost-budget guard, the cancel-on-overbudget action, and the persist. That is intentional. Splitting it up reads cleaner but opens windows where a poll arrives, the budget check passes, the webhook arrives and reduces the budget, and your queue submits the cancel after the job already finished. Atomic in one place beats clean across three.

Step 6 — The webhook handler

Webhooks are the same code path as poll, just with a different trigger. This is the whole reason we factored applyState into its own function.

// src/server.ts (continued)
import { mistralWork } from './providers/mistral'
import { openaiBg } from './providers/openai'
import { applyState } from './reconcile'

app.post('/webhooks/:provider', async c => {
  const name = c.req.param('provider') as 'mistral-work' | 'openai-bg'
  const provider = providers[name]
  if (!provider) return c.json({ error: 'unknown provider' }, 400)

  const body = await c.req.text() // raw body for signature
  const verified = provider.verifyWebhook(c.req.raw.headers, body)
  if (!verified) return c.json({ error: 'bad signature' }, 401)

  const job = store.byProviderId(verified.providerJobId)
  if (!job) return c.json({ ok: true }) // not ours; ignore quietly

  // Translate the webhook payload into the same ProviderState shape we use
  // for polling, then reuse applyState. In practice both providers send a
  // payload that closely mirrors their poll responses.
  const state = await provider.poll(verified.providerJobId)
  applyState(job, state)
  return c.json({ ok: true })
})

The trick that earns its keep: instead of parsing the webhook payload to extract the new state, we let it be a "ping" and re-poll the provider for the authoritative state. This costs one extra API call per webhook — trivial — and saves us from maintaining two parallel parsers.

Step 7 — The cost-budget chart

The whole reason async makes financial sense is that you can let a job run for an hour without paying for an idle HTTP worker. Here is what that looked like for the same fintech workload, comparing sync vs async over two weeks.

Roughly 38% reduction in spend at the same workload. A third of that is the provider-side discount on background tier; the rest is two effects you do not get on sync — fewer retries (since the queue handles retries directly instead of HTTP-level retry storms) and fewer idle-worker minutes (since HTTP workers are not pinned through five-minute calls). At roughly half a million dollars a year on agentic API spend, this is a real number.

Step 8 — Per-job budget enforcement

We saw the budget hook in applyState. Here is the operator-side. Every job carries a budgetUsd. When spentUsd > budgetUsd, the queue cancels the provider job and marks it over_budget. The cancel is best-effort, so the actual cost can drift over the ceiling by whatever inflight tokens are already committed — call it a 5–10% overshoot in practice.

The pattern that matters: budgets are per-job, set by the caller, and sticky. They are not a single global ceiling. A nightly PR-review job might have a $0.20 budget; a cross-repo refactor job might have a $40 budget. Enforcing one number across both means the small jobs cannibalize the big ones.

// In the API: budget defaults are conservative.
const DEFAULT_BUDGETS: Record<string, number> = {
  'pr-review': 0.5,
  refactor: 25,
  audit: 10,
  migration: 40,
}

app.post('/jobs', async c => {
  const body = await c.req.json()
  const budget = body.budgetUsd ?? DEFAULT_BUDGETS[body.kind] ?? 5
  // ...rest of the handler
})

Now the operations question — what do you do with an over_budget job? Two choices. The conservative one: notify the requester, store the partial result, do nothing. The aggressive one: queue a continuation with a fresh budget, scoped to what is left to do. The aggressive one is appealing and almost always wrong unless your task can be cleanly chunked. For coding agents in particular, a refactor that hit budget mid-flight has left the working tree in an unknown state; trying to resume it usually creates worse problems than asking a human to look.

Step 9 — Wire to a real workload

The simplest meaningful workload: a nightly PR-review queue. You list open PRs, submit one async job per PR with a pr-review toolset, and pick up results in the morning.

// scripts/queue-prs.ts
import { listOpenPRs } from './github'

const reviews = await listOpenPRs({ org: 'crashbytes' })

for (const pr of reviews) {
  await fetch('http://localhost:8787/jobs', {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({
      provider: 'mistral-work',
      model: 'mistral-large-latest',
      prompt: `Review PR ${pr.url}. Use the read_file and read_diff tools as
needed. Return a structured review with severity-rated findings.`,
      toolset: ['read_file', 'read_diff', 'comment'],
      kind: 'pr-review',
    }),
  })
}
console.log(`queued ${reviews.length} PR reviews`)

Run this at 11pm. By 7am you have a queue of completed reviews. The whole point of the async shape is that this loop is dumb — there is no wait, no backpressure, no fancy concurrency. The queue handles it. You write a tiny result handler that posts the structured findings back to GitHub and you are done.

The companion approach for sync coding agents — where you migrate provider-by-provider — is in my migration tutorial for moving a coding agent from GPT-5 to DeepSeek V4. The async layer described here sits on top of that — same provider abstraction, different invocation model.

Step 10 — Eval and observability

You cannot ship this without two things. First, an eval harness, because you are going to be silently degrading a long-running agent's quality every time you change a prompt and you will never see it on a single sample. The shape I use for these is the one I wrote up in the multi-model evaluation harness tutorial — you can plug the async queue into it as a runner.

Second, observability. The minimum I would not ship without:

• Per-job timeline. Submit, first-running, last-poll, finished. A five-row sparkline per job tells you instantly whether a job stalled, ran hot, or finished cleanly.
• Provider error rates. Mistral Work and OpenAI background fail differently — Mistral tends to fail with tool_budget_exceeded, OpenAI with tool_loop. Counting these separately catches a bad upstream day before it becomes a customer-visible incident.
• Spent-versus-budget distribution. Plot this weekly. You want most jobs landing well below budget. A bimodal distribution means your budgets are wrong.

Here is the spent-vs-budget shape from the same fintech workload, after budgets had been tuned for two weeks.

Right-skewed and tight. About 0.8% of jobs went over budget — the queue caught them, the operator got a Slack message, the requester got a truncated result with a note explaining why. That is a healthy distribution. If yours is left-skewed or flat you are over-budgeting and paying for it.

Production gotchas

A few things that bit me, in case they save you a Saturday.

• Webhook idempotency. Both Mistral and OpenAI will sometimes deliver the same webhook twice. Make sure applyState is idempotent — the state machine guard helps, but you also want to be careful that any side effects (Slack notifications, GitHub posts) are gated on a state transition, not on receipt of a webhook.
• Clock drift. startedAt and finishedAt are wall-clock from the queue's machine. Provider-reported timestamps drift by 5–30 seconds. Pick one canonical source for SLA math; mixing them produces unreproducible bugs.
• Cancel is best-effort. If you cancel a job at minute 3, the provider may still bill for the next 30 seconds of in-flight tokens. This is fine; budget-overshoot tolerance is the right way to think about it. Reserve roughly 10% of headroom in your budget math.
• Long-running tool calls. A tool call that takes 90 seconds inside an agent counts toward the agent's wall-clock budget but not toward the provider's per-tool-call timeout. Test your tools at the long end.
• Webhook hostname rotation. If your tunnel hostname changes, jobs in flight will silently fail to deliver their webhooks and you will rely on poll. Build in an alarm for "no webhook in last hour despite in-flight jobs"; this catches the failure inside an hour instead of inside a workday.

What async unlocks next

Once the queue is in place, two things become tractable that were not before. First, portfolio management: an engineering org running a hundred async agents overnight is doing something qualitatively different from one running a single sync agent — the conversation shifts from "can the agent do X" to "which agents are worth running for which workloads". I expect this is going to be the operating model for most mid-and-large engineering organizations by Q4 2027 — see my prediction on async agentic-coding spend overtaking sync.

Second, safety in time: a long-running agent that you can pause, inspect, and resume is a fundamentally safer surface than one that runs to completion in twelve seconds with no observable middle state. The question of how teams will actually live with always-running cloud agents is the subject of today's short story. And the financial backdrop — Google's $40B commitment to Anthropic at a $350B mark — sits behind all of this; that money is buying the compute this paradigm runs on, as I dig into in today's analysis of the Google–Anthropic deal.

Ship the queue first. Worry about the philosophical questions next week.

Further reading

• The coding-agent migration tutorial — the sync-side companion to this piece, for moving a single agent between providers.
• The multi-model evaluation harness tutorial — the eval shape that plugs into the queue described here.
• The agent-governance gap analysis on Microsoft Agent 365 — context on why enterprise teams are formalizing the control plane around exactly the kind of async workload this tutorial assumes.