Skip to main content
Crashbytes logoCrashbytes
HomeArticlesByte Sized ExamplesOpen SourceServicesAboutContact
Browse Articles
HomeArticlesByte Sized ExamplesOpen SourceServicesAboutContact
Network
Theme
Browse Articles
Crashbytes logoCrashbytes

Expert insights on web development, technology trends, and programming best practices. Learn from real-world experiences and cutting-edge techniques that help you build better software.

Follow Us

Our Sites

  • ๐Ÿ”ฎ Predictions
  • ๐Ÿ“ฐ Breaking News
  • ๐ŸŽจ AI Art
  • ๐Ÿ“– Short Stories
  • View All โ†’
  • Products โ†’

Sitemap

  • Home
  • All Articles
  • Open Source
  • Services
  • About Us
  • Contact
  • Donate Compute

Popular Topics

  • Serverless
  • Cloud Architecture
  • DevOps
  • Kubernetes
  • Platform Engineering

Resources

  • Privacy Policy
  • Terms of Service
  • Sitemap
  • RSS Feed
  • PGP Key

Stay Updated

Get the latest articles, tutorials, and insights delivered to your inbox. Join our community of developers and never miss an update.

ยฉ 2021-2026 Crashbytesยฎ by Blackhole Software, LLC. All rights reserved.
| Reg. U.S. Pat. & Tm. Off.

Made for the developer community

  1. Home
  2. /
  3. Articles
  4. /
  5. Instrument an MCP Tool-Use Agent with OpenTelemetry Tracing in TypeScript
TutorialJune 29, 202624 min readโ€ข By Michael Eakins

Instrument an MCP Tool-Use Agent with OpenTelemetry Tracing in TypeScript

A hands-on TypeScript tutorial for making an autonomous, tool-using AI agent observable. You build a small, dependency-light agent loop and wrap it in OpenTelemetry traces โ€” a root span per invocation, child spans for every model call and every MCP tool call, using the gen_ai.* and MCP semantic conventions โ€” then prove the span tree with deterministic, in-memory tests. Runs offline with zero API keys.

Quick Takeaways

What you'll learn in this article

24 min read
Intermediate
  • 1

    Companion code on GitHub โ€” the full, tested otel-mcp-agent-tracing-typescript project from this tutorial.

  • 2

    The agent-observability turn โ€” Gemini Spark and OpenTelemetry's MCP spans โ€” the market context for why agent tracing landed all at once this week.

  • 3

    Prediction: OpenTelemetry's GenAI and MCP conventions reach Stable and ship default-on by end of 2027 โ€” my dated, falsifiable claim on where this standardizes.

  • 4

    Build a parallel subagent orchestrator in TypeScript โ€” the same root-span discipline applied to a fleet of concurrent agents.

  • 5

    The Overnight Run โ€” a short story on the difference between watching an agent and controlling it.

Keep reading for detailed implementation, code examples, and real-world results

The week I wrote this, Google shipped a personal agent meant to run twenty-four hours a day and take actions on your behalf, checking in only before the big ones. That is the destination the whole industry has been driving toward for two years, and arriving at it quietly changes the engineering problem. An agent you supervise turn by turn does not really need instrumentation, because you are the instrumentation โ€” you see every step because you approve every step. An agent that works while you sleep is a different animal. It fans a single instruction out into a dozen model calls and tool invocations, most of which you will never witness, and the only way to know what it did is to have decided, in advance, to record it.

That recording is a trace, and this tutorial builds one. We will write a small, honest agent loop โ€” a model that thinks, a registry of tools it can call, and a controller that runs the loop until the model produces a final answer โ€” and then we will make the whole thing observable with OpenTelemetry. By the end you will have an agent that emits a clean span tree for every run: one root span for the invocation, a child span for each model call, a child span for each tool call, each carrying the conventional attributes that let any OpenTelemetry backend read them. And because the agent is deterministic and runs with no API keys, the tests can assert the exact shape of that tree without spending a cent or flaking on the network.

The companion code lives at CrashBytes/ByteSizedExamples/otel-mcp-agent-tracing-typescript. Clone it, run npm install && npm run demo, and watch the span tree print to your console while you follow along.

Why an agent needs a trace, not a log

Start with the failure that motivates everything else. A single user request to a tool-using agent does not produce one model call. It produces a planning step, a tool call, a follow-up model call to interpret the tool's result, maybe another tool call, and a final synthesis. Each of those is a place where things go wrong in a way that a log line cannot explain. The model can decide to call a tool that does not exist. A tool can return an error the model shrugs off and talks past. The agent can loop, calling the same tool with the same bad arguments until it exhausts its step budget and returns something confident and wrong.

None of those failures show up in the metrics most teams start with. Request counts and token totals tell you the agent ran and what it cost; they do not tell you that step four called the weather tool with an empty city and the model papered over the blank result. To see that, you need the run rendered as what it actually is: a tree.

How visible each layer of an agent run is by default (illustrative)

How visible each layer of an agent run is by default (illustrative)
layervisibility
Tool calls18
Model decisions62
Token + cost totals90
Final answer only98

A trace is the right shape because an agent run is a tree and a trace is a tree. The root is the invocation. Beneath it hang the model calls and tool calls, in the order they happened, each a span with its own start time, duration, status, and attributes. When a run misbehaves you do not scroll a wall of log lines hunting for correlation IDs; you open the trace and read the branch that turned red. This is the same move the industry made for microservices a decade ago, which is exactly where OpenTelemetry came from. Agents are the new distributed system, with the twist that the nondeterminism lives inside the nodes.

The same agent failure, two debugging surfaces

Debugging with logs

ShapeFlat stream you reconstruct by hand
CorrelationGrep for an id across lines
A bad tool callHidden unless you logged its args
A loopLooks like normal repeated lines

Debugging with traces

ShapeA tree that mirrors the run
CorrelationParent and child share a trace id
A bad tool callA red span naming the tool and args
A loopVisibly repeated sibling spans

The market has noticed. Agent-observability platforms โ€” Braintrust, LangSmith, Arize Phoenix, Helicone, Datadog's LLM product, and others โ€” now compete on reading these traces. The leverage move is to emit traces in a standard, portable format so you own the data and can point it at any of them. That is what OpenTelemetry's generative-AI semantic conventions give you, and it is what we will use. For the wider context on why this all landed at once, I wrote a companion analysis on the agent-observability turn and OpenTelemetry's new MCP spans.

The concepts you need: traces, spans, conventions

Three OpenTelemetry primitives carry the whole tutorial.

A span is one timed unit of work โ€” it has a name, a start and end, a status, and a bag of key/value attributes. A trace is a set of spans that share a trace id and link to each other through parent/child relationships, forming the tree. A tracer is the object you ask to create spans. That is the entire mental model; everything below is just deciding which spans to create and which attributes to set.

The "which attributes" question is where the semantic conventions come in. OpenTelemetry publishes a standard vocabulary so that a model span from your agent and a model span from someone else's agent describe themselves the same way. The generative-AI conventions define gen_ai.* keys โ€” gen_ai.operation.name, gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens, and so on. As of 2026 the tool layer is covered too: Model Context Protocol attributes like mcp.tool.name and mcp.method.name describe the tool call, and they enrich the existing tool-execution span rather than duplicating it. Following the conventions is what makes your traces legible to backends you have not even chosen yet.

The shape we are building toward

1 root, N children

Every agent run becomes one agent.invoke root span with a child chat span per model call and a child mcp.tool span per tool call, all sharing one trace id.

โ†‘ 3%span kinds: the agent root, the model call, the tool call

A quick word on what this tutorial is not. It is not a guide to a specific agent framework โ€” there is no LangChain or LlamaIndex here, on purpose, because the point is the instrumentation pattern, which you can drop into any loop. And it is not a model-quality tutorial; the "LLM" is a deterministic fake so the focus stays on the trace. Swap the fake for a real provider and the spans do not change.

Advertisement

Project setup

The project is an intermediate-level TypeScript package: ECMAScript modules, strict mode, a handful of source files, and a Vitest suite. Here is the quick start.

git clone https://github.com/CrashBytes/ByteSizedExamples.git
cd ByteSizedExamples/otel-mcp-agent-tracing-typescript
npm install
npm run demo     # runs the agent, prints the span tree to the console
npm test         # asserts the span tree shape with an in-memory exporter

The dependencies are the OpenTelemetry packages and nothing exotic. These are the versions the project pins, all current stable releases at the time of writing:

| Package | Version | Role | | ----------------------------------------- | ---------- | --------------------------------------- | | @opentelemetry/api | ^1.9.1 | Tracer, spans, context โ€” the surface | | @opentelemetry/sdk-trace-node | ^2.8.0 | The Node tracer provider | | @opentelemetry/sdk-trace-base | ^2.8.0 | Processors and exporters (incl. memory) | | @opentelemetry/resources | ^2.8.0 | Service-identity resource | | @opentelemetry/semantic-conventions | ^1.41.1 | The gen_ai.* attribute constants | | @opentelemetry/exporter-trace-otlp-http | ^0.219.0 | Export to a real backend over OTLP |

The file layout keeps each responsibility in its own module:

otel-mcp-agent-tracing-typescript/
โ”œโ”€โ”€ src/
โ”‚   โ”œโ”€โ”€ telemetry.ts     # SDK setup: provider, exporters, shutdown
โ”‚   โ”œโ”€โ”€ tracer.ts        # getTracer + the semantic-convention constants
โ”‚   โ”œโ”€โ”€ llm.ts           # LLM interface, FakeLLM, the traced model call
โ”‚   โ”œโ”€โ”€ mcp-tools.ts     # Tool interface, ToolRegistry, the traced tool call
โ”‚   โ”œโ”€โ”€ agent.ts         # the agent loop and its root span
โ”‚   โ”œโ”€โ”€ types.ts         # shared types
โ”‚   โ””โ”€โ”€ index.ts         # public exports
โ”œโ”€โ”€ examples/demo.ts     # runs the agent end to end
โ””โ”€โ”€ test/                # vitest: agent, tracer, mcp-tools

Setting up telemetry

Everything starts with turning the SDK on. telemetry.ts exposes a single function, setupTelemetry, that configures a tracer provider, wires an exporter, and hands back a small handle you use to flush and shut down. Crucially it supports three exporters: console for eyeballing the tree during development, otlp for shipping to a real backend, and memory for tests that need to read the spans back as data.

import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node'
import {
  SimpleSpanProcessor,
  ConsoleSpanExporter,
  InMemorySpanExporter,
  type ReadableSpan,
} from '@opentelemetry/sdk-trace-base'
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http'
import { resourceFromAttributes } from '@opentelemetry/resources'
import { ATTR_SERVICE_NAME } from '@opentelemetry/semantic-conventions'
import { trace } from '@opentelemetry/api'

export interface TelemetryHandle {
  exporter: 'console' | 'otlp' | 'memory'
  serviceName: string
  shutdown(): Promise<void>
  getFinishedSpans(): ReadableSpan[]
}

export function setupTelemetry(
  options: {
    serviceName?: string
    exporter?: 'console' | 'otlp' | 'memory'
  } = {}
): TelemetryHandle {
  const serviceName =
    options.serviceName ?? process.env.OTEL_SERVICE_NAME ?? 'mcp-agent'

  // Auto-select OTLP when an endpoint is configured, else honor the option.
  const kind =
    options.exporter ??
    (process.env.OTEL_EXPORTER_OTLP_ENDPOINT ? 'otlp' : 'console')

  const memory = kind === 'memory' ? new InMemorySpanExporter() : null
  const exporter =
    kind === 'otlp'
      ? new OTLPTraceExporter()
      : (memory ?? new ConsoleSpanExporter())

  const provider = new NodeTracerProvider({
    resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: serviceName }),
    spanProcessors: [new SimpleSpanProcessor(exporter)],
  })

  // The global tracer provider is process-wide and ignores re-registration,
  // so disable any prior one before registering this run's provider.
  trace.disable()
  provider.register()

  return {
    exporter: kind,
    serviceName,
    async shutdown() {
      await provider.shutdown()
    },
    getFinishedSpans() {
      return memory ? memory.getFinishedSpans() : []
    },
  }
}

A few decisions here earn their keep. The exporter selection is the only place environment variables touch the code: set OTEL_EXPORTER_OTLP_ENDPOINT and the agent ships traces to your collector with no code change, which is exactly the "flip a switch in production" property you want. The provider is built with the current constructor form โ€” a resource describing the service identity and spanProcessors injected directly โ€” because the older new Resource(...) constructor and addSpanProcessor(...) method are deprecated in the version 2 SDK. And the trace.disable() call before register() is not ceremony: the global tracer provider is a process-wide singleton that silently ignores a second registration, so without disabling first, a test that calls setupTelemetry twice would keep exporting into the first run's memory exporter. That one line is the difference between tests that pass and tests that lie.

We use a SimpleSpanProcessor for clarity โ€” it exports each span as it ends. In production you would swap in a BatchSpanProcessor to amortize network calls; it is a one-line change and the rest of the code is identical.

The tracer and the conventions

tracer.ts is small but it is where the standards discipline lives. It hands out the tracer, defines the attribute-key constants so the rest of the code never hard-codes a string, and provides one helper for recording errors correctly.

import {
  trace,
  SpanStatusCode,
  type Span,
  type Tracer,
} from '@opentelemetry/api'
import {
  ATTR_GEN_AI_OPERATION_NAME,
  ATTR_GEN_AI_SYSTEM,
  ATTR_GEN_AI_REQUEST_MODEL,
  ATTR_GEN_AI_USAGE_INPUT_TOKENS,
  ATTR_GEN_AI_USAGE_OUTPUT_TOKENS,
  ATTR_GEN_AI_TOOL_NAME,
} from '@opentelemetry/semantic-conventions/incubating'

const NAME = 'otel-mcp-agent-tracing-typescript'

export function getTracer(): Tracer {
  return trace.getTracer(NAME, '1.0.0')
}

// GenAI keys come from the real (incubating) conventions package.
export const GEN_AI = {
  OPERATION_NAME: ATTR_GEN_AI_OPERATION_NAME, // "gen_ai.operation.name"
  SYSTEM: ATTR_GEN_AI_SYSTEM, // "gen_ai.system"
  REQUEST_MODEL: ATTR_GEN_AI_REQUEST_MODEL, // "gen_ai.request.model"
  USAGE_INPUT_TOKENS: ATTR_GEN_AI_USAGE_INPUT_TOKENS,
  USAGE_OUTPUT_TOKENS: ATTR_GEN_AI_USAGE_OUTPUT_TOKENS,
  TOOL_NAME: ATTR_GEN_AI_TOOL_NAME, // "gen_ai.tool.name"
} as const

// MCP keys are not in the package yet, so we define them as literals.
export const MCP = {
  TOOL_NAME: 'mcp.tool.name',
  METHOD_NAME: 'mcp.method.name',
} as const

export const AGENT = { STEPS: 'agent.steps', TASK: 'agent.task' } as const

export function recordError(span: Span, err: unknown): void {
  const error = err instanceof Error ? err : new Error(String(err))
  span.recordException(error)
  span.setStatus({ code: SpanStatusCode.ERROR, message: error.message })
}

Two details are worth slowing down on. First, the gen_ai.* constants are imported from the package's /incubating entry point, not its root. In version 1.41 of the conventions package the generative-AI keys are still incubating, so they live behind that subpath; importing the real constants (rather than typing the strings) means your spans automatically track the spec as it stabilizes. Second, the MCP keys are local literals because they are newer than the package โ€” the convention exists, the constants are not published yet, and writing them as a single MCP object means there is exactly one place to delete when they ship upstream.

The recordError helper exists because recording an error on a span is two distinct acts that are easy to do by halves: attaching the exception as a span event, and setting the span's status to ERROR. Do only the first and the span looks green in your backend despite carrying an exception; do only the second and you lose the stack trace. The helper does both, every time.

Tracing the model call

Now the first real span. llm.ts defines the model interface, a deterministic fake that stands in for a real provider, and tracedChat โ€” the wrapper that turns a model call into a conventional gen_ai span.

import { SpanKind } from '@opentelemetry/api'
import { getTracer, GEN_AI, recordError } from './tracer.js'
import type { ChatMessage, LLMResponse } from './types.js'

export interface LLM {
  readonly model: string
  readonly system: string
  chat(messages: ChatMessage[]): Promise<LLMResponse>
}

export async function tracedChat(
  llm: LLM,
  messages: ChatMessage[]
): Promise<LLMResponse> {
  const tracer = getTracer()
  // Span name follows the convention: the operation plus the model.
  return tracer.startActiveSpan(
    `chat ${llm.model}`,
    { kind: SpanKind.CLIENT },
    async span => {
      span.setAttribute(GEN_AI.OPERATION_NAME, 'chat')
      span.setAttribute(GEN_AI.SYSTEM, llm.system)
      span.setAttribute(GEN_AI.REQUEST_MODEL, llm.model)
      try {
        const res = await llm.chat(messages)
        span.setAttribute(GEN_AI.USAGE_INPUT_TOKENS, res.usage.inputTokens)
        span.setAttribute(GEN_AI.USAGE_OUTPUT_TOKENS, res.usage.outputTokens)
        return res
      } catch (err) {
        recordError(span, err)
        throw err
      } finally {
        span.end()
      }
    }
  )
}

The mechanics here are the template for every traced operation in the codebase, so they are worth internalizing. startActiveSpan does two jobs at once: it creates the span and it makes that span the active span for the duration of the callback. That second job is what makes the tree assemble itself โ€” any span created inside the callback automatically becomes a child of this one, with no manual parent-passing. The attributes are set from the GEN_AI constants, never from raw strings. Token usage is recorded after the call returns, because you do not know it until then. And the try/catch/finally is load-bearing: errors are recorded on the span and rethrown so the caller still sees them, and span.end() runs in finally so a span never leaks open on an error path.

The FakeLLM that implements this interface is deliberately dumb and fully deterministic โ€” it routes on keywords ("weather" means call the weather tool, "docs" or "cite" means call the doc-search tool, otherwise produce a final answer). That determinism is a feature: it is what lets the tests assert an exact span tree. Replace FakeLLM with a class that calls a real provider and implements the same three-member interface, and not one line of the tracing changes. For the pattern of wrapping a real provider behind a swappable interface like this โ€” with timeouts, retries, and failover โ€” see the companion build of a resilient multi-provider LLM client in TypeScript.

Why a deterministic fake

0 API keys

A keyword-routed FakeLLM makes the agent run identically every time, so the test suite can assert the exact span tree instead of mocking a network and hoping.

โ†‘ 8%deterministic tests the fake makes possible
Advertisement

Tracing the MCP tool call

The tool layer is the part that used to go dark, so this is the span that matters most. mcp-tools.ts defines a tool, a registry that runs tools, and โ€” the whole point โ€” wraps each invocation in an mcp.tool span.

import { SpanKind } from '@opentelemetry/api'
import { getTracer, GEN_AI, MCP, recordError } from './tracer.js'

export interface Tool {
  name: string
  description: string
  execute(args: Record<string, unknown>): Promise<unknown>
}

export class ToolRegistry {
  private readonly tools = new Map<string, Tool>()

  register(tool: Tool): this {
    this.tools.set(tool.name, tool)
    return this
  }

  list(): Tool[] {
    return [...this.tools.values()]
  }

  has(name: string): boolean {
    return this.tools.has(name)
  }

  async call(name: string, args: Record<string, unknown>): Promise<unknown> {
    const tool = this.tools.get(name)
    if (!tool) throw new Error(`Unknown tool: ${name}`)

    const tracer = getTracer()
    return tracer.startActiveSpan(
      `mcp.tool/${name}`,
      { kind: SpanKind.CLIENT },
      async span => {
        span.setAttribute(MCP.TOOL_NAME, name)
        span.setAttribute(MCP.METHOD_NAME, 'tools/call')
        span.setAttribute(GEN_AI.TOOL_NAME, name)
        span.setAttribute(GEN_AI.OPERATION_NAME, 'execute_tool')
        try {
          const result = await tool.execute(args)
          span.setAttribute('mcp.tool.result', JSON.stringify(result))
          return result
        } catch (err) {
          recordError(span, err)
          throw err // surface the failure to the agent loop
        } finally {
          span.end()
        }
      }
    )
  }
}

The span name mcp.tool/<name> keeps every tool call grouped and greppable while still distinguishing which tool ran. The attributes do double duty: the mcp.* keys describe the call in Model Context Protocol terms (mcp.method.name is "tools/call", the MCP method that actually executes a tool), and the gen_ai.tool.name plus gen_ai.operation.name keys describe the same call in the generative-AI vocabulary, so backends that only understand one of the two conventions still get a useful span. The result is stringified onto the span so you can see what the tool returned โ€” which is the single most useful thing to have when the model misreads a tool result.

The error path is the reason this matters. One of the bundled tools, get_weather, throws when handed a blank city. When that happens the catch block calls recordError, the span gets ERROR status and a recorded exception, and the error is rethrown so the agent loop can decide what to do. In your backend that tool call shows up red, named, with the exception attached โ€” instead of vanishing into a model span that simply produced a worse answer.

What the instrumentation now covers

Model decisions traced100.0%
Tool calls traced100.0%
Errors recorded on the right span100.0%
Token usage captured100.0%

The agent loop

With both leaf spans in place, the loop is almost anticlimactic โ€” which is the sign the abstraction is right. agent.ts opens the root span and runs think-act-observe until the model stops asking for tools.

import { SpanKind } from '@opentelemetry/api'
import { getTracer, AGENT, recordError } from './tracer.js'
import { tracedChat, type LLM } from './llm.js'
import { ToolRegistry } from './mcp-tools.js'
import type { AgentResult, ChatMessage } from './types.js'

export class Agent {
  private readonly llm: LLM
  private readonly tools: ToolRegistry
  private readonly maxSteps: number

  constructor(opts: { llm: LLM; tools: ToolRegistry; maxSteps?: number }) {
    this.llm = opts.llm
    this.tools = opts.tools
    this.maxSteps = opts.maxSteps ?? 6
  }

  async run(task: string): Promise<AgentResult> {
    const tracer = getTracer()
    return tracer.startActiveSpan(
      'agent.invoke',
      { kind: SpanKind.INTERNAL },
      async root => {
        root.setAttribute(AGENT.TASK, task)
        const messages: ChatMessage[] = [{ role: 'user', content: task }]
        let steps = 0
        try {
          while (steps < this.maxSteps) {
            steps++
            const reply = await tracedChat(this.llm, messages) // child span
            if (reply.toolCall) {
              const result = await this.tools.call(
                reply.toolCall.name,
                reply.toolCall.args
              ) // child span, nested under the root
              messages.push({
                role: 'tool',
                content: JSON.stringify(result),
              })
              continue
            }
            root.setAttribute(AGENT.STEPS, steps)
            return { answer: reply.content, steps }
          }
          root.setAttribute(AGENT.STEPS, steps)
          return { answer: 'Step budget exhausted.', steps }
        } catch (err) {
          recordError(root, err)
          throw err
        } finally {
          root.end()
        }
      }
    )
  }
}

Because the model call and the tool call both use startActiveSpan and both run inside the root's startActiveSpan callback, the nesting happens for free: every chat span and every mcp.tool span lands as a child of agent.invoke, sharing its trace id. The maxSteps guard is the loop's safety belt โ€” it is what turns a runaway "call the same tool forever" loop into a bounded run that ends with an honest "step budget exhausted" instead of hanging. And agent.steps on the root span gives you an at-a-glance answer to "how much work did this take," which is often the first number you want when a run feels too slow or too expensive. When you graduate from one agent to many, this same root-span-per-unit discipline is what keeps a fleet legible; the parallel subagent orchestrator tutorial extends the idea to concurrent workers.

Cumulative traced operations across the demo run (3 steps)

Cumulative traced operations across the demo run (3 steps)
stepcalls
11
22
33

Running it

npm run demo builds an agent from the FakeLLM and the two bundled tools, runs it on a sample task, and prints the resulting spans to the console. With the task "What's the weather in Paris, and cite the docs?" the agent takes three steps โ€” a model call that asks for the weather tool, the weather tool call, a model call that asks for doc search, the doc-search tool call, and a final model call that answers โ€” and the console exporter prints each span as it ends.

{ name: 'chat fake-router-v1', kind: 2,
  attributes: { 'gen_ai.operation.name': 'chat', 'gen_ai.system': 'fake',
                'gen_ai.request.model': 'fake-router-v1',
                'gen_ai.usage.input_tokens': 14, 'gen_ai.usage.output_tokens': 9 } }
{ name: 'mcp.tool/get_weather', kind: 2,
  attributes: { 'mcp.tool.name': 'get_weather', 'mcp.method.name': 'tools/call',
                'gen_ai.tool.name': 'get_weather',
                'gen_ai.operation.name': 'execute_tool',
                'mcp.tool.result': '{"city":"Paris","summary":"partly cloudy, 20C"}' } }
{ name: 'mcp.tool/search_docs', kind: 2, attributes: { 'mcp.tool.name': 'search_docs', ... } }
{ name: 'agent.invoke', kind: 1,
  attributes: { 'agent.task': "What's the weather in Paris, and cite the docs?",
                'agent.steps': 3 } }

Final answer: The weather in Paris is partly cloudy at 20C. Per the docs:
OpenTelemetry models an agent run as a trace: one root span per invocation with
child spans for each model call and tool call.
Steps taken: 3

Notice the order. The leaf spans print first because they end first; the agent.invoke root prints last because it does not end until its children are done. In a real backend you would not read this as text โ€” you would see the tree rendered, root at the top, the two tool calls and three model calls nested beneath it, the whole run reconstructable at a glance. Point the agent at a collector by setting OTEL_EXPORTER_OTLP_ENDPOINT, and the same spans flow to Jaeger, Grafana Tempo, Datadog, or anything else that speaks OTLP.

Testing the span tree

Here is the payoff for all that determinism: you can test the shape of the observability itself. Because setupTelemetry({ exporter: 'memory' }) returns an in-memory exporter, a test can run the agent and then read the finished spans back as plain objects and assert against them.

import { describe, it, expect, afterEach } from 'vitest'
import {
  setupTelemetry,
  Agent,
  FakeLLM,
  createDefaultToolRegistry,
} from '../src/index.js'

describe('agent span tree', () => {
  let handle: ReturnType<typeof setupTelemetry>
  afterEach(async () => {
    await handle?.shutdown()
  })

  it('nests model and tool spans under one agent.invoke root', async () => {
    handle = setupTelemetry({ exporter: 'memory' })
    const agent = new Agent({
      llm: new FakeLLM(),
      tools: createDefaultToolRegistry(),
    })

    const result = await agent.run(
      "What's the weather in Paris, and cite the docs?"
    )
    expect(result.steps).toBe(3)

    const spans = handle.getFinishedSpans()
    const root = spans.find(s => s.name === 'agent.invoke')!
    const children = spans.filter(
      s => s.parentSpanContext?.spanId === root.spanContext().spanId
    )

    // Every child shares the root's trace id...
    for (const child of children) {
      expect(child.spanContext().traceId).toBe(root.spanContext().traceId)
    }
    // ...and the children are the model calls and the tool calls.
    expect(children.some(s => s.name.startsWith('chat '))).toBe(true)
    expect(children.some(s => s.name.startsWith('mcp.tool/'))).toBe(true)
  })
})

This is a genuinely different kind of test from the unit tests you are used to. It does not assert what the agent answered โ€” it asserts that the agent is observable: that the run produced a single root, that every span belongs to the same trace, and that the model and tool calls are nested where a backend expects them. A separate test reads the gen_ai.* attributes off the model span and checks the exact keys and values; another drives the get_weather tool with a blank city and asserts that the resulting span carries ERROR status and a recorded exception, and that call() rethrows. The full suite is eight tests across three files, and it runs in about a second with no network.

Test suite

8 passing

Span-tree nesting, shared trace id, gen_ai attribute keys and values, and the tool error path โ€” all asserted against an in-memory exporter, no API keys, about one second.

โ†‘ 3%test files: agent, tracer, mcp-tools

One Vitest configuration detail makes this reliable. The OpenTelemetry global tracer provider is process-wide, so two test files registering their own in-memory exporters can race. Pinning Vitest to a single fork (fileParallelism: false) makes each test own its own provider cleanly, which is a small price for tests that assert telemetry without flaking.

Production considerations

The tutorial code is honest but minimal. Three changes turn it into something you would run for real.

Mind what the spans capture. This is the big one. Prompts and tool arguments are exactly the sensitive data you do not want sprayed across a tracing backend โ€” they can contain user messages, secrets, and PII. The conventions anticipate this: capturing message content is opt-in, and you should keep it that way. Record the shape of a call by default (which model, which tool, how many tokens, success or failure) and gate the content (the actual prompt text, the raw arguments) behind an explicit, environment-controlled flag that is off in production unless someone deliberately turns it on for a debugging session.

Batch and sample. Swap the SimpleSpanProcessor for a BatchSpanProcessor so exports do not add latency to every span, and add a sampler if your volume is high โ€” though for agent traces, where each run is rare and expensive relative to a web request, many teams sample at one hundred percent because every run is worth keeping.

Treat instrumentation as default-on. The endgame is that you do not remember to add tracing; it is simply on, the way request tracing is on in a modern web framework. That is also where the standards are headed โ€” I lay out the case in my prediction that OpenTelemetry's GenAI and MCP conventions reach Stable and ship default-on in major agent frameworks by the end of 2027. There is a compliance dimension too: regulations that require reconstructing an AI-assisted decision after the fact are, functionally, asking for exactly this span tree.

A rollout path from console to production

Day 1

Console exporter

Wrap the agent loop, read the tree in your terminal, confirm the nesting is right.

Week 1

OTLP to a backend

Set OTEL_EXPORTER_OTLP_ENDPOINT, batch the processor, and view real traces in Jaeger, Tempo, or Datadog.

Month 1

Content gating + sampling

Keep prompt and argument content opt-in; decide a sampling rate; record only span shape by default.

Ongoing

Default-on

Bake the instrumentation into your agent harness so every run is observable without anyone remembering to add it.

The discipline pays a human dividend too. An agent that runs unattended and records everything it does is honest in a way an uninstrumented one cannot be โ€” though "honest" and "controllable" are not the same thing, a gap I explored in a short story about an engineer reading her agent's overnight trace, The Overnight Run. A trace tells you what happened. It is on you to decide, in advance, what the agent is allowed to do.

Conclusion

You now have the full pattern for making a tool-using agent observable: a tracer and a set of convention constants, a setupTelemetry switch that exports to your console in development and your collector in production, a traced model call and a traced tool call that follow the gen_ai.* and MCP conventions, an agent loop whose root span gathers it all into one tree, and tests that assert the shape of that tree without a single API key. The instrumentation is maybe a hundred lines; the leverage is that any agent loop you write from here can adopt it wholesale, and the traces it emits are portable to any backend you ever choose.

The autonomy is already here. The observability is the half that decides whether you can run it where it counts.

Related resources

  • Companion code on GitHub โ€” the full, tested otel-mcp-agent-tracing-typescript project from this tutorial.
  • The agent-observability turn โ€” Gemini Spark and OpenTelemetry's MCP spans โ€” the market context for why agent tracing landed all at once this week.
  • Prediction: OpenTelemetry's GenAI and MCP conventions reach Stable and ship default-on by end of 2027 โ€” my dated, falsifiable claim on where this standardizes.
  • Build a parallel subagent orchestrator in TypeScript โ€” the same root-span discipline applied to a fleet of concurrent agents.
  • The Overnight Run โ€” a short story on the difference between watching an agent and controlling it.

Signed by Michael Eakins

PGP key fingerprint ends in 08E8 8F19 ยท signed 2026-06-29

Verify โ†’.sig
Advertisement

Was this article helpful?

Your feedback helps us improve our content and create more valuable resources

We appreciate honest feedback - it helps us serve you better

Work with us

This analysis is what we do for clients

CrashBytes consults on enterprise AI strategy and implementation, builds custom web and mobile software, and places senior engineers on corp-to-corp engagements.

See Services

Enjoyed this? Get the next one.

Join developers getting CrashBytes articles, tutorials, and predictions in their inbox. No spam, unsubscribe anytime.

Related Topics

TutorialTypeScriptOpenTelemetryAI AgentsObservability
Back to Articles
โ† PreviousBuild a Verifiable Agent-Commit Provenance Trail in TypeScriptNext โ†’The End of the Seat: Why Agentic Coding Tools Are Killing Per-Seat Pricing

From across the CrashBytes network

More than the blog โ€” predictions, news, fiction, and AI art.

PredictionCustom AI Chips Reach Commodity Status by Q4 2027: Cloud Provider Competition Drives Democratization
NewsWeek In Review July 19-25, 2026 - The Week The Money Moved To The Metering Layer
Short StoryThe Answer Key
AI ArtThe Room That Remembers

Continue Your Learning Journey

Explore more articles related to Tutorial and expand your knowledge.

๐Ÿ“„Tutorial

Building Production Agentic AI Systems - Complete Tool Calling Implementation Tutorial

Step-by-step guide to building a production-ready agentic AI system with tool calling, error handling, rate limiting, and observability. Includes complete working code and GitHub repository with TypeScript implementation using Claude API.

22 min readRead more
๐Ÿ“„Tutorial

Build a Verifiable Agent-Commit Provenance Trail in TypeScript

A hands-on TypeScript tutorial for proving which agent, model, prompt, and supervisor produced a code changeset โ€” and detecting any later tampering. You build canonical changeset hashing, ed25519-signed attestations, and an append-only chained ledger you can verify offline, with zero runtime dependencies and zero API keys.

26 min readRead more
๐Ÿ“„Tutorial

Implementing RAG in a React Native App: On-Device and Cloud Retrieval

Build production Retrieval-Augmented Generation for React Native with one engine that runs both on-device and behind a Claude Opus 4.8 cloud service.

50 min readRead more
๐Ÿ“„Tutorial

Build a Pre-Deployment LLM Evaluation Pipeline in TypeScript

A hands-on TypeScript tutorial for a CI-integrated eval harness that gates LLM releases on capability, safety, and regression checks โ€” the discipline CAISI now requires from labs.

29 min readRead more