Suppose a user reports the wrong temperature for Jakarta. You open the logs and find 41 lines of model output, two tool executions, and one retry at 03:12. Which step handed the bad number to the model? A flat log cannot tell you, because an agent run is a tree, not a line: one agent loop, several model calls, each tool call hanging off the model call that requested it.
OpenTelemetry solved this problem for microservices years ago. Since 2024 the GenAI SIG has been building the same thing for LLM agents, and the result is a small, shared vocabulary you can adopt today: gen_ai.* attributes, a handful of well-known span names, and a rule for when to capture prompts.
Everything below was run on this machine on a two-turn agent loop against a local mock model. Package versions in the prerequisites are the exact ones from that run, and the span dumps quoted later are real output, not illustrations.
What the conventions actually pin down
The GenAI conventions define span operations for the agent layer, separate from the model layer. The ones you will use most:
| Operation | Span name | Kind | Wraps |
|---|---|---|---|
create_agent |
create_agent {gen_ai.agent.name} |
CLIENT | Building the agent: a graph, an SDK assistant, your own class |
invoke_agent |
invoke_agent {gen_ai.agent.name} |
CLIENT or INTERNAL | Calling an agent. CLIENT when it runs elsewhere, INTERNAL for your in-process loop |
invoke_workflow |
invoke_workflow {name} |
CLIENT | One workflow step or subgraph node |
chat |
chat {gen_ai.request.model} |
CLIENT | A single model call |
execute_tool |
execute_tool {gen_ai.tool.name} |
agent runtime | The agent actually running a function, retrieval, or shell command |
Required on almost every span: gen_ai.operation.name and gen_ai.provider.name. Required on model calls: gen_ai.request.model, plus gen_ai.usage.input_tokens and gen_ai.usage.output_tokens whenever the provider returns counts.
There is one attribute the spec deliberately refuses to invent for you: gen_ai.conversation.id. The conventions say instrumentations SHOULD NOT fall back to a fresh UUID or the trace id when no real conversation identifier exists, and that application developers may add the conversation id themselves with a span processor. That sentence is the reason Step 5 below exists, and it is also the difference between a pile of unrelated traces and a conversation you can read end to end.
Prerequisites
- Python 3.10 or newer (the OpenAI instrumentation declares
>=3.10) - The versions from this run:
opentelemetry-sdk==1.44.0,opentelemetry-instrumentation-openai-v2==2.4b0,openai==3.19.2,httpx==0.28.1 - No API key for this walkthrough. The script points the OpenAI client at a local mock, so you can follow along for free and get deterministic output.
Step 1: Install
python3 -m venv .venv
source .venv/bin/activate
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http \
opentelemetry-instrumentation-openai-v2 openai httpx
Two packages are pinned by the conventions, not by me. The OpenAI instrumentation ships as a pre-release (2.4b0), and its default output tracks an older convention version unless you opt in to the newer attributes. Pin both the instrumentation and the OpenAI client, because the instrumentation patches the HTTP layer the client uses and that layer changed: openai 3.x depends on httpx2, while the instrumentation imports httpx. That mismatch is Step 8's first gotcha.
Step 2: A deterministic model endpoint
Run this in a second terminal. It answers a weather question in two steps: first with a tool call, then with a final answer once it sees the tool result.
"""Canned OpenAI-compatible endpoint for local agent tracing experiments."""
import json
from http.server import BaseHTTPRequestHandler, HTTPServer
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
length = int(self.headers.get("Content-Length", 0))
body = json.loads(self.rfile.read(length) or b"{}")
already_ran_tool = any(m.get("role") == "tool" for m in body.get("messages", []))
if not already_ran_tool:
message = {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"city":"Jakarta"}',
},
}
],
}
finish = "tool_calls"
else:
message = {"role": "assistant", "content": "Jakarta is 31C and humid."}
finish = "stop"
payload = {
"id": "chatcmpl-mock",
"object": "chat.completion",
"created": 1789000000,
"model": body.get("model", "gpt-4o-mini"),
"choices": [{"index": 0, "message": message, "finish_reason": finish}],
"usage": {"prompt_tokens": 412, "completion_tokens": 96, "total_tokens": 508},
}
data = json.dumps(payload).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def log_message(self, *args):
pass
if __name__ == "__main__":
HTTPServer(("127.0.0.1", 8099), Handler).serve_forever()
python mock_llm.py
Against a real provider you delete base_url and set your key. Nothing else in Step 3 changes.
Step 3: Instrument the agent, not just the model call
Save this as agent_demo.py. Three things are worth reading closely: the span processor that stamps the conversation id, the invoke_agent span that becomes the parent of everything else, and the execute_tool span wrapped around the actual function call.
"""Two-turn agent loop with OpenTelemetry GenAI conventions."""
import json
import os
import uuid
from contextvars import ContextVar
from opentelemetry import trace
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import SpanProcessor, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from openai import OpenAI
CONVERSATION_ID: ContextVar[str | None] = ContextVar("conversation_id", default=None)
AGENT_NAME = "travel-concierge"
AGENT_ID = "travel-concierge-001"
MODEL = "gpt-4o-mini"
TOOLS = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}
]
class ConversationSpanProcessor(SpanProcessor):
"""Stamps gen_ai.conversation.id on every span opened during a run."""
def __init__(self, inner: SpanProcessor):
self._inner = inner
def on_start(self, span, parent_context=None):
conversation_id = CONVERSATION_ID.get()
if conversation_id is not None:
span.set_attribute("gen_ai.conversation.id", conversation_id)
self._inner.on_start(span, parent_context=parent_context)
def on_end(self, span):
self._inner.on_end(span)
def shutdown(self):
self._inner.shutdown()
def force_flush(self, timeout_millis=30000):
return self._inner.force_flush(timeout_millis=timeout_millis)
def setup_tracing() -> TracerProvider:
provider = TracerProvider(resource=Resource.create({"service.name": "travel-agent"}))
exporter = BatchSpanProcessor(ConsoleSpanExporter())
if os.getenv("NO_CONV_PROCESSOR"):
provider.add_span_processor(exporter)
else:
provider.add_span_processor(ConversationSpanProcessor(exporter))
trace.set_tracer_provider(provider)
return provider
tracer = trace.get_tracer("travel.concierge")
client = OpenAI(api_key="local-mock", base_url="http://127.0.0.1:8099/v1")
def dispatch_tool(name: str, args: dict) -> str:
if name == "get_weather":
return json.dumps({"city": args["city"], "temp_c": 31, "humidity": 78})
raise ValueError(f"unknown tool {name}")
def run_tool(call) -> str:
args = json.loads(call.function.arguments)
with tracer.start_as_current_span(f"execute_tool {call.function.name}") as span:
span.set_attribute("gen_ai.operation.name", "execute_tool")
span.set_attribute("gen_ai.tool.name", call.function.name)
span.set_attribute("gen_ai.tool.type", "function")
span.set_attribute("gen_ai.tool.call.id", call.id)
try:
result = dispatch_tool(call.function.name, args)
span.set_attribute("gen_ai.tool.call.result", result)
return result
except Exception as exc:
span.set_attribute("error.type", type(exc).__name__)
span.set_status(trace.Status(trace.StatusCode.ERROR, str(exc)))
raise
def run_agent(user_message: str, conversation_id: str) -> str:
token = CONVERSATION_ID.set(conversation_id)
try:
with tracer.start_as_current_span(f"invoke_agent {AGENT_NAME}") as span:
span.set_attribute("gen_ai.operation.name", "invoke_agent")
span.set_attribute("gen_ai.agent.name", AGENT_NAME)
span.set_attribute("gen_ai.agent.id", AGENT_ID)
span.set_attribute("gen_ai.provider.name", "openai")
span.set_attribute("gen_ai.request.model", MODEL)
messages = [
{"role": "system", "content": "You answer weather questions with the tool."},
{"role": "user", "content": user_message},
]
for step in range(3):
reply = client.chat.completions.create(
model=MODEL, messages=messages, tools=TOOLS
)
choice = reply.choices[0]
if choice.finish_reason == "tool_calls" and choice.message.tool_calls:
messages.append(choice.message)
for call in choice.message.tool_calls:
messages.append(
{
"role": "tool",
"tool_call_id": call.id,
"content": run_tool(call),
}
)
continue
return choice.message.content or ""
raise RuntimeError("agent did not finish within 3 steps")
finally:
CONVERSATION_ID.reset(token)
if __name__ == "__main__":
provider = setup_tracing()
OpenAIInstrumentor().instrument()
conversation_id = f"conv-{uuid.uuid4().hex[:12]}"
for turn in range(2):
answer = run_agent("How hot is it in Jakarta right now?", conversation_id)
print(f"turn {turn + 1}: {answer}", flush=True)
print(f"conversation_id: {conversation_id}", flush=True)
provider.shutdown()
Step 4: Run it with the two environment variables that matter
Both variables have to be exported in the shell. OTEL_SEMCONV_STABILITY_OPT_IN is read when the instrumentation module loads, so setting it from inside Python after the imports is too late, which is a mistake worth making once on purpose so you recognise the symptom: your attributes silently show up under older names.
export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=span_only
python agent_demo.py > spans.jsonl
Two turns produced eight spans, four per turn, and the tree looks like this:
invoke_agent travel-concierge (INTERNAL, root) gen_ai.conversation.id
├── chat gpt-4o-mini (CLIENT) finish_reasons=['tool_calls']
├── execute_tool get_weather (tool result on span)
└── chat gpt-4o-mini (CLIENT) finish_reasons=['stop']
Here is the first chat span as the console exporter printed it, trimmed to the GenAI attributes:
span='chat gpt-4o-mini' kind=CLIENT parent=invoke_agent
gen_ai.conversation.id = conv-ea5e82771faf
gen_ai.operation.name = chat
gen_ai.provider.name = openai
gen_ai.request.model = gpt-4o-mini
gen_ai.response.model = gpt-4o-mini
gen_ai.response.id = chatcmpl-mock1
gen_ai.response.finish_reasons = ['tool_calls']
gen_ai.usage.input_tokens = 412
gen_ai.usage.output_tokens = 96
And the tool span, which is the one you will actually search for at 03:00:
span='execute_tool get_weather'
gen_ai.operation.name = execute_tool
gen_ai.tool.name = get_weather
gen_ai.tool.type = function
gen_ai.tool.call.id = call_abc123
gen_ai.tool.call.result = {"city": "Jakarta", "temp_c": 31, "humidity": 78}
The nesting is the whole payoff. The parent on both child spans is the invoke_agent span, so a failure inside a tool is one click from the model call that requested it, and gen_ai.tool.call.id lets you join a span to the provider-side tool call id in your logs.
Step 5: The conversation id, which instrumentation will not guess
Set NO_CONV_PROCESSOR=1 and the same script produces zero spans carrying gen_ai.conversation.id. The instrumentation fills in provider, model, token counts, and finish reasons on its own, but it has no idea where one user conversation starts and ends, and the spec explicitly tells it not to invent one.
So you stamp it, from a ContextVar, with a span processor registered as the outermost processor on the provider. Because on_start runs for every span the process opens, the auto-instrumented chat spans that your code never touches get the attribute too. In the run above, eight spans landed on two trace ids and shared one conversation id, one per turn, with grouping across traces instead of a pile of unrelated runs.
In a real service you would not generate the id inside the agent. It comes from the request: your web handler reads the session or thread id and passes it down.
Step 6: Tokens, cost, and the free metrics
Token counts land on every chat span without extra work: gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, plus gen_ai.usage.cache_read.input_tokens when the provider reports cache hits. Cost is those two counts multiplied by your provider's price per token, which is why the convention does not pretend to know currency. Note that gen_ai.usage.input_tokens includes cached input tokens, so do not add the cache read count on top of it when you compute a bill.
The instrumentation also exports metrics from the same conventions: gen_ai.client.token.usage and gen_ai.client.operation.duration for model calls, plus gen_ai.invoke_agent.duration, gen_ai.invoke_agent.tool_calls, and gen_ai.execute_tool.duration for the agent layer. Those give you two alerts worth having on day one: p99 on the agent duration, and a counter on tool call failures.
A useful query once the data is in your backend: group by gen_ai.agent.name and gen_ai.request.model, sum input and output tokens per conversation, and sort ascending. The conversations that burn 40 model calls for a question that needed three are usually a prompt or a tool description problem, not a model problem.
Step 7: Content capture has four values and one trap
Message content is off by default, which is correct, because prompts carry PII. Turning it on is an enum, not a boolean:
true: legacy mode. Content goes out as log events, so a traces-only exporter shows you nothing.span_only: content on span attributes (gen_ai.input.messages,gen_ai.output.messages,gen_ai.system_instructions).event_only: content on events.span_and_event: both.
I ran the loop three times to see the difference. With true and no log exporter configured, the chat spans carried no message content at all: same eight spans, same attributes, no messages. With span_only, all four chat spans carried gen_ai.input.messages and gen_ai.output.messages. If you set the variable and still see no prompts, that is usually the reason.
For production there is a better default than putting prompts on spans at all: keep metadata on the span and upload the payload somewhere you control, using the built-in hook (OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK=upload with an OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH pointing at an fsspec-compatible path or bucket). Then the trace holds a reference and the sensitive text lives under your own retention rules.
Step 8: Ship it, and handle MCP tools
Swapping the console exporter for a real backend is three environment variables:
export OTEL_SERVICE_NAME=travel-agent
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
Any OTLP-speaking backend works, and switching later costs you nothing because the attribute names are the contract, not the vendor SDK.
If your tools arrive over MCP, the conventions add a sub-spec for it. MCP spans carry mcp.method.name (initialize, tools/call, ...), mcp.protocol.version, and mcp.session.id, with the span named {mcp.method.name} {target}. Trace context travels inside the JSON-RPC params._meta property bag as unprefixed traceparent and tracestate, per SEP-414, so the MCP server span becomes a child of your client span even though one HTTP request can carry several MCP messages. One rule saves you duplicate spans: if the MCP instrumentation can tell that GenAI instrumentation already traced the tool execution, it should add MCP attributes to that existing span instead of creating a second one.
Gotchas from the run
ModuleNotFoundError: No module named 'httpx'on first import. The instrumentation importshttpxat module load, whileopenai3.19.2 depends onhttpx2. Installinghttpxexplicitly fixed it. Pin the client and the instrumentation together and re-run after any bump, because the patching path is the part that breaks silently.- The stability opt-in is read at import time. Set it in the process environment, not in Python.
- The conventions are still marked Development. Several attributes shipped renames, so centralise attribute strings in one module and treat upgrades as explicit events.
gen_ai.conversation.idis your job. No processor, no grouping, and the spec forbids the lazy fallback.
Checklist for your own agent
invoke_agentaround the whole loop, oneexecute_toolper tool call,gen_ai.operation.nameon every span.- Provider name, model, both token counts, finish reasons on every chat span.
- Conversation id stamped from a processor, sourced from the real session, not a random UUID.
- Content capture off in production, or uploaded to your own storage.
- Alerts on agent duration p99 and tool failure counts before you need them at 03:00.