Observability 2.0: Tracing AI "Thought Chains" with OpenTelemetry

tercel Posted on May 31 Observability 2.0: Tracing AI "Thought Chains" with OpenTelemetry # agents # ai # llm # monitoring "Why did the Agent do that?" If you are building Agentic systems today, this is the question that keeps you up at night. AI Agents are inherently non-deterministic. They loop, they reason, and they call multiple tools in sequences that are hard to predict. When a multi-step task fails, a traditional stack trace is useless. You don't just need to know where the code crashed; you need to know what the AI was thinking. In this seventeenth article, and the conclusion of our Engine volume, we explore how apcore integrates with OpenTelemetry (OTel) to turn the "Black Box" of AI reasoning into a transparent, traceable "Glass Box." The Concept of the "Thought Span" In traditional distributed tracing, a "Span" represents a single unit of work—like an HTTP request or a database query. In apcore, we introduce the Thought Span . Every time the Executor.call() method is triggered, apcore automatically wraps the execution in an OpenTelemetry span. This span isn't just a timer; it’s a rich container of AI-specific metadata: Input/Output Data : What did the AI send, and what did the module return? (Sensitive data is automatically redacted). ACL Decisions : Which rule allowed or denied this call? Approval Events : Did a human intervene? How long did the Agent wait? AI Guidance : If an error occurred, what self-healing instructions were sent back to the model? Distributed Tracing: From LLM to DB One of the most powerful features of apcore is its ability to propagate the trace_id across network boundaries. Imagine a user makes a request to your web frontend. That request enters a fastapi-apcore adapter, triggers an Orchestrator Agent, which then calls a tool module, which finally queries a PostgreSQL database. Because apcore is W3C Trace-Context compatible , the same trace_id is carried through the entire journey. When you open a tool like Jaeger , Grafa