Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Microsoft OpenTelemetry Distro is a unified observability distribution that provides a single onboarding experience for collecting traces, metrics, and logs from agentic and nonagentic applications. It supports observability for Microsoft Agent 365, Microsoft Foundry, Azure Monitor, and any OpenTelemetry Protocol (OTLP)-compatible backend. The distro supports .NET, Node.js, and Python, and replaces fragmented setup across multiple observability stacks with one import and one configuration call.
Key benefits
The Microsoft OpenTelemetry Distro provides these benefits:
- One package, one API: Replace multiple exporter and instrumentation packages with a single dependency.
- Multi-backend support: Send telemetry to Azure Monitor, any OpenTelemetry Protocol (OTLP)-compatible endpoint such as Datadog, Grafana, or New Relic, and Microsoft Agent 365 at the same time.
- Built-in instrumentations: Use automatic instrumentation for HTTP, databases, Azure SDK, Azure Functions, and more with no extra configuration.
- Standards-based: Build on OpenTelemetry, the industry-standard observability framework.
- Minimal boilerplate: Add one import and one function call to your application entry point.
Installation and configuration
This guidance shows you how to add observability to your application with Microsoft OpenTelemetry Distro. The Distro automatically collects traces, metrics, and logs with built-in instrumentations, and exports the telemetry to Azure Monitor, any OpenTelemetry Protocol (OTLP) endpoint, or Microsoft Agent 365.
Install library
To get started with the Microsoft OpenTelemetry Distro, install the appropriate library for your development platform by using your language's package manager.
Configuration
The Agent 365 exporter doesn't use a connection string. It discovers its endpoint automatically based on tenant. To enable export to Agent 365, set the exporter target and provide a token resolver that returns an access token for a given agent ID and tenant ID.
Call use_microsoft_opentelemetry() to enable observability.
from microsoft.opentelemetry import use_microsoft_opentelemetry
from microsoft.opentelemetry.a365.hosting.token_cache_helpers import AgenticTokenCache
token_cache = AgenticTokenCache()
use_microsoft_opentelemetry(
enable_a365=True,
a365_token_resolver=lambda agent_id, tenant_id: (
(t := asyncio.run(token_cache.get_observability_token(agent_id, tenant_id)))
and t.token or None
),
)
For custom token resolution (instead of the default token resolver), see Manual token resolver.
You can customize the exporter behavior by passing optional a365_* kwargs to use_microsoft_opentelemetry().
| Parameter | Description | Default |
|---|---|---|
a365_use_s2s_endpoint |
When True, uses the service-to-service endpoint path. |
False |
a365_max_queue_size |
Maximum queue size for the batch processor. | 2048 |
a365_scheduled_delay_ms |
Delay in milliseconds between export batches. | 5000 |
a365_exporter_timeout_ms |
Timeout in milliseconds for the export operation. | 30000 |
a365_max_export_batch_size |
Maximum batch size for export operations. | 512 |
Propagate context
To maintain observability across distributed Agent 365 operations, propagate context. When you propagate context through your agents and services, you ensure that traces, logs, and metrics are properly correlated across the entire request lifecycle. This correlation is required for a complete and effective Microsoft Agent 365 monitoring experience.
Baggage attributes
Use BaggageBuilder to set contextual information that flows through all spans in a request.
The SDK implements a SpanProcessor that copies all nonempty baggage entries to newly started spans without overwriting existing attributes.
from microsoft.opentelemetry.a365.core import BaggageBuilder
with (
BaggageBuilder()
.tenant_id("tenant-123")
.agent_id("agent-456")
.conversation_id("conv-789")
.build()
):
# Any spans started in this context will receive these as attributes
pass
To auto-populate the BaggageBuilder from the TurnContext, use the populate helper in the microsoft-opentelemetry package. This helper automatically extracts caller, agent, tenant, channel, and conversation details from the activity.
from microsoft.opentelemetry.a365.core import BaggageBuilder
from microsoft.opentelemetry.a365.hosting.scope_helpers.populate_baggage import populate
builder = BaggageBuilder()
populate(builder, turn_context)
with builder.build():
# Baggage is auto-populated from the TurnContext activity
pass
Baggage middleware
If your agent uses the hosting integration package, register baggage middleware to automatically populate baggage for every incoming request. This step removes the need to call BaggageBuilder manually in each activity handler.
In Python, register baggage middleware through ObservabilityHostingManager.configure() rather than directly on the adapter.
from microsoft.opentelemetry.a365.hosting import ObservabilityHostingManager, ObservabilityHostingOptions
options = ObservabilityHostingOptions(enable_baggage=True)
ObservabilityHostingManager.configure(adapter.middleware_set, options)
The middleware skips baggage setup for async replies (ContinueConversation events) to avoid overwriting baggage that the originating request already set.
Validate data is flowing in product
To view agent telemetry in Microsoft Purview or Microsoft Defender, make sure the following requirements are met:
- Microsoft Purview: Auditing must be turned on for your organization. For instructions, see Turn auditing on or off.
- Microsoft Defender: Advanced hunting must be configured to access the
CloudAppEventstable. For details, see CloudAppEvents table in the advanced hunting schema.
Automatic instrumentation
The Microsoft OpenTelemetry Distro combines standard OpenTelemetry pipelines with Microsoft-curated instrumentation. The Distro can collect application telemetry, infrastructure telemetry, and agent or generative AI telemetry depending on language and configuration.
| Category | What it covers |
|---|---|
| Signal pipelines | Traces, metrics, and logs. |
| Resource detection | Service, host, cloud, and Azure runtime context where supported. |
| Infrastructure instrumentation | HTTP, ASP.NET Core, Azure SDK, database clients, and logging frameworks where supported. |
| Generative AI instrumentation | OpenAI, Azure OpenAI, Semantic Kernel, LangChain, OpenAI Agents SDK, and Agent Framework where supported. |
| Manual agent scopes | Agent invocation, tool execution, inference, and output telemetry where supported. |
| Exporters and processors | Azure Monitor, Microsoft Agent 365, OTLP, console output, span processors, log processors, and metric readers. |
Instrumentation coverage
| Language | Common application instrumentation | Common agent and generative AI instrumentation |
|---|---|---|
| Python | OpenTelemetry resources, processors, readers, logging, metrics, and traces. | Semantic Kernel, OpenAI Agents SDK, Agent Framework, LangChain, Microsoft Agent 365 baggage, and Microsoft Agent 365 scopes. |
| Node.js | HTTP, Azure SDK, Azure Functions, MongoDB, MySQL, PostgreSQL, Redis, Bunyan, and Winston. | OpenAI Agents SDK, LangChain, Microsoft Agent 365 baggage, and Microsoft Agent 365 scopes. |
| .NET | ASP.NET Core, HttpClient, SQL Client, Azure SDK, resource detection, metrics, and logs. | Semantic Kernel, OpenAI and Azure OpenAI, Agent Framework, Microsoft Agent 365 baggage, and Microsoft Agent 365 scopes. |
Automatic instrumentation listens to telemetry signals emitted by supported libraries and frameworks. Manual instrumentation is used when an application needs to describe agent-specific operations, such as invocation, tool execution, inference, or asynchronous output.
Add custom OpenTelemetry sources, meters, processors, or readers when your application emits telemetry that isn't covered by the built-in instrumentations.
Important
Automatic instrumentation populates standard OpenTelemetry attributes only. It doesn't include all attributes that Agent 365 requires. You must add Microsoft-specific attributes through BaggageBuilder. To see which attributes are required, see Store validation attributes.
Built-in instrumentation libraries
Auto-instrumentation listens to telemetry emitted by supported frameworks and forwards it through the Distro's OpenTelemetry pipeline. For agent scenarios, set baggage such as tenant ID and agent ID before the instrumented framework creates spans.
| Framework | Python | Node.js | .NET |
|---|---|---|---|
| Semantic Kernel | Supported | Not supported | Supported |
| OpenAI and OpenAI Agents SDK | Supported | Supported | Supported |
| Agent Framework | Supported | Not supported | Supported |
| LangChain | Supported | Supported | Not listed |
Semantic Kernel
from microsoft.opentelemetry import use_microsoft_opentelemetry
def token_resolver(agent_id, tenant_id):
return "your-token"
use_microsoft_opentelemetry(
enable_a365=True,
a365_token_resolver=token_resolver,
instrumentation_options={
"semantic_kernel": {"enabled": True},
},
)
OpenAI
from microsoft.opentelemetry import use_microsoft_opentelemetry
def token_resolver(agent_id, tenant_id):
return "your-token"
use_microsoft_opentelemetry(
enable_a365=True,
a365_token_resolver=token_resolver,
instrumentation_options={
"openai_agents": {"enabled": True},
},
)
Agent Framework
from microsoft.opentelemetry import use_microsoft_opentelemetry
def token_resolver(agent_id, tenant_id):
return "your-token"
use_microsoft_opentelemetry(
enable_a365=True,
a365_token_resolver=token_resolver,
instrumentation_options={
"agent_framework": {"enabled": True},
},
)
LangChain
Note
Auto-instrumentation for the LangChain framework also supports LangGraph and Deep Agents. The same instrumentation automatically captures telemetry for agents built with any of these frameworks.
from microsoft.opentelemetry import use_microsoft_opentelemetry
def token_resolver(agent_id, tenant_id):
return "your-token"
use_microsoft_opentelemetry(
enable_a365=True,
a365_token_resolver=token_resolver,
instrumentation_options={
"langchain": {"enabled": True},
},
)
Manual instrumentation
Use manual instrumentation when automatic instrumentation doesn't describe the agent operation with enough detail. Manual scopes let an application describe common agent activities in a consistent way across languages.
| Scope | Use for |
|---|---|
InvokeAgentScope |
The start and completion of an agent invocation. |
ExecuteToolScope |
A tool call made by an agent. |
InferenceScope |
An AI model inference operation. |
OutputScope |
Output that must be recorded after the originating scope has already completed. |
Reuse the same request and agent identity values across scopes in a request so related telemetry can be correlated.
Agent invocation
from microsoft.opentelemetry.a365.core import (
AgentDetails,
Channel,
InvokeAgentScope,
InvokeAgentScopeDetails,
Request,
ServiceEndpoint,
)
agent_details = AgentDetails(
agent_id="agent-456",
agent_name="Email Assistant",
agent_description="An AI agent powered by Azure OpenAI",
agentic_user_id="auid-123",
agentic_user_email="agent@contoso.com",
agent_blueprint_id="blueprint-789",
tenant_id="tenant-123",
)
request = Request(
content="Please help me organize my emails",
session_id="session-42",
conversation_id="conv-xyz",
channel=Channel(name="msteams"),
)
scope_details = InvokeAgentScopeDetails(
endpoint=ServiceEndpoint(hostname="myagent.contoso.com", port=443),
)
with InvokeAgentScope.start(
request=request,
scope_details=scope_details,
agent_details=agent_details,
) as scope:
scope.record_input_messages(["Please help me organize my emails"])
# Run the agent invocation.
invoke_scope.record_output_messages(["I found 15 urgent emails."])
Tool execution
from microsoft.opentelemetry.a365.core import (
ExecuteToolScope,
ServiceEndpoint,
ToolCallDetails,
ToolType,
)
tool_details = ToolCallDetails(
tool_name="email-search",
arguments={"query": "from:manager@contoso.com"},
tool_call_id="tool-call-456",
description="Search emails by criteria",
tool_type=ToolType.FUNCTION.value,
endpoint=ServiceEndpoint(
hostname="tools.contoso.com",
port=8080,
protocol="https",
),
)
with ExecuteToolScope.start(
request=request,
details=tool_details,
agent_details=agent_details,
) as scope:
result = search_emails(tool_details.arguments)
scope.record_response(result)
Inference
from microsoft.opentelemetry.a365.core import (
InferenceCallDetails,
InferenceOperationType,
InferenceScope,
)
inference_details = InferenceCallDetails(
operationName=InferenceOperationType.CHAT,
model="gpt-4o-mini",
providerName="azure-openai",
)
with InferenceScope.start(
request=request,
details=inference_details,
agent_details=agent_details,
) as scope:
scope.record_input_messages(["Summarize the following emails for me."])
response = call_llm()
scope.record_output_messages([response.text])
scope.record_input_tokens(response.usage.input_tokens)
scope.record_output_tokens(response.usage.output_tokens)
scope.record_finish_reasons(["stop"])
Output
from microsoft.opentelemetry.a365.core import OutputScope, Response, SpanDetails
# Capture this before exiting the originating InvokeAgentScope context.
parent_context = invoke_scope.get_span_context()
response = Response(
messages=["Here is your organized inbox."],
)
with OutputScope.start(
request=request,
response=response,
agent_details=agent_details,
user_details=None,
span_details=SpanDetails(parent_context=parent_context),
) as scope:
pass
Product documentation should define any product-specific validation requirements for these scopes.
Local validation
Local validation confirms that the application produces telemetry before a product-specific destination is validated. Use console output or a local OTLP endpoint to check that traces, metrics, and logs are created.
Validate with a local OTLP endpoint
Configure the Distro to send telemetry to a local collector or another OTLP-compatible endpoint.
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
from microsoft.opentelemetry import use_microsoft_opentelemetry
use_microsoft_opentelemetry()
Validate with local output
Use local output when you want to confirm instrumentation before sending telemetry to a remote destination.
export ENABLE_A365_OBSERVABILITY_EXPORTER=false
from microsoft.opentelemetry import use_microsoft_opentelemetry
def token_resolver(agent_id, tenant_id):
return "local-validation-token"
use_microsoft_opentelemetry(
enable_a365=True,
a365_token_resolver=token_resolver,
)
# Run instrumented application code.
Review the local output for spans from expected sources, such as HTTP requests, OpenAI or Azure OpenAI calls, agent invocation scopes, tool execution scopes, or inference scopes. Destination-specific validation belongs in the product documentation for that destination.
Manually set up authentication
When you use the Agent 365 exporter, you must provide a mechanism to supply an authentication token. The token resolver works per export batch by using the agent ID and tenant ID from the active baggage context. The distro supports two approaches.
Tip
If you're building agents with the Microsoft 365 Agents SDK, see Observability Authentication Setup for Agent SDK for step-by-step instructions on configuring OBO and S2S token acquisition for both agentic and non-agentic agents.
Manual token resolver
Use a manual resolver when you acquire tokens outside the Agent Framework pipeline, when you're building non-Agent Framework apps, or when you use service-to-service (S2S) authentication (client credentials flow). Agents can generate a token themselves, for example by using Microsoft Authentication Library (MSAL) or any other token acquisition method, but they need to ensure the token has the correct observability scope (api://9b975845-388f-4429-889e-eab1ef63949c/Agent365.Observability.OtelWrite).
Note
For service-to-service (S2S) authentication, you must use this manual token resolver approach. The agentic token cache only supports on-behalf-of (OBO) auth flows.
The following examples show the OBO (on-behalf-of) token resolver pattern — the agent acquires a user token via the agentic auth handler and exchanges it for an observability-scoped token. For S2S (service-to-service) examples and a comparison of OBO vs S2S authentication, see Observability Authentication Setup for Agent SDK.
The resolver must be synchronous. Acquire the token in your async activity handler (or via MSAL) and cache it for the resolver.
from microsoft.opentelemetry import use_microsoft_opentelemetry
from microsoft.opentelemetry.a365.runtime import get_observability_authentication_scope
_cached_token: str | None = None
def my_token_resolver(agent_id: str, tenant_id: str) -> str | None:
return _cached_token
use_microsoft_opentelemetry(enable_a365=True, a365_token_resolver=my_token_resolver)
@AGENT_APP.activity("message", auth_handlers=["AGENTIC"])
async def on_message(context: TurnContext, _state: TurnState):
global _cached_token
_cached_token = await AGENT_APP.auth.exchange_token(
context,
scopes=get_observability_authentication_scope(),
auth_handler_id="AGENTIC",
)
Agentic token cache with Agent Framework apps
For Agent Framework apps that use on-behalf-of (OBO) authentication, the distro automatically registers IExporterTokenCache<AgenticTokenStruct> via DI when you don't set a custom TokenResolver. Your agent calls RegisterObservability() at runtime to supply credentials, and the cache handles token acquisition and refresh.
Note
This approach only supports on-behalf-of (OBO) auth flows. For service-to-service (S2S) authentication, use the manual token resolver instead.
from microsoft.opentelemetry import use_microsoft_opentelemetry
from microsoft.opentelemetry.a365.hosting.token_cache_helpers import AgenticTokenCache, AgenticTokenStruct
from microsoft.opentelemetry.a365.runtime import get_observability_authentication_scope
token_cache = AgenticTokenCache()
_cached_tokens: dict[tuple[str, str], str | None] = {}
# Keep the sync resolver side-effect free; refresh the cache in the async request handler.
def sync_token_resolver(agent_id: str, tenant_id: str) -> str | None:
return _cached_tokens.get((agent_id, tenant_id))
use_microsoft_opentelemetry(enable_a365=True, a365_token_resolver=sync_token_resolver)
@AGENT_APP.activity("message", auth_handlers=["AGENTIC"])
async def on_message(context: TurnContext, _state: TurnState):
agent_id = context.activity.recipient.id
tenant_id = context.activity.recipient.tenant_id
token_cache.register_observability(
agent_id=agent_id,
tenant_id=tenant_id,
token_generator=AgenticTokenStruct(
authorization=AGENT_APP.auth,
turn_context=context,
),
observability_scopes=get_observability_authentication_scope(),
)
_cached_tokens[(agent_id, tenant_id)] = await token_cache.get_observability_token(
agent_id, tenant_id,
)
Store validation attributes
For successful store validation, your agent must implement InvokeAgentScope, InferenceScope, and ExecuteToolScope. Each scope corresponds to a span operation in the canonical schema:
| SDK scope | Span operation | Universal reference code |
|---|---|---|
InvokeAgentScope |
invoke_agent |
IA |
ExecuteToolScope |
execute_tool |
ET |
InferenceScope |
chat |
CH |
OutputScope |
output_messages |
OM |
For the full per-scope required and optional attribute lists - including the per-attribute semantics, value-picking guidance, and which attributes are queryable through Microsoft Defender advanced hunting - see Agent 365 observability attribute reference. The Applies to column identifies which scope each attribute belongs to, and the Required column distinguishes mandatory (M) from optional (O) attributes.
Test your agent with observability
After implementing observability, verify that telemetry is being captured:
- Go to
https://admin.cloud.microsoft/#/agents/all. - Select your agent, and then select Activity.