Azure Agents client library for Java - version 2.6.0

Develop Agents using the Azure AI Foundry platform, leveraging an extensive ecosystem of models, tools, and capabilities from OpenAI, Microsoft, and other LLM providers.

The client library uses a single service version v1 of the AI Foundry data plane REST APIs.

Documentation

Various documentation is available to help you get started

• API reference documentation
• Product documentation

Getting started

Prerequisites

• Java Development Kit (JDK) with version 8 or above
• Azure Subscription

Adding the package to your product

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-agents</artifactId>
    <version>2.6.0</version>
</dependency>

Authentication

Azure Identity package provides the default implementation for authenticating the client.

Key concepts

Create an AgentsClient

To interact with the Azure Agents service, you'll need to create an instance of the AgentsClient class.

AgentsClient agentsClient = new AgentsClientBuilder()
                .credential(new DefaultAzureCredentialBuilder().build())
                .endpoint(endpoint)
                .buildAgentsClient();

Alternatively, you can create an asynchronous client using the AgentsAsyncClient class.

AgentsAsyncClient agentsAsyncClient = new AgentsClientBuilder()
                .credential(new DefaultAzureCredentialBuilder().build())
                .endpoint(endpoint)
                .buildAgentsAsyncClient();

The Agents client library has the following sub-clients which group the different operations that can be performed:

• AgentsClient / AgentsAsyncClient: Perform operations related to agents, such as creating, retrieving, updating, and deleting agents. When allowPreview(true) is configured, these clients can also use preview definitions, hosted-agent sessions, session files, and code package operations.
• BetaAgentsClient / BetaAgentsAsyncClient (preview): Perform preview agent optimization operations.
• ResponsesClient / ResponsesAsyncClient: Create responses that require Azure-specific request fields, such as an explicit AgentReference or structured inputs. For standard OpenAI Responses API calls through a configured agent endpoint, use an agent-scoped OpenAI client. See the OpenAI Responses API documentation for more information.
• BetaMemoryStoresClient / BetaMemoryStoresAsyncClient (preview): Manage memory stores and individual memory items for agents.
• ToolboxesClient / ToolboxesAsyncClient: Manage toolboxes and toolbox versions.
• BetaVoiceAgentWebSocketClient / BetaVoiceAgentWebSocketAsyncClient (preview): Open typed realtime WebSocket sessions with voice agents.
• BetaVoiceAgentsTelephonyClient / BetaVoiceAgentsTelephonyAsyncClient (preview): Manage telephony bindings, calls, transfer targets, and outbound call jobs.
• BetaVoiceAgentsConversationsClient / BetaVoiceAgentsConversationsAsyncClient (preview): Read and delete persisted voice-agent conversations and retrieve their responses, items, and audio.

OpenAI conversation operations are accessed through the OpenAI Official Java SDK's ConversationService. Persisted voice-agent conversation records use the beta voice conversation clients listed above. See the OpenAI Conversation API documentation for more information.

To access each sub-client you need to use your AgentsClientBuilder(). The Agents client library takes the Official OpenAI SDK as a dependency, which is used for all operations, except the ones corresponding to direct Agent management.

AgentsClientBuilder builder = new AgentsClientBuilder()
                .credential(new DefaultAzureCredentialBuilder().build())
                .endpoint(endpoint)
                .allowPreview(true); // Only needed for preview APIs on non-Beta clients that support them.

// Agents sub-clients
AgentsClient agentsClient = builder.buildAgentsClient();
AgentsAsyncClient agentsAsyncClient = builder.buildAgentsAsyncClient();
// Beta* clients automatically opt in to their preview service area.
BetaAgentsClient betaAgentsClient = builder.beta().buildBetaAgentsClient();
BetaAgentsAsyncClient betaAgentsAsyncClient = builder.beta().buildBetaAgentsAsyncClient();
// Responses sub-clients.
ResponsesClient responsesClient = builder.buildResponsesClient();
ResponsesAsyncClient responsesAsyncClient = builder.buildResponsesAsyncClient();
// Memory Stores sub-clients (preview).
BetaMemoryStoresClient memoryStoresClient = builder.beta().buildBetaMemoryStoresClient();
BetaMemoryStoresAsyncClient memoryStoresAsyncClient = builder.beta().buildBetaMemoryStoresAsyncClient();
// Toolboxes sub-clients.
ToolboxesClient toolboxesClient = builder.buildToolboxesClient();
ToolboxesAsyncClient toolboxesAsyncClient = builder.buildToolboxesAsyncClient();

The OpenAI Official Java SDK is imported transitively and can be built directly from the AgentsClientBuilder. Use it to access conversation operations and other OpenAI services:

OpenAIClient openAIClient = builder.buildOpenAIClient();
OpenAIClientAsync openAIAsyncClient = builder.buildOpenAIAsyncClient();

// Agent-scoped OpenAI clients for invoking a configured agent endpoint.
OpenAIClient agentScopedOpenAIClient = builder.buildAgentScopedOpenAIClient(agentName);
OpenAIClientAsync agentScopedOpenAIAsyncClient = builder.buildAgentScopedOpenAIAsyncClient(agentName);

// ResponsesClient wraps the OpenAI SDK's ResponseService with Azure-specific options.
ResponsesClient responsesClient = builder.buildResponsesClient();
ResponseService responseService = responsesClient.getResponseService();

// OpenAI SDK ConversationService accessed from OpenAIClient
ConversationService conversationService = openAIClient.conversations();

Agent version drafts

Draft agent versions are preview candidates that are not promoted to the agent's latest released version. Create one with CreateAgentVersionInput.setDraft(true), and pass true as the includeDrafts argument to listAgentVersions when you need to list draft versions. Build the non-Beta client with allowPreview(true) to opt in to the DraftAgents=V1Preview service feature.

See the full samples in AgentDraftSample.java and AgentDraftAsyncSample.java.

Agent tools

The SDK supports a variety of tools that can be attached to agent definitions. Some tools are generally available, while others are in preview and may change in future releases.

Generally available tools:

Preview tools:

Supported tool classes may also expose optional name, description, and toolConfigs properties for user-defined labels and per-tool configuration.

Preview operation groups and beta clients

Several operation groups in the Agents client library expose preview service features. These features require the Foundry-Features HTTP header. The SDK populates that header for you; you do not need to set the header value manually.

APIs annotated with @Beta are part of the SDK's preview surface, even when they appear on a non-Beta client. These APIs are subject to breaking changes in future releases and should be used with the same compatibility expectations as other preview features.

Use AgentsClientBuilder.allowPreview(true) when building non-Beta clients that support preview service behavior. For example, AgentsClient and AgentsAsyncClient use this builder setting to allow the service to return preview response types:

AgentsClientBuilder builder = new AgentsClientBuilder()
    .credential(new DefaultAzureCredentialBuilder().build())
    .endpoint(endpoint)
    .allowPreview(true);

AgentsClient agentsClient = builder.buildAgentsClient();

Build clients whose names start with Beta from AgentsClientBuilder.beta(). These clients always opt in to their corresponding preview service area. Requests sent by these clients automatically include the appropriate Foundry-Features header, and their APIs can send or return preview/beta request and response types. You do not need to call allowPreview(true) to use a Beta*Client.

The async Beta*AsyncClient counterparts follow the same behavior.

Realtime voice-agent sessions

Create and manage preview voice agents with VoiceAgentDefinition and an AgentsClient or AgentsAsyncClient built using allowPreview(true). Use BetaVoiceAgentWebSocketClient or BetaVoiceAgentWebSocketAsyncClient to open a typed, bidirectional session with an existing voice agent. The client acquires a token for https://ai.azure.com/.default, negotiates the realtime WebSocket subprotocol, and sends the required VoiceAgents=V1Preview feature header automatically.

The session API supports text and audio in the format configured on the voice agent, typed streaming server events, response cancellation, client-executed function tools, and optional persisted conversations. See Realtime voice-agent WebSocket examples for complete samples.

Use VoiceAgentWebSocketConnectionOptions with openWebSocketSession to configure session IDs, agent version selection, structured inputs, persistence, buffering, and timeouts. Options are copied when the session is opened, so later changes do not affect the active session.

VoiceAgentWebSocketConnectionOptions options = new VoiceAgentWebSocketConnectionOptions()
    .setAgentSessionId("session-id")
    .setAgentVersionOverride("2")
    .setStructuredInputs("{\"language\":\"en\"}")
    .setStoreEnabled(true);

The SDK owns the WebSocket route, API version, authentication scope, transport, and preview feature headers. Endpoint, credential, service version, configuration-based proxy settings, and ClientOptions are reused from AgentsClientBuilder. Custom HTTP clients, pipelines, policies, and retry settings are rejected when building a WebSocket client because the native WebSocket transports cannot apply them.

Agent optimization

The preview BetaAgentsClient and BetaAgentsAsyncClient can create and monitor agent optimization jobs. These jobs evaluate an agent against a registered dataset and evaluator, then return scored candidates for instructions, skills, tools, or model improvements. Agent optimization is currently in preview and requires an allow-listed Foundry project. See Agent optimizer in Foundry Agent Service for the service workflow and the complete examples in AgentOptimizationSample.java and AgentOptimizationAsyncSample.java.

Memory item management

BetaMemoryStoresClient and BetaMemoryStoresAsyncClient manage memory stores and individual memory items. In addition to store-level operations, use createMemory, updateMemory, listMemories, getMemory, and deleteMemory to manage individual memories. ListMemoriesOptions supports filtering by scope and MemoryItemKind, including MemoryItemKind.PROCEDURAL. See MemoryStoreItemsSample and MemoryStoreItemsAsyncSample for complete examples.

For conversational memory workflows, use beginUpdateMemories to extract memories from conversation items, searchMemories to retrieve relevant memories, and deleteScope to remove all memories for a scope. See MemoryStoreAdvancedSample and MemoryStoreAdvancedAsyncSample for complete synchronous and asynchronous examples.

Using OpenAI's official library

If you prefer using the OpenAI official Java client library instead, you can do so by including that dependency in your project instead and following the instructions in the linked repository. Additionally, you will have to set up your OpenAIClient as shown below:

OpenAIClient client = OpenAIOkHttpClient.builder()
    .baseUrl(endpoint.endsWith("/") ? endpoint + "openai/v1" : endpoint + "/openai/v1")
    .credential(BearerTokenCredential.create(AuthenticationUtil.getBearerTokenSupplier(
        new DefaultAzureCredentialBuilder().build(), "https://ai.azure.com/.default")))
    .build();

ResponseCreateParams responseRequest = new ResponseCreateParams.Builder()
    .input("Hello, how can you help me?")
    .model(model)
    .build();

Response result = client.responses().create(responseRequest);

For this direct setup, ensure that the AI Foundry project endpoint path ends with openai/v1, as shown above.

Examples

Prompt Agent

This example shows how to create and invoke a PromptAgent with conversation context that can be shared across multiple agents.

Create an Agent

Creating an Agent can be done like in the following code snippet:

PromptAgentDefinition promptAgentDefinition = new PromptAgentDefinition("gpt-4o");
AgentVersionDetails agent = agentsClient.createAgentVersion("my-agent", promptAgentDefinition);

This returns an AgentVersionDetails containing the name and version used to configure the agent endpoint. The following steps also create a Conversation to provide centralized context that can be shared across agents.

Create conversation

First we need to create our Conversation object so we can attach items to it:

Conversation conversation = conversationsClient.create();

The value returned by conversation.id() identifies the conversation when appending messages. Conversation objects can be used by multiple agents as a centralized source of context. To add items:

conversationsClient.items().create(
    ItemCreateParams.builder()
        .conversationId(conversation.id())
        .addItem(EasyInputMessage.builder()
            .role(EasyInputMessage.Role.SYSTEM)
            .content("You are a helpful assistant that speaks like a pirate.")
            .build()
        ).addItem(EasyInputMessage.builder()
            .role(EasyInputMessage.Role.USER)
            .content("Hello, agent!")
            .build()
    ).build()
);

To scope conversation operations to a delegated end user, set FOUNDRY_USER_IDENTITY to an opaque application-generated value and apply it as the x-ms-user-identity header. The caller must have the agents/endpoints/UserIdentityImpersonation/action RBAC permission. See the sync UserIdentityConversation.java and async UserIdentityConversationAsync.java samples.

Configure the agent endpoint

An agent can have multiple versions. Before invoking it through the OpenAI Responses API, configure its endpoint with a version-selection rule and enable the Responses protocol. This example sends all endpoint traffic to the version just created; the endpoint configuration remains in effect until it is updated again:

AgentEndpointConfig endpointConfig = new AgentEndpointConfig()
    .setVersionSelector(new VersionSelector().setVersionSelectionRule(
        new FixedRatioVersionSelectionRule(100).setAgentVersion(agent.getVersion())))
    .setProtocolConfiguration(new ProtocolConfiguration().setResponses(new ResponsesProtocolConfiguration()));

agentsClient.updateAgentDetails(agent.getName(),
    new UpdateAgentDetailsOptions().setAgentEndpoint(endpointConfig));

Text generation with Responses

With the agent endpoint configured, build an agent-scoped OpenAI client and invoke the OpenAI Responses API:

OpenAIClient agentScopedClient = builder.buildAgentScopedOpenAIClient(agent.getName());

Response response = agentScopedClient.responses().create(ResponseCreateParams.builder()
    .conversation(conversation.id())
    .build());
// To extract Azure-specific response details:
AzureCreateResponseDetails azureResults = ResponsesClient.getAzureFields(response);

For asynchronous calls, use buildAgentScopedOpenAIAsyncClient.

Using Agent tools

Agents can be enhanced with specialized tools for various capabilities. For complete working examples, see the tools/ folder under samples.

In the description below, tools are organized by their Foundry connection requirements: "Built-in Tools" (which do not require a Foundry connection) and "Connection-based Tools" (which require a Foundry connection).

Built-in Tools

These tools work immediately without requiring external connections.

Code Interpreter (documentation)

Write and run Python code in a sandboxed environment, process files and work with diverse data formats.

// Create a CodeInterpreterTool with default auto container configuration
CodeInterpreterTool tool = new CodeInterpreterTool();

See the full sample in CodeInterpreterSync.java.

File Search (documentation)

Search through files in a vector store for knowledge retrieval:

// Create a FileSearchTool with the vector store ID
FileSearchTool tool = new FileSearchTool(Collections.singletonList(vectorStore.id()));

See the full sample in FileSearchSync.java.

Image Generation (documentation)

Generate images from text descriptions:

// Create image generation tool with model, quality, and size
ImageGenTool imageGenTool = new ImageGenTool()
    .setModel(ImageGenToolModel.fromString(imageModel))
    .setQuality(ImageGenToolQuality.LOW)
    .setSize(ImageGenToolSize.fromString("1024x1024"));

See the full sample in ImageGenerationSync.java.

Web Search (Preview) (documentation)

Search the web for current information:

// Create a WebSearchPreviewTool
WebSearchPreviewTool tool = new WebSearchPreviewTool();

See the full sample in WebSearchSync.java.

Computer Use (Preview) (documentation)

Interact with computer interfaces through simulated actions and screenshots:

ComputerUsePreviewTool tool = new ComputerUsePreviewTool(
    ComputerEnvironment.WINDOWS,
    1026,
    769
);

See the full sample in ComputerUseSync.java.

Model Context Protocol (MCP) (documentation)

Connect agents to external MCP servers:

// Uses gitmcp.io to expose a GitHub repository as an MCP-compatible server
McpTool tool = new McpTool("api-specs")
    .setServerUrl("https://gitmcp.io/Azure/azure-rest-api-specs")
    .setRequireApproval("always");

See the full sample in McpSync.java.

OpenAPI (documentation)

Call external APIs defined by OpenAPI specifications without additional client-side code:

// Load the OpenAPI spec from a JSON file
Map<String, BinaryData> spec = OpenApiFunctionDefinition.readSpecFromFile(
    SampleUtils.getResourcePath("assets/httpbin_openapi.json"));

OpenApiTool tool = new OpenApiTool(
    new OpenApiFunctionDefinition(
        "httpbin_get",
        spec,
        new OpenApiAnonymousAuthDetails())
        .setDescription("Get request metadata from an OpenAPI endpoint."));

See the full sample in OpenApiSync.java.

Function Tool (documentation)

Define custom functions that allow agents to interact with external APIs, databases, or application logic:

Map<String, Object> locationProp = new LinkedHashMap<String, Object>();
locationProp.put("type", "string");
locationProp.put("description", "The city and state, e.g. Seattle, WA");

Map<String, Object> unitProp = new LinkedHashMap<String, Object>();
unitProp.put("type", "string");
unitProp.put("enum", Arrays.asList("celsius", "fahrenheit"));

Map<String, Object> properties = new LinkedHashMap<String, Object>();
properties.put("location", locationProp);
properties.put("unit", unitProp);

Map<String, BinaryData> parameters = new HashMap<String, BinaryData>();
parameters.put("type", BinaryData.fromObject("object"));
parameters.put("properties", BinaryData.fromObject(properties));
parameters.put("required", BinaryData.fromObject(Arrays.asList("location", "unit")));
parameters.put("additionalProperties", BinaryData.fromObject(false));

FunctionTool tool = new FunctionTool("get_weather", parameters, true)
    .setDescription("Get the current weather in a given location");

See the full sample in FunctionCallSync.java.

Azure Functions

Integrate Azure Functions with agents to extend capabilities via serverless compute. Functions are invoked through Azure Storage Queue triggers, allowing asynchronous execution of custom logic:

// Create Azure Function tool with Storage Queue bindings
AzureFunctionTool azureFunctionTool = new AzureFunctionTool(
    new AzureFunctionDefinition(
        new AzureFunctionDefinitionDetails("queue_trigger", parameters)
            .setDescription("Get weather for a given location"),
        new AzureFunctionBinding(
            new AzureFunctionStorageQueue(queueServiceEndpoint, inputQueueName)),
        new AzureFunctionBinding(
            new AzureFunctionStorageQueue(queueServiceEndpoint, outputQueueName))
    )
);

When the agent handles a response, it enqueues function arguments to the input queue. Your Azure Function processes the request and returns results through the output queue.

See the full sample in AzureFunctionSync.java.

Memory Search (Preview) (documentation)

The Memory Search tool adds memory to an agent, allowing the agent's AI model to search for past information related to the current user prompt:

// Create memory search tool
MemorySearchPreviewTool tool = new MemorySearchPreviewTool(memoryStore.getName(), scope)
    .setUpdateDelaySeconds(1);

See the full sample in MemorySearchSync.java showing how to create an agent with a memory store and use it across multiple conversations.

Connection-Based Tools

These tools require configuring connections in your Microsoft Foundry project and use a projectConnectionId.

Azure AI Search (documentation)

Integrate with Azure AI Search indexes for powerful knowledge retrieval and semantic search capabilities:

// Create Azure AI Search tool with index configuration
AzureAISearchTool aiSearchTool = new AzureAISearchTool(
    new AzureAISearchToolResource(Arrays.asList(
        new AISearchIndexResource()
            .setProjectConnectionId(connectionId)
            .setIndexName(indexName)
            .setQueryType(AzureAISearchQueryType.SIMPLE)
    ))
);

See the full sample in AzureAISearchSync.java.

Bing Grounding (documentation)

Ground agent responses with real-time web search results from Bing to provide up-to-date information:

// Create Bing grounding tool with connection configuration
BingGroundingTool bingTool = new BingGroundingTool(
    new BingGroundingSearchToolParameters(Arrays.asList(
        new BingGroundingSearchConfiguration(bingConnectionId)
    ))
);

See the full sample in BingGroundingSync.java.

Bing Custom Search (Preview) (documentation)

Warning: Grounding with Bing Custom Search uses Grounding with Bing, which has additional costs and terms: terms of use and privacy statement. Customer data will flow outside the Azure compliance boundary.

Use custom-configured Bing search instances for domain-specific or filtered web search results:

// Create Bing Custom Search tool with connection and instance configuration
BingCustomSearchPreviewTool bingCustomSearchTool = new BingCustomSearchPreviewTool(
    new BingCustomSearchToolParameters(Arrays.asList(
        new BingCustomSearchConfiguration(connectionId, instanceName)
    ))
);

See the full sample in BingCustomSearchSync.java.

Microsoft Fabric (Preview) (documentation)

Query data from Microsoft Fabric data sources:

// Create Microsoft Fabric tool with connection configuration
MicrosoftFabricPreviewTool fabricTool = new MicrosoftFabricPreviewTool(
    new FabricDataAgentToolParameters()
        .setProjectConnections(Arrays.asList(
            new ToolProjectConnection(fabricConnectionId)
        ))
);

See the full sample in FabricSync.java.

Fabric IQ (Preview) (documentation)

Connect agents to Fabric IQ project connections for enterprise data grounding:

FabricIqPreviewTool fabricIqTool = new FabricIqPreviewTool(fabricIqConnectionId)
    .setServerLabel("fabric-iq-tool")
    .setRequireApproval("never");

The samples use FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL_NAME, and the fully qualified FABRIC_IQ_PROJECT_CONNECTION_ID. FOUNDRY_AGENT_NAME and FABRIC_IQ_USER_INPUT are optional. The response text and any returned annotations are printed before the temporary agent version is deleted.

See the full samples in FabricIQSync.java and FabricIQAsync.java.

Work IQ (Preview) (documentation)

Ground agent responses in the signed-in user's Microsoft 365 work context through a Work IQ project connection:

// Create a Work IQ tool with a fully qualified project connection resource ID
WorkIqPreviewTool workIqTool = new WorkIqPreviewTool(workIqConnectionId);

Set FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL_NAME, and WORK_IQ_PROJECT_CONNECTION_ID before running the sample. FOUNDRY_AGENT_NAME and WORK_IQ_USER_INPUT are optional. Work IQ uses delegated authentication and honors the signed-in user's Microsoft 365 permissions.

See the full samples in WorkIQSync.java and WorkIQAsync.java.

Microsoft SharePoint (Preview) (documentation)

Search through SharePoint documents for grounding:

// Create SharePoint grounding tool with connection configuration
SharepointPreviewTool sharepointTool = new SharepointPreviewTool(
    new SharepointGroundingToolParameters()
        .setProjectConnections(Arrays.asList(
            new ToolProjectConnection(sharepointConnectionId)
        ))
);

See the full sample in SharePointGroundingSync.java.

Browser Automation (Preview) (documentation)

Interact with web pages through browser automation:

// Create browser automation tool with connection configuration
BrowserAutomationPreviewTool browserTool = new BrowserAutomationPreviewTool(
    new BrowserAutomationToolParameters(
        new BrowserAutomationToolConnectionParameters(connectionId)
    )
);

See the full sample in BrowserAutomationSync.java.

Agent-to-Agent (A2A) (documentation)

Enable agent-to-agent communication with remote A2A endpoints:

// Create agent-to-agent tool with A2A protocol version and connection ID
A2ATool a2aTool = new A2ATool(A2AProtocolVersion.V1_0)
    .setProjectConnectionId(a2aConnectionId);

See the full sample in AgentToAgentSync.java.

MCP with Project Connection (documentation)

MCP integration using project-specific connections for accessing connected MCP servers:

// Create MCP tool with project connection authentication
McpTool mcpTool = new McpTool("api-specs")
    .setServerUrl("https://api.githubcopilot.com/mcp")
    .setProjectConnectionId(mcpConnectionId)
    .setRequireApproval("always");

See the full sample in McpWithConnectionSync.java.

OpenAPI with Project Connection (documentation)

Call external APIs defined by OpenAPI specifications using project connection authentication:

// Create OpenAPI tool with project connection authentication
OpenApiTool openApiTool = new OpenApiTool(
    new OpenApiFunctionDefinition(
        "httpbin_get",
        spec,
        new OpenApiProjectConnectionAuthDetails(
            new OpenApiProjectConnectionSecurityScheme(connectionId)))
        .setDescription("Get request metadata from an OpenAPI endpoint."));

See the full sample in OpenApiWithConnectionSync.java.

Toolbox Tools

Toolbox tools are defined in toolbox versions and managed through ToolboxesClient / ToolboxesAsyncClient. Toolbox versions use ToolboxTool subclasses rather than agent Tool subclasses. Use invokeLatestToolboxMcp to invoke the latest toolbox version through its MCP endpoint.

Toolbox Search

Toolbox Search lets an agent search the available toolbox tools at runtime. The GA implementation is ToolSearchToolboxTool (toolbox_search), and the preview implementation ToolboxSearchPreviewToolboxTool (toolbox_search_preview) is maintained alongside it for backward compatibility.

ToolSearchToolboxTool toolboxSearchTool = new ToolSearchToolboxTool()
    .setName("search_tools")
    .setDescription("Search over available toolbox tools at runtime.");

ToolboxVersionDetails version = toolboxesClient.createToolboxVersion(
    toolboxName,
    Collections.singletonList(toolboxSearchTool),
    "Toolbox version with a Toolbox Search tool.",
    null,
    null,
    null);

System.out.printf("Created toolbox: %s%n", version.getName());
System.out.printf("Toolbox version: %s%n", version.getVersion());
for (ToolboxTool tool : version.getTools()) {
    System.out.printf("Tool type: %s%n", tool.getType());
}

See the full sample in ToolboxSearchToolboxSample.java.

Reminder (preview)

The Reminder tool lets a hosted agent schedule itself to run again at a future time. It is connectionless and is available only to hosted agents, not prompt agents.

ReminderPreviewToolboxTool reminderTool = new ReminderPreviewToolboxTool()
    .setName("schedule_reminder")
    .setDescription("Schedule a reminder that re-invokes this agent at a future time.");

ToolboxVersionDetails version = toolboxesClient.createToolboxVersion(
    toolboxName,
    Collections.<ToolboxTool>singletonList(reminderTool),
    "Built-in reminder tool for a self-scheduling agent.",
    null,
    null,
    null);

System.out.printf("Created toolbox: %s%n", version.getName());
System.out.printf("Toolbox version: %s%n", version.getVersion());
System.out.printf("Tool type: %s%n", version.getTools().get(0).getType());

See the full samples in ReminderPreviewToolboxSample.java and ReminderPreviewToolboxAsyncSample.java.

Shell

The Shell toolbox tool runs commands in an isolated container. This example uses an automatically provisioned container, which has outbound network access disabled by default. A prompt agent consumes the toolbox through its versioned MCP endpoint.

ShellToolboxTool shellTool = new ShellToolboxTool(new ToolboxShellContainerAutoEnvironment())
    .setDescription("Runs shell commands in a sandboxed container.");

ToolboxVersionDetails toolboxVersion = toolboxesClient.createToolboxVersion(
    toolboxName,
    Collections.<ToolboxTool>singletonList(shellTool),
    "Toolbox with a shell tool running in an auto-provisioned container.",
    null,
    null,
    null);

See the full end-to-end sample in ShellToolboxSample.java.

Streaming responses

An agent-scoped OpenAI client can stream response events as they arrive instead of waiting for the complete response. This is useful for displaying text in real time and observing tool execution progress.

Synchronous streaming

The OpenAI SDK's synchronous createStreaming method returns a StreamResponse<ResponseStreamEvent>. Close it with try-with-resources, and use ResponseAccumulator to collect the events into a final Response:

// Use ResponseAccumulator to collect streamed events into a final Response
ResponseAccumulator responseAccumulator = ResponseAccumulator.create();

// Stream response - text is printed as it arrives
try (StreamResponse<ResponseStreamEvent> events = openAIClient.responses().createStreaming(
        ResponseCreateParams.builder()
            .input("Tell me a short story about a brave explorer.")
            .build())) {

    events.stream().forEach(event -> {
        responseAccumulator.accumulate(event);
        event.outputTextDelta()
            .ifPresent(textEvent -> System.out.print(textEvent.delta()));
    });
}
System.out.println(); // newline after streamed text

// Access the complete accumulated response
Response response = responseAccumulator.response();
System.out.println("\nResponse ID: " + response.id());

See the full samples in SimpleStreamingSync.java, FunctionCallStreamingSync.java, and CodeInterpreterStreamingSync.java.

Asynchronous streaming

The OpenAI SDK's asynchronous createStreaming method returns an AsyncStreamResponse<ResponseStreamEvent>. Use StreamingResponseUtils.toFlux to adapt it to a Reactor Flux and manage the underlying stream lifecycle:

// Adapt OpenAI streaming events to a Reactor Flux.
Mono<Void> streamingCompletion = Mono.defer(() -> {
    ResponseAccumulator responseAccumulator = ResponseAccumulator.create();
    AsyncStreamResponse<ResponseStreamEvent> stream = openAIAsyncClient.responses().createStreaming(
        ResponseCreateParams.builder()
            .input("Tell me a short story about a brave explorer.")
            .build());

    return StreamingResponseUtils.toFlux(stream)
        .doOnNext(event -> responseAccumulator.accumulate(event)
            .outputTextDelta()
            .ifPresent(textEvent -> System.out.print(textEvent.delta())))
        .then()
        .doOnSuccess(unused -> {
            System.out.println(); // newline after streamed text

            // Access the complete accumulated response
            Response response = responseAccumulator.response();
            System.out.println("\nResponse ID: " + response.id());
        });
});

See the full samples in SimpleStreamingAsync.java, FunctionCallStreamingAsync.java, and CodeInterpreterStreamingAsync.java.

Stream hosted agent session logs

Hosted agent session logs can be streamed as SessionLogEvent values after a session has been created. The data property contains the log payload as an opaque string.

Session log streams are long-lived and may remain open until the client cancels or the session ends. Bound the stream with take, timeout, by disposing the subscription, or by breaking iteration when appropriate.

Synchronous session log streaming

The synchronous session log method returns IterableStream<SessionLogEvent>, which can be consumed with a standard for-each loop:

IterableStream<SessionLogEvent> sessionLogs =
    agentsClient.getSessionLogStream(agentName, agentVersion, sessionId);

int logsRead = 0;
for (SessionLogEvent event : sessionLogs) {
    System.out.printf("[%s] %s%n", event.getEvent(), event.getData());

    // Session log streams are long-lived; connection is closed on client disconnection
    if (++logsRead == 100) {
        break;
    }
}

Asynchronous session log streaming

The asynchronous session log method returns Flux<SessionLogEvent>, integrating naturally with Reactor pipelines:

agentsAsyncClient.getSessionLogStream(agentName, agentVersion, sessionId)
    .take(100)
    .doOnNext(event -> System.out.printf("[%s] %s%n", event.getEvent(), event.getData()))
    .blockLast();

Structured inputs

Structured inputs allow you to define named parameters on an agent that get substituted into its prompt template at runtime. This is useful when you want the same agent definition to handle different users or contexts by simply changing the input values.

Define structured inputs on an agent

When creating the agent, declare each structured input with a description and whether it is required. Use {{inputName}} placeholders in the instructions to reference them:

// Create an agent with structured input definitions
Map<String, StructuredInputDefinition> structuredInputDefinitions = new LinkedHashMap<>();
structuredInputDefinitions.put("userName",
    new StructuredInputDefinition().setDescription("User's name").setRequired(true));
structuredInputDefinitions.put("userRole",
    new StructuredInputDefinition().setDescription("User's role").setRequired(true));

AgentVersionDetails agent = agentsClient.createAgentVersion("structured-input-agent",
    new PromptAgentDefinition(model)
        .setInstructions("You are a helpful assistant. "
            + "The user's name is {{userName}} and their role is {{userRole}}. "
            + "Greet them and confirm their details.")
        .setStructuredInputs(structuredInputDefinitions));

Create a response with structured input values

When creating a response, pass a Map<String, BinaryData> whose keys match the structured input names declared on the agent. The values are substituted into the prompt template before the model processes the request:

// Build the structured input values that match the agent's definitions
Map<String, BinaryData> structuredInputValues = new LinkedHashMap<>();
structuredInputValues.put("userName", BinaryData.fromObject("Alice Smith"));
structuredInputValues.put("userRole", BinaryData.fromObject("Senior Developer"));

// Create a response using AzureCreateResponse, which flattens agent_reference
// and structured_inputs as top-level properties in the request body
Response response = responsesClient.createAzureResponse(
    new AzureCreateResponseOptions()
        .setAgentReference(new AgentReference(agent.getName()).setVersion(agent.getVersion()))
        .setStructuredInputs(structuredInputValues),
    ResponseCreateParams.builder().input("Hello! Can you confirm my details?")
);

Streaming is also supported via createStreamingAzureResponse, which returns an IterableStream<ResponseStreamEvent> (sync) or Flux<ResponseStreamEvent> (async).

See the full sample in CreateResponseWithStructuredInput.java.

Voice agent samples (preview)

The following voice-agent samples cover agent management and persisted conversations.

Authenticate with DefaultAzureCredential. Every voice sample requires FOUNDRY_PROJECT_ENDPOINT. Samples that create explicit definitions optionally use FOUNDRY_VOICE_MODEL, FOUNDRY_VOICE_MODEL_TYPE, and FOUNDRY_VOICE_AGENT_NAME. The persisted-conversation samples require FOUNDRY_VOICE_AGENT_NAME and FOUNDRY_VOICE_CONVERSATION_ID.

Realtime voice-agent WebSocket examples (preview)

Realtime WebSocket sessions provide bidirectional text and audio communication with a voice agent. Create the voice agent before opening a session; the lifecycle samples above demonstrate how to create one.

Create a realtime WebSocket client

Build a synchronous or asynchronous preview client from the same AgentsClientBuilder. Beta clients automatically send the required preview feature header.

AgentsClientBuilder builder = new AgentsClientBuilder()
    .credential(new DefaultAzureCredentialBuilder().build())
    .endpoint(endpoint);

BetaVoiceAgentWebSocketClient realtimeClient
    = builder.beta().buildBetaVoiceAgentWebSocketClient();
BetaVoiceAgentWebSocketAsyncClient realtimeAsyncClient
    = builder.beta().buildBetaVoiceAgentWebSocketAsyncClient();

Send a synchronous text turn

Connections require an https:// or wss:// project endpoint. Insecure endpoints are rejected before acquiring a token. This also applies to localhost; use certificate-verified TLS for local servers.

Unknown server event types are returned as RawRealtimeServerEvent; getRawEvent() preserves the complete JSON object. Use sendEvent(BinaryData) to send raw JSON objects, including event types or fields not modeled by this SDK. Sessions receive UTF-8 JSON in text or binary WebSocket messages.

VoiceAgentWebSocketConnectionOptions is copied when a connection is opened. Configure it before connecting; later changes do not affect the active session:

• setReceiveBufferCapacity sets a bounded event queue (default 256, range 1-65536).
• setOverflowStrategy defaults to ERROR, which closes an overflowing connection. DROP_OLDEST and DROP_LATEST explicitly permit data loss and should only be used when the application can tolerate missing events.
• setMaxMessageSize limits accepted message bytes (default 32 MiB). Oversized messages terminate the connection. The sync transport checks size after receiving a complete message; this does not bound the transport's allocation.
• Malformed JSON or invalid UTF-8 terminates reception by default. Set setMalformedEventHandler to report and skip malformed events while continuing reception. This callback must not block; throwing from it terminates the session.

VoiceAgentWebSocketConnectionOptions options
    = new VoiceAgentWebSocketConnectionOptions()
        .setReceiveBufferCapacity(512)
        .setMaxMessageSize(8 * 1024 * 1024)
        .setOverflowStrategy(VoiceAgentWebSocketOverflowStrategy.ERROR);
try (BetaVoiceAgentWebSocketSessionClient session = realtimeClient.openWebSocketSession(agentName, options)) {
    session.sendEvent(BinaryData.fromString(
        "{\"type\":\"response.create\",\"event_id\":\"response-1\"}"));
    for (RealtimeServerEvent event : session.receiveEvents()) {
        if (event instanceof RawRealtimeServerEvent) {
            BinaryData payload
                = ((RawRealtimeServerEvent) event).getRawEvent();
            System.out.println("Received an unrecognized event with " + payload.getLength() + " bytes.");
        }
    }
}

Connect to the voice agent, add the user's text to the conversation, and request a response. Consume the typed server events until the response finishes. A session supports only one consumer of receiveEvents().

For bounded synchronous waits, use receiveEvents(Duration) with a positive per-event timeout. A timeout raises IllegalStateException with a TimeoutException cause, leaves the session open, and allows the same iterator to retry. Use close(code, reason) or asynchronous closeAsync(code, reason) to send a custom close frame. Close reasons must fit in 123 UTF-8 bytes and close codes must be valid WebSocket codes. The first asynchronous close request wins.

try (BetaVoiceAgentWebSocketSessionClient session = realtimeClient.openWebSocketSession(agentName)) {
    session.sendText("Hello! Tell me about the services you provide.");
    session.createResponse();

    for (RealtimeServerEvent event : session.receiveEvents()) {
        if (event instanceof RealtimeResponseTextDeltaEvent) {
            System.out.print(((RealtimeResponseTextDeltaEvent) event).getDelta());
        } else if (event instanceof RealtimeErrorEvent) {
            RealtimeErrorEvent error = (RealtimeErrorEvent) event;
            System.out.println("Session error: " + error.getError().message());
        } else if (event instanceof RealtimeResponseDoneEvent) {
            break;
        }
    }
}

Use sendText and createResponse again for subsequent turns while the session remains open. Call cancelResponse to interrupt an active response.

Send an asynchronous text turn

The asynchronous client returns a Mono when connecting and a Flux<RealtimeServerEvent> when receiving events. Mono.usingWhen closes the session on completion, error, or cancellation.

Mono.usingWhen(
    realtimeAsyncClient.openWebSocketSession(agentName),
    session -> session.sendText("Hello! Tell me about the services you provide.")
        .then(session.createResponse())
        .thenMany(session.receiveEvents())
        .doOnNext(event -> {
            if (event instanceof RealtimeResponseTextDeltaEvent) {
                System.out.print(((RealtimeResponseTextDeltaEvent) event).getDelta());
            }
        })
        .takeUntil(event -> event instanceof RealtimeResponseDoneEvent)
        .then(),
    BetaVoiceAgentWebSocketSessionAsyncClient::closeAsync,
    (session, error) -> session.closeAsync(),
    BetaVoiceAgentWebSocketSessionAsyncClient::closeAsync)
    .block();

Stream audio and handle function tools

Use appendInputAudio to send bytes in the input format configured on the voice agent (the included live audio sample uses PCM16). Use commitInputAudio to commit buffered audio when server-side voice activity detection is not configured, and clearInputAudio to discard pending input. Audio output arrives through RealtimeResponseAudioDeltaEvent events. When a RealtimeResponseFunctionCallArgumentsDoneEvent requests a client-side tool, execute the function and call sendFunctionCallOutput with its call ID and serialized result; the helper also requests the next response.

All realtime examples require FOUNDRY_PROJECT_ENDPOINT and optionally use FOUNDRY_VOICE_AGENT_NAME. The function-tool example also optionally uses FOUNDRY_VOICE_MODEL and FOUNDRY_VOICE_MODEL_TYPE. The asynchronous text and audio examples delete their generated agents by default; set FOUNDRY_KEEP_VOICE_AGENT=true to retain them.

The live audio example requires a Java Sound-compatible microphone and speaker. It streams signed, little-endian, mono PCM16 audio at 24 kHz. These examples use WebSocket transport. Although the generated protocol models include WebRTC signaling events, the Java client does not provide a WebRTC peer connection or media implementation.

Service API versions

The client library targets the latest service API version by default. The service client builder accepts an optional service API version parameter to specify which API version to communicate.

Select a service API version

You have the flexibility to explicitly select a supported service API version when initializing a service client via the service client builder. This ensures that the client can communicate with services using the specified API version.

When selecting an API version, it is important to verify that there are no breaking changes compared to the latest API version. If there are significant differences, API calls may fail due to incompatibility.

Always ensure that the chosen API version is fully supported and operational for your specific use case and that it aligns with the service's versioning policy.

Troubleshooting

Enable client logging

You can set the AZURE_LOG_LEVEL environment variable to view logging statements made in the client library. For example, setting AZURE_LOG_LEVEL=2 would show all informational, warning, and error log messages. The log levels can be found here: log levels.

To log full HTTP request and response bodies (including headers), set:

export AZURE_LOG_LEVEL=verbose
export AZURE_HTTP_LOG_DETAIL_LEVEL=body_and_headers

Default HTTP Client

All client libraries by default use the Netty HTTP client. Configuring or changing the HTTP client is detailed in the HTTP clients wiki.

Default SSL library

All client libraries, by default, use the Tomcat-native Boring SSL library to enable native-level performance for SSL operations. The Boring SSL library is an uber jar containing native libraries for Linux / macOS / Windows, and provides better performance compared to the default SSL implementation within the JDK. For more information, including how to reduce the dependency size, refer to the performance tuning section of the wiki.

Next steps

Contributing

For details on contributing to this repository, see the contributing guide.

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request