Microsoft Azure

Azure for AI Agents

The essential Azure services for building, deploying, and scaling AI agent systems. From Azure OpenAI Service for GPT & o-series models to Azure Functions for serverless compute, Cosmos DB for global-scale state, and Service Bus for enterprise messaging.

Concepts Services Setup Guide Usage Best Practices FAQ

Core Concepts

Foundational Azure concepts for AI agent infrastructure.

Subscriptions & Resource Groups

Azure organizes resources into Subscriptions (billing boundary) containing Resource Groups (logical containers). Group all agent resources — OpenAI, Functions, Cosmos DB — in one resource group for unified management and RBAC.

Entra ID & Managed Identity

Azure's identity platform (formerly Azure AD). Use Managed Identities for agent workloads — no secrets to manage. Supports RBAC with built-in roles like Cognitive Services User for OpenAI access.

Serverless & Containers

Azure Functions: Event-driven, pay-per-execution, up to 10 min (Consumption) or unlimited (Premium). Container Apps: Managed containers with auto-scale. AKS: Full Kubernetes for complex multi-agent systems.

Virtual Networks & Private Endpoints

Use Private Endpoints to access Azure OpenAI, Cosmos DB, and Storage without internet exposure. VNet Integration connects Functions and Container Apps to private networks. NSGs control traffic at the subnet level.

Azure OpenAI Service

Enterprise-grade access to OpenAI models (GPT-4o, o3, o4-mini) with content filtering, private networking, and data sovereignty. Deploy models to specific regions. Supports function calling and Assistants API for agents.

Enterprise Integration

Azure's strength is enterprise integration. Logic Apps: Low-code workflow automation with 400+ connectors. Event Grid: Event routing at scale. API Management: Secure, rate-limit, and monetize agent APIs.

Key Services for AI Agents

The essential Azure services powering modern AI agent architectures.

Service Category Description AI Agent Use Case
Azure OpenAIAI/MLManaged GPT-4o, o3, o4-mini with enterprise controlsAgent reasoning, function calling, Assistants API
Azure FunctionsComputeServerless functions, event-driven, multiple runtimesAgent tool execution, webhooks, triggers
Container AppsComputeManaged containers with Dapr, KEDA auto-scalingLong-running agents, microservices, APIs
AKSComputeManaged Kubernetes with GPU node poolsMulti-agent clusters, custom model hosting
Cosmos DBDatabaseGlobal-scale NoSQL with vector searchAgent state, session memory, vector embeddings
Service BusMessagingEnterprise messaging with queues & topicsTask distribution, async agent communication
Blob StorageStorageObject storage with tiered accessDocument storage, RAG knowledge bases, logs
Key VaultSecurityManaged secrets, keys, and certificatesAPI keys, connection strings, encryption keys
Azure MonitorObservabilityMetrics, logs, traces, Application InsightsAgent monitoring, cost alerts, performance
AI SearchSearchVector + keyword + semantic searchRAG retrieval, knowledge base search
Durable FunctionsOrchestrationStateful workflows in serverless functionsMulti-step agent pipelines, fan-out/fan-in

Setup Guide

Get your Azure development environment ready.

Azure CLI (Required)

# Install Azure CLI (macOS)
brew install azure-cli

# Login
az login

# Set subscription
az account set \
  --subscription "My AI Sub"

# Create resource group
az group create \
  --name ai-agent-rg \
  --location eastus

# Verify
az account show --output table

Python SDK (azure-identity)

# Install Azure SDKs
pip install openai
pip install azure-identity
pip install azure-cosmos
pip install azure-storage-blob

# Quick test with DefaultCredential
python3 -c "
  from azure.identity import \
    DefaultAzureCredential
  cred = DefaultAzureCredential()
  token = cred.get_token(
    'https://management.azure.com'
  )
  print('Auth OK')
"

OpenAI Resource Setup

# Create Azure OpenAI resource
az cognitiveservices account create \
  --name my-openai \
  --resource-group ai-agent-rg \
  --kind OpenAI \
  --sku S0 \
  --location eastus

# Deploy a model
az cognitiveservices account \
  deployment create \
  --name my-openai \
  --resource-group ai-agent-rg \
  --deployment-name gpt-4o \
  --model-name gpt-4o \
  --model-version "2024-11-20" \
  --model-format OpenAI \
  --sku-capacity 10 \
  --sku-name Standard

Usage Examples

Practical patterns for AI agent infrastructure on Azure.

Azure OpenAI — GPT Inference
# Invoke GPT-4o via Azure OpenAI
from openai import AzureOpenAI

client = AzureOpenAI(
  azure_endpoint="https://my-openai"
    ".openai.azure.com",
  api_key=os.getenv("AZURE_KEY"),
  api_version="2024-10-21"
)

response = client.chat.completions
  .create(
  model="gpt-4o",
  messages=[{
    "role": "user",
    "content": "Plan an agent"
    " architecture"
  }]
)

print(response.choices[0]
  .message.content)
Cosmos DB — Agent State
# Store and retrieve agent state
from azure.cosmos import CosmosClient

client = CosmosClient(url, credential)
db = client.get_database_client(
  "agent-db"
)
container = db.get_container_client(
  "sessions"
)

# Save agent session
container.upsert_item({
  "id": "sess_abc123",
  "agent_id": "research-agent",
  "state": "reasoning",
  "messages": [...],
  "ttl": 86400
})

# Query sessions
items = container.query_items(
  query="SELECT * FROM c"
    " WHERE c.state='active'"
)
Azure Functions — Tool Execution
# function_app.py (v2 model)
import azure.functions as func
import json

app = func.FunctionApp()

@app.route(route="agent/tool")
def execute_tool(req: func.HttpRequest):
  body = req.get_json()
  tool = body["tool_name"]
  params = body["parameters"]

  if tool == "search":
    result = search(params)
  elif tool == "read_blob":
    result = read_blob(params)

  return func.HttpResponse(
    json.dumps(result),
    mimetype="application/json"
  )
Service Bus — Agent Messaging
# Distribute tasks across agents
from azure.servicebus import \
  ServiceBusClient, ServiceBusMessage

client = ServiceBusClient
  .from_connection_string(conn_str)

# Send task to queue
with client.get_queue_sender(
  "agent-tasks"
) as sender:
  sender.send_messages(
    ServiceBusMessage(json.dumps({
      "task": "analyze_doc",
      "blob": "docs/report.pdf",
      "priority": "high"
    }))
  )

Best Practices

Production guidelines for Azure-based AI agent systems.

🔐 Security

  • • Use Managed Identity everywhere, never store keys
  • • Store secrets in Key Vault with RBAC access
  • • Enable Defender for Cloud for threat detection
  • • Use Private Endpoints for OpenAI and Cosmos DB
  • • Enable content filtering on Azure OpenAI deployments

💰 Cost Optimization

  • • Set Azure Budgets with action groups for alerts
  • • Use Provisioned Throughput for predictable OpenAI loads
  • • Choose Cosmos DB serverless for dev/low-traffic
  • • Use GPT-4o mini for high-volume, simple tasks
  • • Enable Blob lifecycle management for auto-tiering

📊 Observability

  • • Use Application Insights for distributed tracing
  • • Enable diagnostic settings on all AI resources
  • • Set alerts on OpenAI 429 throttling responses
  • • Log agent decisions with structured logging to Log Analytics
  • • Use Azure Workbooks for custom agent dashboards

Frequently Asked Questions

How is Azure OpenAI different from OpenAI's API?
Same models, different platform. Azure OpenAI adds enterprise features: Managed Identity auth, Private Endpoints, content filtering, data residency guarantees, and SLA-backed uptime. You deploy models to your Azure subscription with full control over region and capacity. Use Azure for production, OpenAI's API for prototyping.
Should I use Azure Functions or Container Apps?
Azure Functions for short-lived event-driven tasks (tool execution, queue processing, HTTP triggers). Container Apps for long-running agents, custom runtimes, WebSocket connections, and microservices. Container Apps supports Dapr for service-to-service communication and KEDA for scaling on any metric.
How do I keep costs under control with Azure OpenAI?
Use Provisioned Throughput Units (PTUs) for predictable pricing on high-volume workloads. Set token-per-minute rate limits per deployment. Use GPT-4o mini for simple tasks. Enable prompt caching. Monitor usage via Azure Monitor metrics and set budget alerts with action groups.
Cosmos DB or Azure SQL for agent memory?
Cosmos DB for agent sessions, global distribution, and flexible schema — supports NoSQL, MongoDB, and PostgreSQL APIs. Built-in vector search for RAG. Azure SQL for structured relational data, complex joins, and existing SQL workloads. Cosmos DB is the default choice for new agent architectures.
How do I deploy a multi-agent system on Azure?
Use Durable Functions for orchestrating agent workflows — supports fan-out/fan-in, human-in-the-loop, and eternal orchestrations. Each agent tool runs as a Function. Use Service Bus for async communication, Cosmos DB for shared state. For complex systems, use Azure Container Apps with Dapr for service mesh.
Which Azure region should I use for AI agents?
East US and East US 2 have the broadest Azure OpenAI model availability. Sweden Central is the primary EU region. West US 3 is another strong option. Always check the Azure OpenAI model availability matrix — not all models are available in all regions.
How do I set up a RAG pipeline on Azure?
Use Azure AI Search — it supports vector search, semantic ranking, and hybrid (keyword + vector) search in one service. Ingest documents from Blob Storage, embed with text-embedding-3-large via Azure OpenAI, and store in AI Search. Use the "On Your Data" feature to connect AI Search directly to Azure OpenAI for zero-code RAG.
What is the Assistants API and should I use it?
The Assistants API is OpenAI's managed agent runtime — it handles conversation state, tool execution, file retrieval, and code interpreter automatically. Available on Azure OpenAI. Great for rapid prototyping, but limits customization. For production agents, consider using the OpenAI SDK directly with your own orchestration for full control.