Cloud Best Practices for AI Agents (AWS, GCP, Azure)

1. Compute Sandboxing & Code Execution

Isolating dynamically generated agent code using virtualized execution layers and microVM boundaries.

Runtime Security

AWS MicroVM Isolation

ECS Fargate & Firecracker

Run agent loops inside ECS Fargate tasks backed by Firecracker microVMs. Each task runs in its own lightweight VM with a dedicated kernel, providing hardware-level isolation without the overhead of traditional VMs.

Set readonlyRootFilesystem: true to prevent persistent writes to the container image.
Mount ephemeral tmpfs volumes for scratch files that auto-clear on task termination.
Enforce task memory ceilings and CPU limits to prevent resource starvation across the cluster.
Use AWS Lambda for lightweight, stateless tool execution (e.g., web search, calculator).

                        # ECS Task Definition (excerpt)
                        "containerDefinitions": [{

                          "readonlyRootFilesystem": true,

                          "memory": 4096,

                          "cpu": 2048,

                          "mountPoints": [{ "containerPath": "/tmp",

                            "sourceVolume": "ephemeral-scratch" }]

                        }]

Google Cloud Syscall Interception

Cloud Run & gVisor

Cloud Run isolates every container instance using gVisor, a user-space kernel that intercepts and filters all system calls before they reach the host kernel. This prevents container escape attacks even if the agent code is compromised.

Set --max-instances to cap horizontal scaling and prevent runaway cost from recursive agent loops.
Use --concurrency to control how many requests a single instance handles simultaneously.
For heavy orchestration workloads, deploy on GKE Autopilot with per-pod resource quotas.

                        # Cloud Run deploy with limits
                        gcloud run deploy agent-svc \

                          --max-instances=10 \

                          --concurrency=80 \

                          --cpu=2 --memory=4Gi \

                          --no-allow-unauthenticated

Azure KEDA Autoscaling

Container Apps & Dynamic Sessions

Host agent runtimes on Azure Container Apps (ACA) with Hyper-V isolation. ACA provides native Dynamic Sessions — fully isolated sandbox environments specifically designed for running AI-generated code safely.

Configure KEDA scaling rules to scale-to-zero when no queue messages are pending.
Use AKS (Azure Kubernetes Service) for large-scale agent swarm orchestration with custom node pools.
Enable VNet integration so all outbound traffic routes through your private network.

                        # KEDA Scale-to-Zero Rule
                        scaleRules: [{

                          name: "queue-trigger",

                          azureQueue: {

                            queueName: "agent-jobs",

                            queueLength: 1

                          }

                        }]

2. IAM & Identity Management

Enforcing least-privilege access policies for autonomous agents that operate at machine speed.

Zero Trust

AWS IAM Policies

Scoped IAM & MCP Abstraction

Agents operate autonomously — they will execute any tool they have permission to access. Use MCP (Model Context Protocol) servers as a security abstraction layer between the agent and AWS resources for centralized authorization and logging via CloudTrail.

Scope IAM policies to specific resource ARNs — never use Resource: "*" for agent roles.
Attach Permission Boundaries to prevent privilege escalation even if an agent creates new IAM entities.
Use STS AssumeRole with session policies for time-limited, scoped credentials per agent task.
Enable AWS CloudTrail with data events to audit every API call the agent makes.

                        # Scoped IAM Policy for Agent Role
                        {

                          "Effect": "Allow",

                          "Action": [

                            "bedrock:InvokeModel",

                            "s3:GetObject"

                          ],

                          "Resource": [

                            "arn:aws:bedrock:*::foundation-model/anthropic.claude*",

                            "arn:aws:s3:::agent-knowledge-base/*"

                          ]

                        }

Google Cloud Workload Identity

Dedicated Service Accounts & WIF

Create a dedicated Service Account per agent or tool function. Never reuse identities with broad permissions. Use Workload Identity Federation (WIF) to issue short-lived tokens instead of long-lived service account keys.

Assign the most granular IAM roles possible (e.g., roles/aiplatform.user not roles/editor).
Combine IAM with database-native access controls (e.g., Row-Level Security in Cloud SQL).
Apply Organization Policies at the folder/project level before production deployment.
Separate Security Admin duties from Developer roles using IAM Conditions.

                        # Create dedicated agent SA
                        gcloud iam service-accounts create agent-tool-sa \

                          --display-name="Agent Tool Runner"

                        # Bind minimal role
                        gcloud projects add-iam-policy-binding $PROJECT \

                          --member="serviceAccount:agent-tool-sa@$PROJECT.iam.gserviceaccount.com" \

                          --role="roles/aiplatform.user"

Azure Entra ID

Managed Identities & RBAC

Use Microsoft Entra ID (formerly Azure AD) Managed Identities for all service-to-service communication. This eliminates connection strings and hardcoded secrets entirely from your agent codebase.

Use system-assigned Managed Identities on Container Apps and AKS pods.
Assign granular RBAC roles: Cognitive Services OpenAI User for model access, Cosmos DB Data Reader for data plane access.
Enable Conditional Access Policies to restrict agent identity usage to specific network locations.
Use the AI Landing Zone accelerator for centralized governance across model, policy, and MCP catalogs.

                        # Assign RBAC role to Managed Identity
                        az role assignment create \

                          --role "Cognitive Services OpenAI User" \

                          --assignee-object-id <managed-identity-oid> \

                          --scope /subscriptions/.../resourceGroups/.../providers/Microsoft.CognitiveServices/accounts/my-openai

3. Network Perimeters & Private Model Access

Routing all prompt traffic privately to prevent data exfiltration and eliminate public internet exposure.

Network Isolation

AWS PrivateLink

VPC Endpoints & Zero-Trust Networking

Never route LLM requests over public IP subnets. Create VPC Interface Endpoints powered by AWS PrivateLink for Bedrock, S3, and Secrets Manager.

Enable Private DNS on VPC endpoints so SDK calls automatically resolve to private IPs.
Attach VPC Endpoint Policies to restrict which models/actions are accessible through the endpoint.
Enforce mutual TLS (mTLS) for bidirectional authentication between agents and MCP servers.
Use Security Groups as stateful firewalls on every endpoint interface.

                        # Create Bedrock VPC Endpoint
                        aws ec2 create-vpc-endpoint \

                          --vpc-id vpc-abc12345 \

                          --service-name com.amazonaws.us-east-1.bedrock-runtime \

                          --vpc-endpoint-type Interface \

                          --private-dns-enabled

Google Cloud VPC-SC & PSC

VPC Service Controls & Private Service Connect

Construct a VPC Service Controls (VPC-SC) perimeter enclosing Vertex AI, Cloud Storage, and BigQuery. Use Private Service Connect (PSC) to route all traffic through Google's private backbone.

VPC-SC blocks API calls from outside the perimeter boundary, even with valid credentials.
PSC creates private endpoints within your VPC — no public IP addresses are allocated.
Restrict egress with Firewall Rules to prevent agents from exfiltrating data to unauthorized destinations.
Use Cloud NAT only for explicitly allowlisted external API calls (e.g., third-party tool APIs).

                        # VPC-SC Perimeter Config (YAML)
                        resources:

                          - projects/ai-agent-prod

                        restrictedServices:

                          - aiplatform.googleapis.com

                          - storage.googleapis.com

                          - bigquery.googleapis.com

Azure Private Endpoints

VNet Integration & NSGs

Disable public network access on all Azure AI resources. Map services into your private Virtual Network using Private Endpoints with Private DNS Zones for automatic hostname resolution.

Create Private Endpoints for Azure OpenAI, Cosmos DB, AI Search, and Key Vault.
Integrate Private DNS Zones (e.g., privatelink.openai.azure.com) for seamless SDK compatibility.
Apply Network Security Groups (NSGs) to prevent lateral movement within the VNet.
Configure VNet integration on Container Apps for secure outbound traffic routing.

                        # ARM Template (excerpt)
                        "publicNetworkAccess": "Disabled",

                        "privateEndpointConnections": [{

                          "name": "pe-openai-prod",

                          "properties": {

                            "privateLinkServiceId": "[resourceId(...)]"

                          }

                        }]

4. Database & Vector Storage

Choosing the right persistence layer for agent memory, RAG knowledge bases, and session state.

Data Layer

AWS DynamoDB & Aurora

Tiered Agent Memory

Use a tiered approach: DynamoDB for ultra-low-latency session state and key-value lookups, Aurora PostgreSQL with pgvector for semantic search and RAG knowledge bases.

DynamoDB: Single-digit millisecond latency for agent session state, conversation history, and tool execution logs.
Aurora PostgreSQL: Full SQL support plus pgvector extension for vector similarity search in the same transaction as structured queries.
S3 + S3 Vectors: Cost-optimized knowledge bases for large-scale document corpora used in RAG pipelines.
Build a semantic ontology using graph stores to help agents navigate complex enterprise data.

                        # Aurora pgvector similarity search
                        SELECT id, content,

                          embedding <=> $query_vector AS distance

                        FROM knowledge_chunks

                        ORDER BY distance LIMIT 5;

Google Cloud AlloyDB & Cloud SQL

AlloyDB AI with ScaNN

AlloyDB is Google's premium AI-native database featuring ScaNN (Scalable Nearest Neighbors) — a proprietary indexing technology that significantly outperforms standard pgvector indexes in speed and accuracy for large-scale datasets.

AlloyDB AI: Built-in embedding generation + ScaNN indexes for high-performance RAG at scale.
Cloud SQL: Standard managed PostgreSQL with pgvector (HNSW + IVFFlat) for simpler workloads.
Use Tools for Data Agents to allow agents to execute text-to-SQL queries securely via MCP Toolbox.
Apply Row-Level Security (RLS) policies to enforce multi-tenant data isolation at the database level.

                        # AlloyDB ScaNN index creation
                        CREATE INDEX ON knowledge_chunks

                          USING scann (embedding vector_cosine_ops)

                          WITH (num_leaves = 100);

Azure Cosmos DB & Redis

Cosmos DB Unified AI Database

Azure Cosmos DB has evolved into a "Unified AI Database" with native vector indexing, combining document, key-value, graph, and vector models in a single globally distributed service.

Cosmos DB: Native vector search with global replication — ideal for planet-scale agent memory and RAG.
Azure Managed Redis: High-speed transient session memory for conversation history and short-term context.
Use partition keys based on tenantId for strict multi-tenant data separation.
Access data via the Cosmos DB MCP Toolkit for natural language querying from agents.

                        # Cosmos DB vector search query
                        SELECT TOP 5 c.id, c.content,

                          VectorDistance(c.embedding, @queryVector)

                          AS score

                        FROM knowledge c

                        ORDER BY VectorDistance(c.embedding, @queryVector)

5. Envelope Encryption & Key Management

Securing model API tokens, database credentials, and third-party integrations in production key stores.

Secrets Protection

AWS KMS + Secrets Manager

KMS Envelope Encryption

Store all third-party API keys in AWS Secrets Manager encrypted with Customer Managed Keys (CMK) in KMS. Enable automatic rotation and restrict decryption to the agent's Fargate Task Role only.

Never store credentials as environment variables or in git repositories.
Bind kms:Decrypt permissions exclusively to the agent task execution role.
Enable automatic secret rotation with Lambda rotation functions.

                        # IAM Policy for Secrets Access
                        {

                          "Effect": "Allow",

                          "Action": "secretsmanager:GetSecretValue",

                          "Resource": "arn:aws:secretsmanager:*:*:secret:agent-keys-*",

                          "Condition": {

                            "StringEquals": { "kms:ViaService": "secretsmanager.*.amazonaws.com" }

                          }

                        }

Google Cloud Cloud KMS + Secret Manager

Secret Manager & CMEK

Use GCP Secret Manager with Customer-Managed Encryption Keys (CMEK) via Cloud KMS. Bind the secretAccessor role directly to the compute Service Account — disable broad viewer permissions.

Inject secrets into Cloud Run containers at startup — never bake them into images.
Configure secret version rotation and pin to latest version aliases.
Audit access logs via Cloud Audit Logs to detect unauthorized retrieval attempts.

                        # Bind Secret Accessor to Service Account
                        gcloud secrets add-iam-policy-binding agent-keys \

                          --member="serviceAccount:agent-sa@$PROJECT.iam.gserviceaccount.com" \

                          --role="roles/secretmanager.secretAccessor"

Azure Key Vault + Managed Identity

Key Vault with RBAC Authorization

Deploy Azure Key Vault with RBAC authorization model. Bind Key Vault Secrets User permissions solely to the system-assigned Managed Identity on your compute — no connection strings, no hardcoded keys.

Enable CMK (Customer-Managed Keys) for sensitive data encryption at rest.
Configure key rotation policies with automatic notification alerts.
Use Key Vault references in Container Apps environment variables for seamless injection.

                        # Assign Key Vault role to Managed Identity
                        az role assignment create \

                          --role "Key Vault Secrets User" \

                          --assignee-object-id <managed-identity-oid> \

                          --scope /subscriptions/.../Microsoft.KeyVault/vaults/agent-vault

6. Guardrails, Prompt Defense & Content Safety

Intercepting prompt injection attacks, PII leakage, and validating model outputs before execution commits.

Safety & Compliance

AWS Bedrock Guardrails

Bedrock Guardrails & AgentCore Policies

Configure Amazon Bedrock Guardrails to scan both inputs and outputs. Layer with AgentCore Policies to govern which specific tools and actions an agent is permitted to take at runtime.

Redact PII automatically (email, phone, SSN, SSH keys) from prompts and responses.
Configure custom word/regex filters to detect prompt injection patterns (e.g., "ignore previous instructions").
Set toxicity thresholds for categories like hate speech, violence, and self-harm.
Enable guardrail tracing to log every intervention for audit review.

                        # Python Bedrock invocation with guardrails
                        response = bedrock.invoke_model(

                          modelId="anthropic.claude-sonnet-4-20250514",

                          guardrailConfig={

                            "guardrailIdentifier": "gd-prod-safety",

                            "guardrailVersion": "1",

                            "trace": "enabled"

                          }

                        )

Google Cloud Safety Sidecar

LlamaGuard Sidecar & Vertex Safety

Deploy a LlamaGuard model as a sidecar container within your Cloud Run service or GKE pod. Every incoming prompt is analyzed for harmful intent before being forwarded to the primary model.

Sub-millisecond local policy evaluations — no external network call required.
Configure Vertex AI safety settings to block harmful categories at the API level.
Validate outbound structured payloads (JSON, SQL) against schemas before execution.
Use grounding checks to detect and flag hallucinated citations.

                        # Vertex AI Safety Settings
                        safety_settings = [

                          SafetySetting(

                            category=HarmCategory.HARASSMENT,

                            threshold=HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE

                          )

                        ]

Azure Content Safety API

Azure AI Content Safety & Prompt Shields

Enable Azure AI Content Safety to scan both user prompts and model outputs in real-time. The service includes dedicated Prompt Shield capabilities that specifically detect and block jailbreak and injection attacks.

Classify content across categories: Hate, SelfHarm, Violence, Sexual.
Prompt Shields detect both direct and indirect prompt injection attacks.
Use Groundedness Detection to verify model outputs against source documents.
Integrate with Microsoft Defender for Cloud for runtime threat detection.

                        # Content Safety with Prompt Shields
                        result = client.analyze_text(

                          text=user_prompt,

                          categories=["Hate","SelfHarm","Violence"],

                          output_type="EightSeverityLevels"

                        )

7. Observability, Tracing & Cost Attribution

Treating AI agents as distributed systems — capturing reasoning paths, tool executions, token costs, and hallucination rates.

OpenTelemetry

AWS CloudWatch & X-Ray

CloudWatch + OTel Integration

Stream agent reasoning traces to Amazon CloudWatch Logs. Use the AWS Distro for OpenTelemetry (ADOT) to instrument agent code with the OTel GenAI Semantic Conventions for standardized telemetry.

Set up Subscription Filters to trigger Lambda alerts on injection pattern keywords.
Use CloudWatch Metrics to track token consumption, latency percentiles, and error rates per model.
Enable X-Ray tracing for end-to-end distributed trace visualization across agent tool calls.
Use the FinOps Agent to automatically monitor and optimize agent infrastructure costs.

                        # CloudWatch Metric Filter
                        aws logs put-metric-filter \

                          --log-group-name /agent/reasoning \

                          --filter-name "PromptInjectionDetector" \

                          --filter-pattern '"ignore previous instructions"' \

                          --metric-transformations metricName=InjectionAttempts,metricValue=1

Google Cloud Cloud Ops Suite

Cloud Logging, Trace & Monitoring

Use the integrated Google Cloud Operations Suite: Cloud Logging for structured logs, Cloud Trace for distributed tracing, and Cloud Monitoring for dashboards and alerting — all with native OTel collector support.

Export structured JSON logs with agent metadata (model version, prompt template hash, user segment).
Create log-based metrics to track 429 rate-limit errors and alert on agent loop exhaustion.
Build Cloud Monitoring dashboards tracking token throughput, latency, and cost per agent workflow.
Monitor for configuration drift using Security Command Center findings.

                        # Log-based metric for rate limits
                        gcloud logging metrics create agent_rate_limits \

                          --description="Track 429 errors from Vertex AI" \

                          --log-filter='resource.type="cloud_run_revision" AND httpRequest.status=429'

Azure Azure Monitor

Azure Monitor & Application Insights

Azure Monitor now treats AI agents as first-class artifacts, offering built-in views for agent fleet monitoring, cost breakdowns, and human-in-the-loop evaluation metrics — all powered by OTel semantic conventions.

Enable Diagnostic Settings on Azure OpenAI and Container Apps to stream to Log Analytics.
Use Application Insights with OTel SDK for step-level agent trace visualization.
Create KQL queries to analyze token consumption, model latency, and tool execution patterns.
Bind to Microsoft Defender for Cloud for real-time security posture assessment.

                        # KQL query for agent token usage
                        AppTraces

                        | where Properties.ai_model != ""

                        | summarize TotalTokens = sum(toint(Properties.token_count)),

                          AvgLatency = avg(DurationMs)

                        | by bin(TimeGenerated, 1h), Properties.ai_model

8. Cloud Services Comparison Matrix

Side-by-side mapping of equivalent services across AWS, GCP, and Azure for AI agent infrastructure.

Reference Guide

Category	AWS	Google Cloud	Azure
🖥️ Container Compute	ECS Fargate (Firecracker)	Cloud Run (gVisor)	Container Apps (Hyper-V)
⚡ Serverless Functions	Lambda	Cloud Functions	Azure Functions
☸️ Kubernetes	EKS	GKE Autopilot	AKS
🤖 AI Model API	Bedrock	Vertex AI / Gemini	Azure OpenAI Service
🛡️ Agent Platform	Bedrock AgentCore	Agent Engine (ADK)	Azure AI Foundry
🔐 IAM System	IAM Policies + STS	Cloud IAM + WIF	Entra ID + Managed Identity
🌐 Private Networking	PrivateLink + VPC Endpoints	Private Service Connect + VPC-SC	Private Endpoints + VNet
🗄️ Relational DB	Aurora PostgreSQL	AlloyDB / Cloud SQL	Azure SQL / PostgreSQL Flex
📊 NoSQL / Document	DynamoDB	Firestore / Bigtable	Cosmos DB
🔍 Vector Search	Aurora pgvector / OpenSearch	AlloyDB ScaNN / pgvector	Cosmos DB Native Vectors
⚡ Session Cache	ElastiCache (Redis)	Memorystore (Redis)	Azure Managed Redis
🔑 Secrets Store	Secrets Manager + KMS	Secret Manager + Cloud KMS	Key Vault + CMK
🛡️ Content Safety	Bedrock Guardrails	Vertex Safety + LlamaGuard	Azure AI Content Safety
📡 Logging	CloudWatch Logs	Cloud Logging	Azure Monitor / Log Analytics
📈 Tracing	X-Ray + ADOT	Cloud Trace	Application Insights
🔒 Security Posture	Security Hub + GuardDuty	Security Command Center	Microsoft Defender for Cloud
📋 Audit Trail	CloudTrail	Cloud Audit Logs	Azure Activity Log
📦 Object Storage	S3 + S3 Vectors	Cloud Storage	Blob Storage
🔎 Search	OpenSearch	Vertex AI Search	Azure AI Search
📬 Message Queue	SQS / EventBridge	Pub/Sub / Cloud Tasks	Service Bus / Event Grid

Cloud Deployment Architecture for AI Agents

Compute

IAM

Networking

Database

Secrets

Guardrails

Observability

Comparison