Cloud Best Practices for AI Agents (AWS, GCP, Azure)

Pillar 1: Compute Sandboxing & Code Execution

Isolating dynamically generated code loops using virtualized execution layers.

Runtime Security

AWS MicroVM Isolation

Fargate Read-Only Tasks

Run agent loops inside AWS ECS Fargate. Fargate tasks run in virtualized micro-VM partitions using **AWS Firecracker** under the hood, providing strict kernel-level boundaries.

Set `readonlyRootFilesystem` to `true` inside task definitions.
Mount temporary storage via `tmpfs` for ephemeral files.
Enforce task memory allocation ceilings to prevent CPU starvation.

# Fargate Storage Rule "readonlyRootFilesystem": true,
"mountPoints": [
{ "containerPath": "/tmp", "sourceVolume": "temp-vol" }
]

Google Cloud Syscall Interception

Cloud Run & gVisor

Deploy agent runtimes as containers in GCP Cloud Run. Cloud Run automatically isolates containers using **gVisor**, a user-space kernel that intercepts and filters system calls.

Intercepts raw calls to block access to host resources.
Enforce max concurrency scaling to avoid running out of memory.
Limit memory footprint by setting hard container request scopes.

# Cloud Run Limits Config gcloud run deploy agent-svc \
  --max-instances=10 \
  --concurrency=80 \
  --cpu=2 --memory=4Gi

Azure KEDA Dynamic Scale

Azure Container Apps

Host Python agent workers on Azure Container Apps (ACA) in dedicated Environments, isolating the task behind virtualization borders.

Isolates the container inside a hypervisor boundary.
Configure KEDA scaling rules to shut down instances when queue triggers are empty.
Define container limits using Azure Resource Manager configurations.

# KEDA Scale to Zero rule type: "azure-queue"
metadata: {
"queueName": "agent-jobs",
"queueLength": "1"
}

Pillar 2: Network Perimeters & Private Model Access

Routing prompt payloads privately to prevent exfiltration and internet sniffing.

Network Isolation

AWS VPC Endpoint

Bedrock & PrivateLink

Do not route LLM requests over public IP subnets. Create a VPC Interface Endpoint (powered by **AWS PrivateLink**) for Bedrock.

Direct all API traffic through the endpoint’s local private IP addresses.
Enforce DNS resolution inside the VPC to override standard API routes automatically.

# CLI create VPC Endpoint aws ec2 create-vpc-endpoint \
--vpc-id vpc-abc12345 \
--service-name com.amazonaws.us-east-1.bedrock

Google Cloud Service Perimeter

Vertex AI VPC-SC

Construct a **VPC Service Controls (VPC-SC)** security perimeter around your projects. Wrap the Vertex AI API inside the perimeter.

Blocks API calls coming from outside the perimeter boundary.
Restricts data export routes to target storage resources.

# VPC-SC configuration resources:
- projects/ai-agent-project-id
restrictedServices:
- aiplatform.googleapis.com

Azure VNet Integration

Azure OpenAI Private Link

Disable all public network access on the Azure OpenAI resource. Map the service into the private Virtual Network subnet.

Create a Private Endpoint linking Azure OpenAI.
Integrate Private DNS Zones for direct hostname resolution (`*.openai.azure.com`).

# ARM Template parameter "publicNetworkAccess": "Disabled",
"privateEndpointConnections": [
{ "name": "pe-openai" }
]

Pillar 3: Envelope Encryption & Key Management

Securing model API tokens, databases, and client credentials in production key stores.

Secrets Protection

AWS IAM & KMS policy

Secrets Manager & KMS

Encrypt agent variables inside AWS Secrets Manager. Restrict access using strict IAM policies that require a specific KMS Customer Managed Key (CMK) for decryption.

# IAM Policy snippet {
  "Effect": "Allow",
  "Action": "secretsmanager:GetSecretValue",
  "Resource": "arn:aws:secretsmanager:*:*:secret:agent-keys-*"
}

Google Cloud Access Binding

Secret Manager & IAM

Bind the `secretAccessor` role directly to the Compute or Cloud Run Service Account. Disable broad service-wide viewer permissions on the secret resource.

# Grant Service Account Access gcloud secrets add-iam-policy-binding agent-keys \
--member="serviceAccount:agent-sa@project.iam.gserviceaccount.com" \
--role="roles/secretmanager.secretAccessor"

Azure Managed Identity

Key Vault & Identities

Deploy Azure Key Vault with Role-Based Access Control (RBAC) authorization models. Bind Key Vault Secrets User permissions solely to system-assigned Managed Identities on Azure compute.

# Az Role Assignment az role assignment create \
  --role "Key Vault Secrets User" \
  --assignee-object-id <managed-identity-id> \
  --scope /subscriptions/.../providers/Microsoft.KeyVault/vaults/vault-name

Pillar 4: Guardrails, Prompt Defense & Auditing

Intercepting malicious input sequences and validating model outputs before execution commits.

Audit & Compliance

AWS Jailbreak Blocker

Bedrock Guardrails

Configure **Amazon Bedrock Guardrails** to intercept prompt inputs. Set thresholds for toxic classifications and configure custom regex patterns to detect and block prompt injection patterns.

Redact PII automatically (e.g. email, SSH keys).
Enforce strict filters to stop instructions override.

# Python Bedrock invocation config guardrailConfig={
  "guardrailIdentifier": "gd-id1",
  "guardrailVersion": "1",
  "trace": "enabled"
}

Google Cloud Sidecar Scan

LlamaGuard sidecar

Deploy a lightweight **LlamaGuard** container as a sidecar inside your Cloud Run service or GKE task pod. Run incoming prompts through this container to analyze user intent before passing requests to the model.

Performs sub-millisecond local policy evaluations.
Prevents unsafe outputs from executing system shell APIs.

# Sidecar connection validation resp = requests.post(
"http://localhost:8080/v1/chat/completions",
json={"prompt": user_input}
)

Azure Safety Endpoint

Azure Content Safety

Enable **Azure AI Content Safety** diagnostics. Route both incoming user questions and outgoing model JSON outputs to the validation API.

Intercepts prompt injection attacks dynamically.
Evaluates JSON structures to verify schema conformity.

# Content Safety request snippet client.analyze_text(
text="evaluate prompt content...",
categories=["Hate", "SelfHarm", "Violence"]
)

Cloud Deployment Architecture

1. Code Sandboxing

2. Network Perimeters

3. Key Protection

4. Input Guardrails