Kubernetes Tutorial — Concepts, Setup & Deployment

1 Core Concepts

Pods

The smallest deployable unit. A Pod wraps one or more containers that share networking (localhost) and storage volumes. Agent containers + sidecar tool containers run together in a single Pod.

Deployments

Declares the desired state: which container image, how many replicas, update strategy (rolling/recreate). The Deployment controller ensures reality matches the spec. Supports RollingUpdate with maxSurge and maxUnavailable.

Services

Stable network endpoint for a set of Pods. Types: ClusterIP (internal), NodePort (fixed port on nodes), LoadBalancer (cloud LB), ExternalName (DNS alias). Uses label selectors for routing.

Namespaces

Virtual clusters within a physical cluster. Isolate resources by team, environment (dev/staging/prod), or tenant. Apply ResourceQuotas and LimitRanges per namespace to prevent resource starvation.

ConfigMaps & Secrets

ConfigMaps inject non-sensitive config (env vars, config files) into Pods. Secrets store API keys, tokens, and certificates (base64-encoded, optionally encrypted at rest via KMS).

Ingress

HTTP/HTTPS routing layer that maps external hostnames and paths to internal Services. Supports TLS termination, path-based routing, and rate limiting. Common controllers: nginx-ingress, Traefik, Istio Gateway.

HPA (Autoscaling)

Horizontal Pod Autoscaler automatically scales replica count based on CPU, memory, or custom metrics (e.g., queue depth, request latency). Essential for bursty AI agent workloads.

Helm Charts

Package manager for Kubernetes. A Chart bundles all YAML manifests (Deployment, Service, Ingress, ConfigMap) into a versioned, parameterized template. values.yaml allows environment-specific overrides.

2 Architecture

Control Plane

kube-apiserver
REST gateway for all cluster operations

etcd
Distributed key-value store (cluster state)

kube-scheduler
Assigns Pods to Nodes based on constraints

controller-manager
Runs reconciliation loops (Deployment, ReplicaSet)

Worker Node

kubelet
Node agent — manages Pod lifecycle

kube-proxy
Network rules for Service routing (iptables/IPVS)

Container Runtime
containerd / CRI-O runs OCI containers

Pods
Your agent containers run here

Add-Ons

CoreDNS
Cluster-internal DNS resolution

Metrics Server
CPU/memory metrics for HPA

Ingress Controller
HTTP routing (nginx, Traefik)

GPU Device Plugin
NVIDIA GPU scheduling for ML workloads

3 Setup Guide

minikube (Local Development)

# Install minikube (macOS)
brew install minikube
# Start cluster (2 CPUs, 4GB RAM)
minikube start \
--cpus=2 --memory=4096 \
--driver=docker
# Enable addons
minikube addons enable metrics-server
minikube addons enable ingress
# Verify
kubectl get nodes
# NAME       STATUS   ROLES
# minikube   Ready    control-plane
# Open dashboard
minikube dashboard

kind (K8s in Docker)

# Install kind
brew install kind
# Create multi-node cluster
cat <<EOF | kind create cluster \
--config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
EOF
# Verify
kubectl get nodes
# 3 nodes (1 control, 2 workers)
# Install kubectl separately
brew install kubectl

4 Usage Examples

Deployment + Service YAML

# agent-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ai-agent
spec:
replicas: 3
selector:
matchLabels:
app: ai-agent
template:
metadata:
labels:
app: ai-agent
spec:
containers:
- name: agent
image: myregistry/ai-agent:v1.2
ports:
- containerPort: 8080
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "1"
memory: "1Gi"
envFrom:
- secretRef:
name: agent-api-keys

kubectl Essential Commands

# Apply manifests
kubectl apply -f agent-deployment.yaml
# Check rollout status
kubectl rollout status deploy/ai-agent
# Scale replicas
kubectl scale deploy/ai-agent --replicas=5
# View logs (follow)
kubectl logs -f deploy/ai-agent --tail=100
# Exec into running Pod
kubectl exec -it deploy/ai-agent -- /bin/sh
# Port-forward to localhost
kubectl port-forward svc/ai-agent 8080:80
# Create Secret from literal
kubectl create secret generic \
agent-api-keys \
--from-literal=OPENAI_KEY=sk-xxx
# HPA autoscaling
kubectl autoscale deploy/ai-agent \
--min=2 --max=10 --cpu-percent=70
# Rollback to previous version
kubectl rollout undo deploy/ai-agent

5 Best Practices

Resource Requests & Limits

Always set requests and limits for CPU and memory. Requests guarantee scheduling; limits prevent runaway containers. Set memory limit = 2× request for bursty agent workloads. Never omit requests — it causes unpredictable scheduling.

Liveness & Readiness Probes

Configure readinessProbe (traffic routing) and livenessProbe (restart on hang). Use HTTP GET on /healthz. Set initialDelaySeconds high enough for model loading (30–120s for LLM agents).

Image Tagging

Never use :latest in production. Use immutable tags like v1.2.3-sha-abc123. Set imagePullPolicy: IfNotPresent for tagged images. Use a private registry with vulnerability scanning.

Pod Disruption Budgets

Set PodDisruptionBudget with minAvailable: 50% to ensure cluster upgrades and node drains don't take down all replicas simultaneously. Critical for maintaining agent availability during maintenance.

Network Policies

Default-deny all ingress/egress traffic per namespace, then explicitly allow required flows. Prevents a compromised agent Pod from accessing unrelated services. Use Calico or Cilium as the CNI plugin.

GitOps with ArgoCD

Store all manifests in Git. Use ArgoCD or Flux to auto-sync cluster state from the repo. Enables audit trails, rollback via git revert, and multi-cluster promotion (dev → staging → prod).

6 Scaling & Production

Cluster Autoscaler

Automatically adds or removes worker nodes based on pending Pod requests. When HPA scales Pods beyond available cluster capacity, the Cluster Autoscaler provisions new nodes from the cloud provider's node pool within 1–3 minutes.

Vertical Pod Autoscaler

VPA adjusts CPU/memory requests and limits based on actual usage. Run in "recommend" mode first to observe suggestions, then "auto" to apply. Essential for right-sizing LLM agent containers that have unpredictable memory patterns.

KEDA (Event-Driven)

Kubernetes Event-Driven Autoscaling scales based on external metrics: Kafka consumer lag, Redis queue length, Prometheus queries. Can scale to zero Pods during idle periods — critical for cost-effective agent workloads.

Multi-Cluster Federation

Run agents across multiple clusters in different regions for latency and reliability. Use Submariner for cross-cluster networking or Liqo for seamless workload offloading. Manage with ArgoCD ApplicationSets for multi-cluster GitOps.

GPU Node Pools

Create dedicated GPU node pools with taints (nvidia.com/gpu=true:NoSchedule) so only GPU workloads land there. Use GPU time-slicing or MIG (Multi-Instance GPU) to share A100s across multiple agent Pods for cost efficiency.

Cost Optimization

Use spot/preemptible instances for fault-tolerant agent workloads (70% cost savings). Combine with PodDisruptionBudgets and graceful shutdown. Monitor costs with Kubecost or OpenCost per namespace and label.

7 FAQ

minikube vs. kind vs. k3s — which should I use locally?→

minikube is best for beginners — single-node with a dashboard and addons. kind is better for CI/CD and multi-node testing (runs entirely in Docker). k3s is a lightweight production-grade distro ideal for edge or resource-constrained environments.

How do I schedule GPU workloads?→

Install the NVIDIA GPU Operator (or device plugin). Then request GPUs in your Pod spec: resources.limits: nvidia.com/gpu: 1. The scheduler assigns the Pod to a node with available GPUs. Use node selectors or taints/tolerations for dedicated GPU node pools.

What is the difference between a Deployment and a StatefulSet?→

Deployments manage stateless Pods — replicas are interchangeable. StatefulSets give each Pod a stable hostname (e.g., redis-0, redis-1) and persistent storage. Use StatefulSets for databases, Kafka brokers, and anything that needs stable identity.

How does HPA work with custom metrics?→

HPA v2 supports custom metrics via the Custom Metrics API. Install Prometheus Adapter or KEDA to expose application metrics (e.g., Kafka consumer lag, request queue depth). KEDA can even scale to zero Pods when there's no work, which is cost-effective for bursty agent workloads.

EKS vs. GKE vs. AKS — which managed K8s should I use?→

GKE has the best auto-upgrade, Autopilot mode, and GPU support. EKS integrates tightly with AWS IAM, ECR, and ALB. AKS excels with Azure AD integration and Azure DevOps. All three support GPU node pools and cluster autoscaler — pick based on your cloud provider commitment.