HPA - Automatic Scaling Based on Real Traffic
What is HPA in Simple Terms?
HPA is your automatic traffic manager. During Swiggy's dinner rush (7pm–9pm), order volume spikes 10x. HPA detects that CPU is hitting 80%, automatically adds more pods to handle the load, then scales back down at midnight when traffic drops — saving infrastructure costs without any human intervention.
How HPA Decides to Scale
◈ DIAGRAM
+--------------------------------------------------+| HPA controller polls Metrics Server every 15s | <- continuous monitoring+--------------------------------------------------+ | v+--------------------------------------------------+| Current avg CPU across pods: 85% || Target avg CPU configured: 70% || Pods needed = ceil(current / target * replicas) |+--------------------------------------------------+ | +-----------+-----------+ | | v v+---------------------+ +---------------------+| CPU > target | | CPU < target || -> Scale UP | | -> Scale DOWN || Add pods (up to max)| | Remove pods (to min) |+---------------------+ +---------------------+The scale formula Kubernetes uses: desiredReplicas = ceil(currentReplicas * (currentMetric / targetMetric))
Example HPA Manifest
YAML
# hpa.yaml — autoscale the order-service based on CPU and memoryapiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: order-service-hpa namespace: productionspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: order-service minReplicas: 3 # Always keep at least 3 pods — never scale to zero maxReplicas: 20 # Hard ceiling — prevents runaway scaling metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 # Scale when avg CPU across all pods hits 70% - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 # Also scale if memory pressure builds upDeployment Must Have Resource Requests Defined
HPA cannot calculate utilization percentage without knowing what the pod requested. This is a mandatory prerequisite:
YAML
# deployment.yaml — resources.requests MUST be set for HPA to workspec: containers: - name: order-service image: registry.swiggy.in/order-service:v3.1.0 resources: requests: cpu: "250m" # HPA uses this as the baseline for % calculation memory: "256Mi" limits: cpu: "1000m" memory: "512Mi"Checking HPA Status
Bash
# See current replica count, targets, and scaling activitykubectl get hpa -n production # Output:# NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS# order-service-hpa Deployment/order-svc 68%/70% 3 20 8 # Detailed status including last scale eventkubectl describe hpa order-service-hpa -n production # Watch HPA react to live traffic in real timekubectl get hpa order-service-hpa -n production -wScaling Behavior — Controlling Scale Up and Scale Down Speed
By default, HPA can be aggressive on scale-up and slow on scale-down. You can tune this:
YAML
# Add behavior block to control scaling velocityspec: behavior: scaleUp: stabilizationWindowSeconds: 0 # Scale up immediately when needed policies: - type: Pods value: 4 # Add at most 4 pods per scaling event periodSeconds: 60 scaleDown: stabilizationWindowSeconds: 300 # Wait 5 minutes before scaling down policies: - type: Percent value: 10 # Remove at most 10% of pods per minute periodSeconds: 60The stabilizationWindowSeconds on scale-down prevents flapping — HPA won't remove pods immediately after a traffic spike drops, giving headroom for the next wave.
Troubleshooting HPA
| Symptom | Likely Cause | Fix |
|---|---|---|
HPA shows <unknown>/70% |
Metrics Server not installed | Install metrics-server in kube-system |
| HPA not scaling up | resources.requests not set on deployment |
Add CPU/memory requests to pod spec |
HPA stuck at minReplicas |
Current CPU below threshold | Check actual usage with kubectl top pods |
| HPA scaling too aggressively | No stabilization window | Add behavior.scaleDown.stabilizationWindowSeconds |
Replicas hit maxReplicas and stop |
Max ceiling reached | Raise maxReplicas or investigate pod performance |
Bash
# Verify Metrics Server is working (prerequisite for HPA)kubectl top pods -n production # Check why HPA is not scaling — events section is keykubectl describe hpa order-service-hpa -n production | grep -A 20 Events # Manually simulate load to test HPA behaviorkubectl run load-gen --image=busybox -it --rm -- \ /bin/sh -c "while true; do wget -q -O- http://order-service.production; done"REMEMBER THIS**Remember:** HPA requires the Metrics Server to be installed in the cluster. Without it, HPA cannot read CPU or memory metrics and will report `` targets — staying completely inactive. Run `kubectl top pods` to verify Metrics Server is working before creating any HPA.
COMMON MISTAKE / WARNING**Common Mistake:** Setting `minReplicas: 1` in production. If that single pod is being replaced during a scale-up event, your service has zero availability for the seconds it takes to start a new pod. Always set `minReplicas: 3` or higher for any production workload at Razorpay or PhonePe scale.
PLACEMENT PRO TIP**Tip:** Set `scaleDown.stabilizationWindowSeconds: 300` in the HPA behavior block. Without this, HPA will remove pods immediately after a traffic spike drops — only to add them back 2 minutes later when the next spike arrives. The stabilization window prevents this flapping and keeps your pod count stable during volatile traffic patterns like Hotstar's live streaming events.
COMMON MISTAKE / WARNING**Security:** HPA with `maxReplicas` set too high can become a cost explosion vector. A traffic spike — or a DDoS — can trigger HPA to spin up hundreds of pods, consuming all cluster node capacity and triggering expensive cloud autoscaling. Always set a sensible `maxReplicas` ceiling and configure cluster-level resource quotas per namespace to cap total pod resource consumption.