ServiceAccount ā Giving Pods Their Own Identity
Why Pods Need an Identity
Your application pods often need to talk to the Kubernetes API ā to discover other services, watch ConfigMaps for config reloads, or scale other workloads. The cluster needs to know: who is this pod, and what is it allowed to do?
ServiceAccounts are the answer. Think of them as IAM roles for pods.
+------------------------------------------+| Pod (running your app) | <- "I am serviceaccount:| | payment-processor"+------------------------------------------+ | v+------------------------------------------+| Kubernetes API Server | <- Receives request with| | Bearer token from pod+------------------------------------------+ | v+------------------------------------------+| RBAC Authorization Check | <- Does payment-processor| | have GET on secrets?+------------------------------------------+ | | v v +------------+ +------------+ | ALLOWED | | DENIED | | 200 OK | | 403 Error | +------------+ +------------+The Default ServiceAccount Problem
Every namespace gets a default ServiceAccount automatically. If you don't set serviceAccountName in your pod spec, your pods use this default SA.
+------------------------------------------+| Namespace: payments-prod || || default ServiceAccount (auto-created) | <- ALL pods use this unless| | you specify otherwise| payments-api pod āā> default SA || fraud-checker pod āā> default SA | <- Both pods share the same| batch-job pod āā> default SA | identity and permissions+------------------------------------------+ā ļø Security: In older clusters, the default SA often has broad permissions inherited from cluster-admin bindings added during setup. In production, always create dedicated ServiceAccounts with the minimum permissions each workload actually needs ā never rely on the default SA.Creating a Scoped ServiceAccount ā Real Example
This example sets up a ServiceAccount for a Prometheus pod that needs to scrape metrics endpoints across the cluster:
1# 1. Create the ServiceAccount2apiVersion: v13kind: ServiceAccount4metadata:5 name: prometheus-scraper6 namespace: monitoring78# 2. Create a ClusterRole with exactly the permissions needed9apiVersion: rbac.authorization.k8s.io/v110kind: ClusterRole11metadata:12 name: prometheus-metrics-reader13rules:14 - apiGroups: [""]15 resources: ["nodes", "pods", "services", "endpoints"]16 verbs: ["get", "list", "watch"] # Read-only ā cannot create or delete17 - apiGroups: [""]18 resources: ["nodes/metrics"]19 verbs: ["get"]2021# 3. Bind the ClusterRole to the ServiceAccount22apiVersion: rbac.authorization.k8s.io/v123kind: ClusterRoleBinding24metadata:25 name: prometheus-scraper-binding26subjects:27 - kind: ServiceAccount28 name: prometheus-scraper29 namespace: monitoring30roleRef:31 kind: ClusterRole32 name: prometheus-metrics-reader33 apiGroup: rbac.authorization.k8s.io3435# 4. Assign the ServiceAccount to the pod36apiVersion: apps/v137kind: Deployment38metadata:39 name: prometheus40 namespace: monitoring41spec:42 template:43 spec:44 serviceAccountName: prometheus-scraper # <- This is what links pod to SA45 containers:46 - name: prometheus47 image: prom/prometheus:v2.48.0How the Token Gets Into the Pod
Kubernetes automatically mounts the ServiceAccount token into every pod as a projected volume:
+------------------------------------------+| Pod filesystem || || /var/run/secrets/ || kubernetes.io/ || serviceaccount/ || token <- JWT Bearer token || ca.crt <- API server CA cert || namespace <- Current namespace |+------------------------------------------+1# Read the token from inside a running pod2kubectl exec -it api-server-7d9f8b -n production -- \3 cat /var/run/secrets/kubernetes.io/serviceaccount/token4 5# Use the token to call the Kubernetes API from inside the pod6TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)7curl -k -H "Authorization: Bearer $TOKEN" \8 https://kubernetes.default.svc/api/v1/namespaces/production/pods9 10# Inspect the token's contents (decoded JWT)11kubectl create token prometheus-scraper -n monitoring --duration=1h | \12 cut -d. -f2 | base64 -d 2>/dev/null | jqDisabling Auto-Mount for Non-API Pods
Most application pods don't need Kubernetes API access at all. Disabling the token mount reduces the attack surface ā a compromised pod cannot use the token to probe the API:
1spec:2 serviceAccountName: payments-api-sa3 automountServiceAccountToken: false # Don't mount the token ā app doesn't need it4 containers:5 - name: payments-api6 image: registry.razorpay.in/payments-api:v3.1.2Troubleshooting Common ServiceAccount Problems
| Problem | Symptom | Fix |
|---|---|---|
| Pod gets 403 calling the API | Error: Forbidden in app logs |
SA lacks the required verb ā check with kubectl auth can-i as the SA |
Pod stuck in Pending |
ServiceAccount not found event |
SA not created before the pod ā create SA first, then deploy |
| Prometheus scraping fails | 403 Forbidden on /metrics |
ClusterRoleBinding is in wrong namespace or references wrong SA name |
| Token expired in long-running pods | API calls start failing after 24h | Use projected service account tokens with expirationSeconds: 86400 and enable token rotation |
| CI bot has too much access | Blast radius concern | Replace cluster-admin SA with a minimal ci-deployer Role bound only to deployments/patch |
š” Tip: At Swiggy's scale, every microservice has its own dedicated ServiceAccount with the least-privilege permissions it actually needs. This means a compromised payment pod cannot read secrets from the delivery namespace ā the blast radius is contained to one workload.
š Remember: kubectl auth can-i get pods --as=system:serviceaccount:monitoring:prometheus-scraper is the fastest way to verify what a ServiceAccount is allowed to do without deploying anything. Always test this before going to production.š“ Common Mistake: Creating a ServiceAccount but forgetting to addserviceAccountNameto the pod spec. The pod silently falls back to thedefaultSA instead of the scoped one ā and you won't notice until a permission error surfaces in production.
Quick Reference
| Command | Purpose |
|---|---|
kubectl get serviceaccounts -n <ns> |
List all SAs in a namespace |
kubectl describe sa <name> -n <ns> |
See mounted secrets and token references |
kubectl create token <sa-name> -n <ns> |
Generate a short-lived token for testing |
kubectl auth can-i get pods --as=system:serviceaccount:<ns>:<sa> |
Test what an SA is allowed to do |
kubectl get rolebindings -n <ns> -o wide |
See which SAs are bound to which roles |