What is ResourceQuota? | DevOps Dictionary

ResourceQuota — Capping What Each Team Can Consume

Why This Matters in Shared Clusters

Without quotas on a multi-tenant cluster, one team running a memory leak can OOMKill every other team's pods by exhausting node memory. At Razorpay, where multiple teams share the same production cluster, ResourceQuotas are the enforcement boundary between teams.

◈ DIAGRAM

+----------------------------------------------------+
| mumbai-prod-cluster  (total: 128 CPU, 512Gi RAM)   |
+----------------------------------------------------+
        |               |               |
        v               v               v
+---------------+ +---------------+ +---------------+
| payments-team | | risk-team     | | data-team     |
| namespace     | | namespace     | | namespace     |
|               | |               | |               |
| Quota:        | | Quota:        | | Quota:        |
| 32 CPU        | | 16 CPU        | | 64 CPU        |
| 128Gi RAM     | | 64Gi RAM      | | 256Gi RAM     |
| 100 pods      | | 50 pods       | | 200 pods      |
+---------------+ +---------------+ +---------------+

A Complete ResourceQuota Manifest

YAML

apiVersion: v1
kind: ResourceQuota
metadata:
  name: payments-team-quota
  namespace: payments-prod
spec:
  hard:
    # ── Compute Resources ──
    requests.cpu: "32"               # Total CPU requests across ALL pods in namespace
    requests.memory: 128Gi           # Total memory requests across ALL pods
    limits.cpu: "64"                 # Total CPU limits across ALL pods
    limits.memory: 256Gi             # Total memory limits across ALL pods
    # ── Object Count Limits ──
    pods: "100"                      # Max number of pods
    services: "20"                   # Max number of Services
    secrets: "50"                    # Max number of Secrets
    configmaps: "30"                 # Max number of ConfigMaps
    persistentvolumeclaims: "20"     # Max number of PVCs
    # ── Service Type Limits ──
    services.loadbalancers: "3"      # Limit expensive cloud load balancers
    services.nodeports: "0"          # Block NodePort services entirely

What Happens When a Quota Is Hit

Bash

+------------------------------------------+
| kubectl apply -f new-deployment.yaml     | <- Team tries to deploy
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Admission Controller checks quota        | <- Checks: used + requested <= hard
+------------------------------------------+
            |               |
            v               v
+------------------+    +------------------------------------------+
| UNDER QUOTA      |    | QUOTA EXCEEDED                           |
| Pod created OK   |    | Error: exceeded quota: payments-team-    |
|                  |    | quota, requested: requests.memory=4Gi,   |
|                  |    | used: requests.memory=126Gi,             |
|                  |    | limited: requests.memory=128Gi           |
+------------------+    +------------------------------------------+

Bash

# What the error looks like in practice
kubectl apply -f new-deployment.yaml
# Error from server (Forbidden): pods "api-server-xyz" is forbidden:
# exceeded quota: payments-team-quota, requested: requests.memory=4Gi,
# used: requests.memory=126Gi, limited: requests.memory=128Gi

The pod goes into Pending and won't schedule until other pods are removed or the quota is raised.

ResourceQuota Requires Resource Requests on Every Pod

REMEMBER THIS
**Remember:** If a namespace has a ResourceQuota for CPU or memory, **every pod in that namespace MUST specify resource requests and limits**. Pods without requests will be rejected outright — even if the namespace has plenty of quota headroom remaining.

YAML

# This pod will be REJECTED if the namespace has a CPU/memory quota
containers:
  - name: api
    image: api:latest
    # Missing resources block -> rejected with "must specify requests" error
 
# This pod will be ACCEPTED
containers:
  - name: api
    image: api:latest
    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
      limits:
        cpu: "1"
        memory: "1Gi"

Viewing Quota Usage

Bash

# Summary view — all quotas in a namespace
kubectl get quota -n payments-prod
 
# Detailed usage breakdown — see used vs hard for every resource
kubectl describe quota payments-team-quota -n payments-prod
# Resource                Used   Hard
# --------                ----   ----
# limits.cpu              28     64
# limits.memory           96Gi   256Gi
# persistentvolumeclaims  14     20
# pods                    67     100
# requests.cpu            14     32
# requests.memory         48Gi   128Gi
# secrets                 31     50
# services                12     20
# services.loadbalancers  2      3
# services.nodeports      0      0
 
# Check quota across all namespaces at once
kubectl get quota -A

ResourceQuota vs LimitRange — How They Work Together

◈ DIAGRAM

+------------------------------------------+
| ResourceQuota                            | <- Namespace ceiling:
|                                          |    "This namespace gets 32 CPU total"
| Enforced at: admission time              |
| Scope: entire namespace                  |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| LimitRange                               | <- Per-container guardrails:
|                                          |    "Each container: 100m-4 CPU"
| Enforced at: admission time              |
| Scope: individual container              |
+------------------------------------------+

Use both together. ResourceQuota without LimitRange means a single container can claim all 32 CPU in one pod. LimitRange without ResourceQuota means per-container limits are set but the namespace has no ceiling — 1000 small pods could still exhaust the cluster.

Troubleshooting Common ResourceQuota Problems

Problem	Symptom	Fix
New pods rejected with Forbidden	`exceeded quota` error on `kubectl apply`	Check `kubectl describe quota` to see which resource is exhausted — scale down unused deployments or request a quota increase
Pod rejected with "must specify requests"	`failed quota` even when under limits	Namespace has ResourceQuota but pod spec has no `resources` block — add explicit CPU and memory requests
Quota shows 0 used but pods exist	Old quota object not matching namespace	Quota `metadata.namespace` doesn't match the namespace you're deploying into — verify with `kubectl get quota -A`
Services.nodeports: 0 but NodePort needed	Service creation blocked	Request a quota change or use an Ingress instead of NodePort — NodePorts are blocked intentionally in multi-tenant clusters
Quota raised but pod still pending	Pod stuck after quota increase	The pod was rejected before the quota increase — delete and reapply the pod after the quota is updated

PLACEMENT PRO TIP
**Tip:** Set up a Prometheus alert when any namespace hits 80% of its quota. At 100%, new deployments silently fail — which is very confusing during an incident when you're trying to scale up your pods to handle a traffic spike.

COMMON MISTAKE / WARNING
**Security:** Use `services.nodeports: "0"` in production quotas to block NodePort services across all team namespaces. NodePorts expose services directly on the node's public IP — on a shared cluster like Razorpay's, this bypasses all ingress-level authentication and rate limiting.

COMMON MISTAKE / WARNING
**Common Mistake:** Setting a quota only on `requests.cpu` and `requests.memory` but not on `limits.cpu` and `limits.memory`. A developer can set `requests: 100m` but `limits: 64 CPU` on a single container — the quota passes (requests are under the cap), but the container can burst and consume the entire node.