What Is Terraform Remote State?
The state file is Terraform's memory. It records every resource Terraform has ever created — the EC2 instance ID, the S3 bucket name, the RDS endpoint. Without it, Terraform has no idea what exists in the real world.
By default, that memory lives in a file called terraform.tfstate on the laptop of whoever ran terraform apply last. That works fine when you are the only engineer. The moment a second engineer joins, you have a problem — they have a different state file, or no state file at all. Two engineers working against different state files is how infrastructure gets duplicated, orphaned, or broken.
Remote state solves this by moving the state file off individual laptops and into a shared, durable location that every engineer and every CI/CD pipeline reads from and writes to.
At Zerodha, the infrastructure team keeps state in an S3 bucket in ap-south-1. Every engineer runs terraform plan against the same state. Every CI/CD pipeline applies against the same state. Nobody can accidentally work against a stale local copy.
+------------------------------------------+| WITHOUT remote state (dangerous) || || Engineer A laptop: terraform.tfstate || Engineer B laptop: terraform.tfstate || CI/CD pipeline: no state file || Result: three different views of || infrastructure, conflicts guaranteed |+------------------------------------------+ | v+------------------------------------------+| WITH remote state (safe) || || Engineer A ---+ || Engineer B ---+---> S3 terraform.tfstate || CI/CD ---+ || Result: one shared source of truth |+------------------------------------------+Setting Up Remote State on AWS S3
You need two AWS resources before configuring remote state: an S3 bucket to store the state file, and a DynamoDB table to handle locking (covered in the State Lock glossary term).
# bootstrap/main.tf# Create these resources ONCE manually or with a separate bootstrap config# Never store the state of your state bucket in itself resource "aws_s3_bucket" "terraform_state" { bucket = "razorpay-terraform-state-ap-south-1" # must be globally unique # Prevent accidental deletion of this critical bucket lifecycle { prevent_destroy = true }} # Enable versioning — recover any previous state file version if corruption occursresource "aws_s3_bucket_versioning" "terraform_state" { bucket = aws_s3_bucket.terraform_state.id versioning_configuration { status = "Enabled" }} # Encrypt state at rest — state files contain sensitive values in plaintextresource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" { bucket = aws_s3_bucket.terraform_state.id rule { apply_server_side_encryption_by_default { sse_algorithm = "aws:kms" # KMS for state — stronger than AES256 } }} # Block all public access — state files must never be publicresource "aws_s3_bucket_public_access_block" "terraform_state" { bucket = aws_s3_bucket.terraform_state.id block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true} # DynamoDB table for state lockingresource "aws_dynamodb_table" "terraform_locks" { name = "terraform-state-lock" billing_mode = "PAY_PER_REQUEST" # no capacity planning needed hash_key = "LockID" # exact key name Terraform expects attribute { name = "LockID" type = "S" # string }}Configuring the Backend
Once the S3 bucket and DynamoDB table exist, add the backend block to your Terraform configuration:
# versions.tfterraform { required_version = ">= 1.6.0" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } backend "s3" { bucket = "razorpay-terraform-state-ap-south-1" key = "prod/payments-api/terraform.tfstate" # path inside the bucket region = "ap-south-1" dynamodb_table = "terraform-state-lock" # enables locking encrypt = true # encrypt the state file at rest }}The key argument is the path inside the bucket where Terraform stores the state file. Use a hierarchy that reflects your organisation:
s3://razorpay-terraform-state-ap-south-1/ prod/ payments-api/terraform.tfstate notifications/terraform.tfstate shared-networking/terraform.tfstate staging/ payments-api/terraform.tfstate dev/ payments-api/terraform.tfstateRunning terraform init After Adding a Backend
After adding the backend block for the first time, run terraform init. Terraform detects the new backend and offers to migrate any existing local state to S3:
terraform init # Initializing the backend...# Do you want to copy existing state to the new backend?# Pre-existing state was found while migrating the previous backend.# Would you like to copy this state to the new backend?# Enter a value: yes## Successfully configured the backend "s3"!# Terraform will automatically use this backend unless the backend# configuration changes.Reading Remote State From Another Configuration
The most powerful feature of remote state is cross-configuration references. The networking team's Terraform configuration outputs the VPC ID. The compute team's configuration reads that output without needing to hardcode or duplicate the value.
# In the networking configuration — outputs.tfoutput "vpc_id" { value = aws_vpc.main.id} output "private_subnet_ids" { value = aws_subnet.private[*].id}# In the compute configuration — reads networking outputs via remote statedata "terraform_remote_state" "networking" { backend = "s3" config = { bucket = "razorpay-terraform-state-ap-south-1" key = "prod/shared-networking/terraform.tfstate" region = "ap-south-1" }} resource "aws_instance" "app" { # Reference the VPC output from the networking team's state subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0] # No hardcoded subnet ID — always reads the current value from state}Troubleshooting Remote State
| Error | Root Cause | Fix |
|---|---|---|
NoSuchBucket: The specified bucket does not exist |
S3 bucket not created yet | Create the bucket before running terraform init |
AccessDenied on S3 |
IAM permissions missing | Add s3:GetObject, s3:PutObject, s3:DeleteObject, s3:ListBucket |
ResourceNotFoundException on DynamoDB |
Lock table not created | Create the DynamoDB table with LockID as partition key |
Error: Backend configuration changed |
Backend block was modified | Run terraform init -reconfigure |
Error: Failed to read state |
State file corrupted | Restore from S3 versioned backup |
REMEMBER THIS**Remember:** The S3 bucket for state must exist before you run `terraform init` with the backend configured. You cannot use Terraform to create the bucket that stores its own state — bootstrap it manually or with a separate one-time script.
COMMON MISTAKE / WARNING**Common Mistake:** Putting the backend configuration in a `terraform.tfvars` file or using variables inside the backend block. Backend blocks cannot use variables — all values must be hardcoded literals or passed via `-backend-config` flags.