terraformtfstate
Terraform tfstate – A Practical Playbook
Key Takeaways
- tfstate is the source of truth for all resources Terraform manages.
- Never store
.tfstatefiles in a public repo; always use a remote backend with encryption and locking. - Choose a backend that matches your team size, compliance needs, and cloud provider.
- Implement automated backups, versioning, and state drift detection.
- Use
terraform statesub‑commands for safe migrations and selective resource moves.
Why tfstate Matters
The .tfstate file records the mapping between Terraform configuration and real‑world infrastructure. When the state is inaccurate, Terraform can:
- Attempt to recreate existing resources (causing downtime).
- Fail to apply legitimate changes (leading to configuration drift).
- Expose sensitive data if stored insecurely.
Managing state correctly is therefore a security, reliability, and compliance imperative.
Remote Backend Comparison
| Backend | Storage Type | Locking Mechanism | Encryption at Rest | Versioning | Typical Use‑Case |
|---|---|---|---|---|---|
| Local (default) | File system | None | Depends on OS | No | Single‑developer testing |
| AWS S3 + DynamoDB | Object bucket + NoSQL table | DynamoDB conditional write | AES‑256 (S3) + optional KMS | Enabled via bucket versioning | Production workloads on AWS |
| Azure Blob + Cosmos DB | Blob container + Cosmos DB | Cosmos DB optimistic concurrency | Server‑side encryption (AES‑256) | Blob versioning (preview) | Azure‑centric teams |
| Google Cloud Storage (GCS) | Bucket objects | Object generation pre‑condition | Google‑managed CMEK | Object versioning | GCP environments |
| Terraform Cloud/Enterprise | Managed SaaS | Built‑in state locking | End‑to‑end encryption | Unlimited version history | Teams needing UI, policy, and VCS integration |
Step‑by‑Step Playbook
1. Initialise a Secure Remote Backend
- Identify the cloud provider that hosts the majority of your resources.
- Create a dedicated storage bucket (S3, GCS, Azure Blob) with versioning enabled.
- Provision a lock table/service (DynamoDB, Cosmos DB, GCS generation pre‑condition).
- Add the backend block to
backend.tfand runterraform init -backend-config=backend.tfvars.
2. Enforce State Encryption & Access Controls
- Enable server‑side encryption (SSE‑S3, KMS, CMEK) on the storage bucket.
- Restrict bucket access to the Terraform execution role using IAM policies.
- Audit bucket ACLs weekly with
aws s3api get-bucket-aclor equivalent.
3. Implement Automated Backup & Retention
- Leverage native bucket versioning for point‑in‑time recovery.
- Configure lifecycle rules to retain the last N versions (e.g., 30) and delete older ones.
- Schedule a nightly copy of the latest state to a separate “archive” bucket for disaster recovery.
4. Manage State Drift
- Run
terraform plan -detailed-exitcodein CI on every PR. - Enable
terraform refreshin a scheduled pipeline to detect out‑of‑band changes. - When drift is detected, update the state using
terraform importorterraform state replace-provider.
5. Perform Safe State Migrations
- Export the current state:
terraform state pull > backup.tfstate. - Update the backend configuration to the new target.
- Run
terraform init -migrate-stateto push the state to the new backend. - Validate migration by running
terraform planand confirming no changes.
6. Use State‑Isolation for Environments
- Adopt workspace naming conventions:
dev-,staging-,prod-. - Store each workspace’s state in a dedicated bucket folder (e.g.,
s3://my‑tfstate/dev/). - Restrict production workspace to a dedicated IAM role with MFA.
Essential Checklists
Initial Setup Checklist
- [ ] Create remote storage bucket with versioning.
- [ ] Provision lock table/service.
- [ ] Define
backend.tfwith variables for bucket name, key, region. - [ ] Run
terraform initand