Written by Sujith Quintelier • Aug 6, 2025

🖥️ Zero to Hero: Terraform for Azure•Part 2

Master Terraform state management with Azure remote backends. Learn state locking, team collaboration, and secure, scalable infrastructure practices.

Aug 6, 2025

Introduction

In Part 1, we set up our development environment and deployed our first Azure resource group using Terraform. Whilst this gave us a taste of Infrastructure as Code, we were using local state files: a practice that won’t scale for team environments or production workloads.

In this second part of our series, we’ll explore Terraform state management and remote backends. You’ll learn why state is crucial, how to configure Azure Storage as a remote backend, implement state locking for team safety, and master essential state management commands.

By the end of this post, you’ll have production-ready state management that enables secure team collaboration and robust infrastructure management.

Prerequisites

Before we begin, ensure you have:

Completed Part 1 of this series
Terraform and Azure CLI installed and configured
An active Azure subscription
Basic understanding of Azure resource groups and storage accounts

Understanding Terraform State

What is Terraform State?

Terraform state is a JSON file that maps your configuration files to real-world resources. It serves as Terraform’s “memory” of what infrastructure exists and how it relates to your configuration.

When you ran terraform apply in Part 1, Terraform created a file called terraform.tfstate in your working directory. Let’s examine what’s inside:

# Navigate to your terraform-azure directory from Part 1
cd terraform-azure

# View the state file (if it exists)
cat terraform.tfstate

The state file contains crucial information:

Resource metadata and attributes
Resource dependencies
Provider configuration details
Resource addresses and unique identifiers

Why State Matters

Terraform state is essential for several reasons:

Performance: State caching avoids costly API calls to cloud providers
Collaboration: Teams need shared state to avoid conflicts
Metadata Storage: State stores resource metadata not available via APIs
Resource Mapping: Links configuration to real infrastructure
Dependency Tracking: Maintains resource relationship information

Local State Limitations

Storing state locally creates several problems:

No Team Collaboration: Multiple developers can’t share state
No Locking: Concurrent operations can corrupt state
No Backup: Local files can be lost or corrupted
Security Risk: Sensitive data stored in local files
No Audit Trail: No history of infrastructure changes

Setting Up Azure Storage Remote Backend

Step 1: Create Azure Storage Resources

Let’s create the Azure resources needed for remote state storage.

Important Note: Creating the storage account for Terraform state using Terraform itself creates a chicken-and-egg problem - you need a place to store state before you can manage infrastructure with Terraform. Unless you have a fully working landing zone system with existing state management, the storage account should be created using Azure CLI or other tools.

Option A: Azure CLI (Recommended for State Storage)

Create the backend storage using Azure CLI:

# Set variables for consistency
RESOURCE_GROUP_NAME="terraform-backend-rg"
STORAGE_ACCOUNT_NAME="tfstate$(openssl rand -hex 4)"  # Generates unique suffix
CONTAINER_NAME="tfstate"
LOCATION="West Europe"

# Create resource group
az group create \
  --name $RESOURCE_GROUP_NAME \
  --location "$LOCATION"

# Create storage account
az storage account create \
  --name $STORAGE_ACCOUNT_NAME \
  --resource-group $RESOURCE_GROUP_NAME \
  --location "$LOCATION" \
  --sku Standard_LRS \
  --kind StorageV2 \
  --allow-blob-public-access false \
  --min-tls-version TLS1_2

# Create container
az storage container create \
  --name $CONTAINER_NAME \
  --account-name $STORAGE_ACCOUNT_NAME \
  --auth-mode login

# Display information for backend configuration
echo "Backend Configuration Details:"
echo "Resource Group: $RESOURCE_GROUP_NAME"
echo "Storage Account: $STORAGE_ACCOUNT_NAME"
echo "Container: $CONTAINER_NAME"

Option B: Terraform (For Other Infrastructure)

If you already have a backend storage solution or are creating infrastructure other than the state storage itself, you can use Terraform. Create a separate directory for this:

mkdir terraform-infrastructure
cd terraform-infrastructure

Create infrastructure.tf for your application resources:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
  
  # Configure remote backend (created via Azure CLI)
  backend "azurerm" {
    resource_group_name  = "terraform-backend-rg"
    storage_account_name = "tfstate<your-suffix>"
    container_name       = "tfstate"
    key                  = "infrastructure/terraform.tfstate"
  }
}

provider "azurerm" {
  features {}
  # subscription_id is required in provider 4.0+
  # Set via ARM_SUBSCRIPTION_ID environment variable
}

# Example application infrastructure (NOT the backend storage)
resource "azurerm_resource_group" "app" {
  name     = "myapp-rg"
  location = "West Europe"
  
  tags = {
    Environment = "Development"
    ManagedBy   = "Terraform"
  }
}

resource "azurerm_storage_account" "app" {
  name                     = "myapp${random_string.app_suffix.result}"
  resource_group_name      = azurerm_resource_group.app.name
  location                 = azurerm_resource_group.app.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  
  tags = {
    Environment = "Development"
    Purpose     = "Application"
  }
}

resource "random_string" "app_suffix" {
  length  = 8
  special = false
  upper   = false
}

Why This Approach Works

Backend Storage: Created manually via Azure CLI (no circular dependency)
Application Infrastructure: Managed by Terraform using the pre-existing backend
Clear Separation: State storage is infrastructure foundation, everything else is managed code

Deploy the application infrastructure:

# Initialize and apply the infrastructure
terraform init
terraform apply

Important: Use the storage account name from the Azure CLI output in your backend configuration.

Enterprise Landing Zone Approach

In enterprise environments with established landing zones, you might have:

Centralised Backend Management: Shared storage accounts managed by platform teams
Terraform Cloud/Enterprise: Managed remote state solutions
Azure DevOps/GitHub: Pipeline-managed backend configuration
Multi-Subscription Architecture: Separate subscriptions for state management

For our learning series, the Azure CLI approach provides the cleanest foundation.

Important: Azure RM Provider 4.0 Requirements

Azure RM provider 4.0 introduced stricter authentication requirements. You must specify a subscription ID using one of these methods:

Method 1: Environment Variable (Recommended)

# Set subscription ID via environment variable
export ARM_SUBSCRIPTION_ID="your-subscription-id"

# Get your subscription ID if needed
az account show --query id --output tsv

Method 2: Provider Configuration (Not Recommended for Production)

provider "azurerm" {
  features {}
  subscription_id = "your-subscription-id"  # Avoid hardcoding in production
}

Method 3: Azure CLI Default (Sometimes Works)

# Ensure you're logged in and have a default subscription
az login
az account set --subscription "your-subscription-id"

Best Practice: Always use environment variables (ARM_SUBSCRIPTION_ID) to avoid hardcoding sensitive values in your Terraform configuration.

Step 2: Configure Remote Backend

Now let’s configure our Terraform project to use the remote backend. We’ll work with our original terraform-azure directory:

cd ../terraform-azure

Best Practice: Secure Backend Configuration

Rather than hardcoding backend configuration in main.tf, we’ll use a secure approach that prevents the chicken-and-egg problem and improves security.

Clean Main Configuration

Update your main.tf to declare the backend without sensitive details:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
  
  # Partial backend configuration - details provided during init
  backend "azurerm" {}
}

provider "azurerm" {
  features {}
  # subscription_id is required in provider 4.0+
  # Set via ARM_SUBSCRIPTION_ID environment variable
}

resource "azurerm_resource_group" "example" {
  name     = "terraform-rg"
  location = "West Europe"
}

Backend Configuration for CI/CD

Create backend.hcl for pipeline use (add to .gitignore):

# backend.hcl - Used in CI/CD pipelines
resource_group_name  = "terraform-backend-rg"
storage_account_name = "tfstate<your-suffix>"
container_name       = "tfstate"
key                  = "terraform.tfstate"

Local Development Override

Create override.tf for local development (add to .gitignore):

# override.tf - Local development only
terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-backend-rg"
    storage_account_name = "tfstate<your-suffix>"
    container_name       = "tfstate"
    key                  = "terraform.tfstate"
  }
}

Step 3: Migrate to Remote Backend

Option A: Local Development (using override)

If you created override.tf, simply initialise:

# Migrate existing state to remote backend
terraform init -migrate-state

Option B: CI/CD Pipeline (using backend config file)

In CI/CD pipelines, use the backend configuration file:

# Initialize with backend configuration file
terraform init -backend-config=backend.hcl -migrate-state

Option C: Pipeline with Environment Variables

For even better security, use environment variables in pipelines:

# Set backend configuration via environment variables
export TF_VAR_resource_group_name="terraform-backend-rg"
export TF_VAR_storage_account_name="tfstate<your-suffix>"
export TF_VAR_container_name="tfstate"
export TF_VAR_key="terraform.tfstate"

# Initialize with backend config from environment
terraform init \
  -backend-config="resource_group_name=$TF_VAR_resource_group_name" \
  -backend-config="storage_account_name=$TF_VAR_storage_account_name" \
  -backend-config="container_name=$TF_VAR_container_name" \
  -backend-config="key=$TF_VAR_key" \
  -migrate-state

Verify Migration

Regardless of the method used, verify the migration:

# Check that state is now remote
terraform state list

You should see your existing resources listed, but the state is now stored remotely in Azure Storage.

State Locking with Azure Blob Storage

Azure Storage provides automatic state locking when using the azurerm backend. This prevents multiple users from running Terraform operations simultaneously, which could corrupt the state.

How State Locking Works

Acquisition: Terraform acquires a lease on the state blob
Operation: Terraform performs the requested operation
Release: Terraform releases the lease when complete
Conflict Prevention: If another operation is in progress, Terraform waits or fails

Testing State Locking

Let’s test state locking behaviour:

# In one terminal, start a long-running operation
terraform plan

# In another terminal (different location), try another operation
terraform plan

The second operation will wait for the first to complete or will fail with a lock acquisition error.

Force Unlock (Emergency Only)

If Terraform crashes and leaves a lock, you can force unlock:

# Only use in emergencies - can lead to state corruption
terraform force-unlock <lock-id>

Warning: Only use force-unlock when you’re certain no other Terraform operations are running.

Essential State Management Commands

Terraform provides several commands for inspecting and managing state:

Listing Resources

# List all resources in state
terraform state list

# List resources with a filter
terraform state list "azurerm_resource_group.*"

Inspecting Resources

# Show detailed information about a resource
terraform state show azurerm_resource_group.example

# Show all state information (large output)
terraform show

Moving Resources

# Rename a resource in state (useful for refactoring)
terraform state mv azurerm_resource_group.example azurerm_resource_group.main

Removing Resources

# Remove a resource from state (doesn't delete the actual resource)
terraform state rm azurerm_resource_group.example

Importing Existing Resources

If you have existing Azure resources not managed by Terraform, you can import them:

# Import an existing resource group
terraform import azurerm_resource_group.existing /subscriptions/{subscription-id}/resourceGroups/{resource-group-name}

Backing Up State

# Create a backup of current state
terraform state pull > terraform.tfstate.backup

Practical Example: Multi-Environment State Management

Let’s create a more realistic example with separate state files for different environments:

Environment-Specific Backend Configuration

Create separate backend configurations for each environment:

environments/dev/main.tf:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
  
  backend "azurerm" {
    resource_group_name  = "terraform-backend-rg"
    storage_account_name = "tfstate<your-suffix>"
    container_name       = "tfstate"
    key                  = "dev/terraform.tfstate"  # Environment-specific key
  }
}

provider "azurerm" {
  features {}
  # subscription_id is required in provider 4.0+
  # Set via ARM_SUBSCRIPTION_ID environment variable
}

resource "azurerm_resource_group" "main" {
  name     = "myapp-dev-rg"
  location = "West Europe"
  
  tags = {
    Environment = "Development"
    ManagedBy   = "Terraform"
  }
}

environments/prod/main.tf:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
  
  backend "azurerm" {
    resource_group_name  = "terraform-backend-rg"
    storage_account_name = "tfstate<your-suffix>"
    container_name       = "tfstate"
    key                  = "prod/terraform.tfstate"  # Environment-specific key
  }
}

provider "azurerm" {
  features {}
  # subscription_id is required in provider 4.0+
  # Set via ARM_SUBSCRIPTION_ID environment variable
}

resource "azurerm_resource_group" "main" {
  name     = "myapp-prod-rg"
  location = "West Europe"
  
  tags = {
    Environment = "Production"
    ManagedBy   = "Terraform"
  }
}

Deploying Multiple Environments

# Deploy development environment
cd environments/dev
terraform init
terraform apply

# Deploy production environment
cd ../prod
terraform init
terraform apply

Each environment maintains its own state file in Azure Storage, preventing conflicts between environments.

Team Collaboration Best Practices

1. Secure Backend Configuration Management

Recommended Approach: Keep sensitive backend configuration out of version control:

# Project structure
terraform-project/
├── main.tf                    # Clean, no sensitive data
├── variables.tf
├── outputs.tf
├── .gitignore                 # Include backend configs
├── backend.hcl               # CI/CD backend config (gitignored)
└── override.tf               # Local dev config (gitignored)

.gitignore entries:

# Backend configuration files
backend.hcl
override.tf
*.auto.tfvars
.terraform/
terraform.tfstate*

Team Workflow:

Developers: Use override.tf for local development
CI/CD Pipelines: Use backend.hcl or environment variables
Shared Code: Keep main.tf clean with partial backend configuration

2. Environment-Specific Backend Strategies

Separate Backend Files per Environment

# Directory structure
environments/
├── dev/
│   ├── main.tf
│   └── backend-dev.hcl
├── staging/
│   ├── main.tf
│   └── backend-staging.hcl
└── prod/
    ├── main.tf
    └── backend-prod.hcl

Dynamic Backend Configuration in Pipelines

# GitHub Actions example
- name: Configure Backend
  run: |
    terraform init \
      -backend-config="key=${{ env.ENVIRONMENT }}/terraform.tfstate" \
      -backend-config="storage_account_name=${{ secrets.STORAGE_ACCOUNT }}"

3. Access Control

Implement proper access controls for your backend storage:

# Grant specific users access to the storage account
az role assignment create \
  --assignee user@company.com \
  --role "Storage Blob Data Contributor" \
  --scope "/subscriptions/{subscription-id}/resourceGroups/terraform-backend-rg/providers/Microsoft.Storage/storageAccounts/{storage-account-name}"

3. Authentication Methods

Configure authentication for team collaboration. Note: Azure RM provider 4.0+ requires subscription_id to be explicitly set.

Option 1: Azure CLI (Development):

az login

# Provider 4.0+ requires explicit subscription configuration
export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)

Option 2: Service Principal (CI/CD - Legacy):

# Set environment variables - subscription_id is now mandatory
export ARM_CLIENT_ID="<service-principal-id>"
export ARM_CLIENT_SECRET="<service-principal-password>"
export ARM_SUBSCRIPTION_ID="<subscription-id>"
export ARM_TENANT_ID="<tenant-id>"

Option 3: Federated Credentials (CI/CD - Recommended):

Modern CI/CD systems should use OpenID Connect (OIDC) federated credentials for secure, secret-free authentication:

# Set environment variables for federated auth
export ARM_CLIENT_ID="<service-principal-id>"
export ARM_SUBSCRIPTION_ID="<subscription-id>"
export ARM_TENANT_ID="<tenant-id>"
export ARM_USE_OIDC=true
# No ARM_CLIENT_SECRET needed!

GitHub Actions Example:

- name: Azure Login
  uses: azure/login@v1
  with:
    client-id: ${{ secrets.AZURE_CLIENT_ID }}
    tenant-id: ${{ secrets.AZURE_TENANT_ID }}
    subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

- name: Terraform Apply
  run: terraform apply -auto-approve
  env:
    ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
    ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    ARM_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
    ARM_USE_OIDC: true

Azure DevOps Example:

- task: AzureCLI@2
  inputs:
    azureSubscription: 'your-service-connection'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      export ARM_CLIENT_ID=$(az account show --query user.name --output tsv)
      export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
      export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)
      export ARM_USE_OIDC=true
      terraform apply -auto-approve

Option 4: Managed Identity (Azure VMs):

provider "azurerm" {
  features {}
  use_msi = true
  # subscription_id still required even with managed identity
  subscription_id = var.subscription_id  # Use variable, not hardcoded
}

Troubleshooting Common Issues

Issue 1: State Lock Timeout

Problem: Terraform hangs waiting for state lock.

Solution:

# Check for stuck locks in Azure portal or CLI
az storage blob lease show \
  --account-name <storage-account> \
  --container-name tfstate \
  --name terraform.tfstate

# Force unlock if necessary (use with caution)
terraform force-unlock <lock-id>

Issue 2: Authentication Errors

Problem: “Failed to get existing workspaces” or similar auth errors.

Solution:

# Re-authenticate with Azure
az login
az account show  # Verify correct subscription

# Or set environment variables for service principal
export ARM_CLIENT_ID="..."
export ARM_CLIENT_SECRET="..."
export ARM_SUBSCRIPTION_ID="..."
export ARM_TENANT_ID="..."

For Federated Credentials Issues:

# Verify OIDC setup
export ARM_CLIENT_ID="..."
export ARM_SUBSCRIPTION_ID="..."
export ARM_TENANT_ID="..."
export ARM_USE_OIDC=true

# Check if federated credential is properly configured in Azure AD
az ad sp federated-credential list --id $ARM_CLIENT_ID

Issue 3: State File Corruption

Problem: State file becomes corrupted or inconsistent.

Solution:

# Restore from backup
terraform state pull > current-state-backup.json
# Manually restore from previous backup or fix issues
terraform state push fixed-state.json

# Or refresh state from actual infrastructure
terraform refresh

Issue 4: Backend Migration Issues

Problem: Errors during backend migration.

Solution:

# Backup current state first
cp terraform.tfstate terraform.tfstate.backup

# Re-run migration with debugging
TF_LOG=DEBUG terraform init -migrate-state

# If migration fails, manually copy state
terraform state pull > local-backup.json
# Configure new backend
terraform init
terraform state push local-backup.json

Security Considerations

1. State File Security

State files contain sensitive information:

Resource configurations
Output values (potentially secrets)
Provider credentials
Resource metadata

Best Practices:

Enable encryption at rest for Azure Storage
Use private storage containers
Implement proper access controls
Audit access to state files
Never commit state files to version control

2. Backend Security Configuration

# Secure backend configuration example
terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-backend-rg"
    storage_account_name = "securebackendstore"
    container_name       = "tfstate"
    key                  = "prod/terraform.tfstate"
    
    # Additional security options
    use_azuread_auth     = true  # Use Azure AD authentication
    use_msi              = true  # Use managed identity when available
  }
}

3. Secrets Management

Never store secrets in Terraform state or configuration:

# BAD: Don't put secrets in configuration
resource "azurerm_key_vault_secret" "bad_example" {
  name         = "database-password"
  value        = "super-secret-password"  # This will be in state!
  key_vault_id = azurerm_key_vault.main.id
}

# GOOD: Use sensitive variables or external secret management
variable "database_password" {
  description = "Database password"
  type        = string
  sensitive   = true
}

resource "azurerm_key_vault_secret" "good_example" {
  name         = "database-password"
  value        = var.database_password
  key_vault_id = azurerm_key_vault.main.id
}

Best Practices Summary

Do’s ✅

Always use remote backends for team environments
Enable state locking to prevent conflicts
Use environment-specific state files for isolation
Implement proper access controls on backend storage
Backup state files regularly
Use consistent backend configurations across the team
Monitor backend storage for capacity and performance
Document backend setup for team onboarding

Don’ts ❌

Don’t commit state files to version control
Don’t share storage account keys directly
Don’t use the same state file for multiple environments
Don’t skip state locking in team environments
Don’t store secrets in Terraform configuration
Don’t ignore backend authentication security
Don’t force-unlock unless absolutely necessary
Don’t manually edit state files

Next Steps

Congratulations! You now have production-ready state management with Azure remote backends. Your infrastructure state is secure, shareable, and protected against conflicts.

In Part 3: Variables, Outputs & Data Sources, we’ll build on this foundation to create dynamic and reusable Terraform configurations. You’ll learn how to:

Use input variables for flexibility
Output values for resource sharing
Leverage data sources for existing resources
Manage environment-specific configurations
Implement variable validation and sensitive handling

This knowledge will transform your static configurations into powerful, reusable infrastructure code that adapts to different environments and use cases.

Stay tuned for more hands-on Terraform learning! 🚀

Additional Resources

Zero to Hero: Terraform for Azure

Cloud Infrastructure as Code IaC DevOps

Terraform Azure IaC Cloud Infrastructure State Management Azure Storage Team Collaboration