Remote Terraform state

Introduction

Learn how to configure Terraform to use remote state storage in an Atlas S3 bucket. Remote state provides better collaboration, state locking, and persistence for your Terraform deployments.

Prerequisites

Before you begin, ensure you have:

• An Atlas Cloud account
• Terraform installed on your local machine
• Atlas S3 bucket credentials (see below)

Step 1: Create an S3 Bucket for Remote State

If you haven’t already created an S3 bucket, follow our Website hosting on a static S3 bucket tutorial to create one. For remote state storage:

1. Navigate to Atlas Storage in your Atlas Cloud
2. Click “Create Bucket”
3. Use a descriptive name like terraform-state
4. Set the access policy to “Private” (recommended for state files)
5. Click “OK”

Step 2: Get Your S3 Bucket Credentials

1. Navigate to your bucket in Atlas Storage
2. Click on the bucket name to view details
3. Go to the “Details” tab
4. Copy the Access Key and Secret Key
5. Note your bucket name and region (if applicable)

Step 3: Configure Terraform Backend

Add the following backend configuration to your Terraform project. You can either:

Option A: Add to existing main.tf

Add this block at the top of your main.tf:

terraform {
  backend "s3" {
    bucket                      = "terraform-state"
    key                         = "keycloak/terraform.tfstate"
    region                      = "us-east-1"  # any AWS region string; the S3 SDK signs using SigV4 but Atlas ignores the region
    endpoint_url                = "https://s3.runatlas.is"
    access_key                  = var.aws_access_key
    secret_key                  = var.aws_secret_key
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    force_path_style            = true
  }
 
  required_providers {
    cloudstack = {
      source  = "cloudstack/cloudstack"
      version = "0.6.0-rc3"
    }
  }
}

Option B: Create separate backend.tf

terraform {
  backend "s3" {
    bucket                      = "terraform-state"
    key                         = "keycloak/terraform.tfstate"
    region                      = "us-east-1"
    endpoint_url                = "https://s3.runatlas.is"
    access_key                  = var.aws_access_key
    secret_key                  = var.aws_secret_key
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    force_path_style            = true
  }
}

Step 4: Add S3 Credentials to Variables

Add these variables to your variables.tf:

variable "aws_access_key" {
  description = "Atlas S3 access key"
  type        = string
  sensitive   = true
}
 
variable "aws_secret_key" {
  description = "Atlas S3 secret key"
  type        = string
  sensitive   = true
}

Add these credentials to your terraform.tfvars:

# Atlas S3 Configuration (for remote state)
aws_access_key = "your-s3-access-key"
aws_secret_key = "your-s3-secret-key"
 
# CloudStack Configuration
cloudstack_api_url    = "https://sky.runatlas.is/client/api"
cloudstack_api_key    = "your-cloudstack-api-key"
cloudstack_secret_key = "your-cloudstack-secret-key"
 
# Infrastructure Configuration
zone                      = "is1"
instance_service_offering = "Atlas.a5"
instance_template         = "Ubuntu 24.04 LTS"
network_offering          = "DefaultIsolatedNetworkOfferingWithSourceNatService"
environment               = "production"
 
# Keycloak Configuration
keycloak_admin_password = "YourSecureAdminPassword123!"
keycloak_db_password    = "YourSecureDBPassword456!"

Step 5: Initialize Terraform with Remote State

Initialize Terraform to configure the remote backend:

terraform init

Terraform will prompt you to confirm the migration to remote state. Type yes to proceed.

Step 6: Verify Remote State Configuration

Check that your state is now stored remotely:

terraform state pull

This will display the current state file content from your S3 bucket.

Best Practices

Security Considerations

• Private Bucket: Keep your state bucket private
• Versioning: Enable bucket versioning to track state changes

Organization

• Key Structure: Use descriptive key paths like project/environment/terraform.tfstate
• Separate Buckets: Consider separate buckets for different environments (dev/staging/prod) instead of path-based environments

Example: Complete Configuration

Here’s a complete example for a Keycloak deployment:

backend.tf:

terraform {
  backend "s3" {
    bucket        = "terraform-state"
    key           = "keycloak/production/terraform.tfstate"
    region        = "us-east-1"
    endpoint_url  = "https://s3.runatlas.is"
    access_key    = var.aws_access_key
    secret_key    = var.aws_secret_key
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    force_path_style            = true
  }
}

terraform.tfvars:

# Atlas CloudStack Configuration
cloudstack_api_url    = "https://sky.runatlas.is/client/api"
cloudstack_api_key    = "your-cloudstack-api-key"
cloudstack_secret_key = "your-cloudstack-secret-key"
 
# Atlas S3 Configuration (for remote state)
aws_access_key = "your-s3-access-key"
aws_secret_key = "your-s3-secret-key"
 
# Infrastructure Configuration
zone                      = "is1"
instance_service_offering = "Atlas.a5"
instance_template         = "Ubuntu 24.04 LTS"
environment               = "production"
 
# Keycloak Configuration
keycloak_admin_password = "YourSecureAdminPassword123!"
keycloak_db_password    = "YourSecureDBPassword456!"

Congratulations! You’ve successfully configured remote Terraform state storage with Atlas S3 🎉