Introducing s3-compliance: An OpenTofu Module for Secure and Standardized S3 Buckets

In this blog post, I give you an overview of the s3-compliance OpenTofu module, for provisioning and managing Amazon S3 buckets, while adhering to data classification standards. What is s3-compliance? S3-compliance is an opinionated OpenTofu module designed to: Create and manage one or more S3 buckets using a parameterized deployment. Embed organizational standards and policies into your infrastructure provisioning, based on data classifications. Provide flexibility for users to specify certain settings while enforcing pre-approved defaults. Not all types of data require every S3 bucket feature such as versioning or object lock. The s3-compliance module addresses these diverse needs by allowing different sets of configurations for different data classifications. This tailored approach ensures that buckets are configured with only the necessary features for their specific data classification, striking a balance between security, efficiency and cost. With this module, teams can focus on their work while ensuring their provisioned S3 buckets align with organization standards and recommended practices. Data Classifications The s3-compliance module uses data classifications (e.g., public, internal, compliance) to determine which features to mandate. For example: Public Data: Minimal security features, with public access intentionally enabled where appropriate. Internal Data: Encryption using KMS and logging are mandatory; versioning is optional. Regulated Data: Strict compliance requirements, including encryption, versioning, object lock, and logging. By matching user-provided configurations to data classifications, the module allows flexibility for some settings, while enforcing others, thereby ensuring efficiency and compliance. Compliance File The s3-compliance.tf file embedded in the module defines organizational standards for various data classifications, including: Public access restrictions Encryption settings CloudTrail Logging Versioning Object locks for immutable storage Security teams vet this file to ensure compliance with organizational and regulatory requirements. A workflow may be triggered upon any changes to this compliance file to solicit the security team's approval. Alternatively, the security team may host this file in their own repository and make it available as a module output for consumption by the s3-compliance module. Input parameters Given below is the variable structure for the input parameters, showing that the s3_buckets variable accepts a list of S3 buckets for provisioning. Most of the optional configurations have defaults pre-defined as per the s3-compliance file. variable "s3_buckets" { description = "List of S3 bucket configurations." type = list(object({ # Basic configuration name = string data_classification = string # Defaults are "public", "internal" and "compliance" public_access_enabled = optional(bool) versioning_enabled = optional(bool) logging_enabled = optional(bool) tags = optional(map(string), {}) # Encryption settings kms_master_key_id = optional(string) # Use S3-managed keys by default compliance_standard = optional(string) # e.g., "PCI-DSS", "HIPAA", "ISO27001" # Object Lock settings object_lock = optional(object({ mode = optional(string) # "GOVERNANCE" or "COMPLIANCE" retention_days = optional(number) # Number of days to retain objects in locked state }), null) # Lifecycle configuration lifecycle_transitions = optional(object({ intelligent_tiering_days = optional(number, null) # Days for transitioning objects to the Intelligent Tiering class glacier_ir_days = optional(number, null) # Days for transitioning objects to the Glacier Instant Retrieval class glacier_fr_days = optional(number, null) # Days for transitioning objects to the Glacier Flexible Retrieval class glacier_da_days = optional(number, null) # Days for transitioning objects to the Glacier Deep Archive class }), null) expiration_days = optional(number, null) # Expiration after the latest transition })) Example Usage The s3-compliance module may be used as shown below. # Provision S3 buckets for a sales application module "sales-s3" { source = "git::https://github.com/cybergavin/terraform-aws-s3-compliance.git?ref=57e686755fd9c0c89b57ba1babe3f741ad6d6515 org = var.org app_id = var.app_id environment = var.environment s3_buckets = var.s3_buckets s3_logs = var.s3_logs global_tags = var.global_tags } An example s3_buckets variable for provisioning three S3 buckets for a sales application, is given below. s3_buckets = [ { name = "catalogs" data_classification = "public" public_access_enabled = true tags = { "description" = "Sales product

variable "s3_buckets" {
  description = "List of S3 bucket configurations."
  type = list(object({
    # Basic configuration
    name                  = string
    data_classification   = string # Defaults are "public", "internal" and "compliance"
    public_access_enabled = optional(bool)
    versioning_enabled    = optional(bool)
    logging_enabled       = optional(bool)
    tags                  = optional(map(string), {})

    # Encryption settings
    kms_master_key_id   = optional(string) # Use S3-managed keys by default
    compliance_standard = optional(string) # e.g., "PCI-DSS", "HIPAA", "ISO27001"

    # Object Lock settings
    object_lock = optional(object({
      mode           = optional(string) # "GOVERNANCE" or "COMPLIANCE"
      retention_days = optional(number) # Number of days to retain objects in locked state
    }), null)

    # Lifecycle configuration
    lifecycle_transitions = optional(object({
      intelligent_tiering_days = optional(number, null) # Days for transitioning objects to the Intelligent Tiering class
      glacier_ir_days          = optional(number, null) # Days for transitioning objects to the Glacier Instant Retrieval class
      glacier_fr_days          = optional(number, null) # Days for transitioning objects to the Glacier Flexible Retrieval class
      glacier_da_days          = optional(number, null) # Days for transitioning objects to the Glacier Deep Archive class
    }), null)

    expiration_days = optional(number, null) # Expiration after the latest transition
  }))

# Provision S3 buckets for a sales application
module "sales-s3" {
  source = "git::https://github.com/cybergavin/terraform-aws-s3-compliance.git?ref=57e686755fd9c0c89b57ba1babe3f741ad6d6515
  org         = var.org
  app_id      = var.app_id
  environment = var.environment
  s3_buckets  = var.s3_buckets
  s3_logs     = var.s3_logs
  global_tags = var.global_tags
}

s3_buckets = [
    {
        name = "catalogs"
        data_classification = "public"
        public_access_enabled = true
        tags = {
            "description" = "Sales product catalogs"
        }
    },
    {
        name = "inventory"
        data_classification = "internal"
        tags = {
            "description" = "Sales inventory"
        }
        lifecycle_transitions = {
            intelligent_tiering_days = 180
        }
        expiration_days = 365
    },
    {
        name = "payment"
        data_classification = "compliance"
        compliance_standard = "PCI-DSS"
        tags = {
            "description" = "Sales payment transactions"
        }
        object_lock = {
            mode = "COMPLIANCE"
            retention_days = "2555"
        }
        lifecycle_transitions = {
            intelligent_tiering_days = 180
            glacier_ir_days = 365
            glacier_fr_days = 730
        }
        expiration_days = 2555
    }
]

Related Posts

Node v23.6.0 (Current)

Jan 14, 2025

Postgres vs. MySQL

Jan 16, 2025

TS1103: 'for await' loops are only allowed within async...

Jan 15, 2025