Caching is one of the most effective ways to improve application performance while reducing costs. In this guide, I'll show you how to implement a cache-aside pattern using DynamoDB, ElastiCache Redis, AWS Lambda, and API Gateway - all provisioned with Terraform.

📘 Part 1: Infrastructure Setup

1. DynamoDB Table

First, let's create a Products table in DynamoDB:

resource "aws_dynamodb_table" "products" {
  name           = "Products"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "productId"

  attribute {
    name = "productId"
    type = "S"
  }
}

2. ElastiCache Redis Cluster

We'll deploy Redis inside a VPC for better security and performance:

resource "aws_elasticache_cluster" "products_cache" {
  cluster_id           = "products-cache"
  engine               = "redis"
  node_type            = "cache.t3.micro"
  num_cache_nodes      = 1
  parameter_group_name = "default.redis6.x"
  engine_version       = "6.x"
  port                 = 6379
  security_group_ids   = [aws_security_group.redis.id]
  subnet_group_name    = aws_elasticache_subnet_group.redis.name
}

resource "aws_elasticache_subnet_group" "redis" {
  name       = "redis-subnet-group"
  subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id]
}

3. Networking Configuration

Proper VPC setup is crucial:

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "private_a" {
  vpc_id            = aws_vpc.main.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-east-1a"
}

resource "aws_security_group" "redis" {
  name        = "redis-sg"
  description = "Allow access to Redis"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port   = 6379
    to_port     = 6379
    protocol    = "tcp"
    security_groups = [aws_security_group.lambda.id]
  }
}

📘 Part 2: Lambda Function Logic

Here's our Python Lambda function implementing the cache-aside pattern:

import os
import json
import boto3
import redis
from datetime import datetime

# Initialize clients
dynamodb = boto3.resource('dynamodb')
products_table = dynamodb.Table('Products')

# Redis connection
redis_client = redis.Redis(
    host=os.environ['REDIS_HOST'],
    port=6379,
    decode_responses=True
)

def lambda_handler(event, context):
    product_id = event['pathParameters']['productId']
    cache_key = f"product:{product_id}"

    # Try to get from Redis first
    cached_product = redis_client.get(cache_key)

    if cached_product:
        print("Cache hit!")
        return {
            'statusCode': 200,
            'body': cached_product
        }

    print("Cache miss - fetching from DynamoDB")
    # Get from DynamoDB
    response = products_table.get_item(Key={'productId': product_id})

    if 'Item' not in response:
        return {'statusCode': 404, 'body': 'Product not found'}

    product = response['Item']
    product_json = json.dumps(product)

    # Cache with 5 minute TTL
    redis_client.setex(cache_key, 300, product_json)

    return {
        'statusCode': 200,
        'body': product_json
    }

📘 Part 3: API Gateway Configuration

Let's expose our Lambda through API Gateway:

resource "aws_api_gateway_rest_api" "products_api" {
  name = "products-api"
}

resource "aws_api_gateway_resource" "product" {
  rest_api_id = aws_api_gateway_rest_api.products_api.id
  parent_id   = aws_api_gateway_rest_api.products_api.root_resource_id
  path_part   = "product"
}

resource "aws_api_gateway_resource" "product_id" {
  rest_api_id = aws_api_gateway_rest_api.products_api.id
  parent_id   = aws_api_gateway_resource.product.id
  path_part   = "{productId}"
}

resource "aws_api_gateway_method" "get_product" {
  rest_api_id   = aws_api_gateway_rest_api.products_api.id
  resource_id   = aws_api_gateway_resource.product_id.id
  http_method   = "GET"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "lambda" {
  rest_api_id = aws_api_gateway_rest_api.products_api.id
  resource_id = aws_api_gateway_resource.product_id.id
  http_method = aws_api_gateway_method.get_product.http_method

  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = aws_lambda_function.get_product.invoke_arn
}

📘 Part 4: Monitoring and Optimization

1. Adding TTLs

We already implemented TTLs in our Lambda function (setex with 300 seconds), but let's add CloudWatch metrics to track cache performance:

from aws_lambda_powertools import Metrics
metrics = Metrics()

def lambda_handler(event, context):
    # ... existing code ...

    if cached_product:
        metrics.add_metric(name="CacheHits", unit="Count", value=1)
        # ... return cached product ...
    else:
        metrics.add_metric(name="CacheMisses", unit="Count", value=1)
        # ... fetch from DynamoDB ...

2. Terraform for Monitoring

Add CloudWatch alarms and dashboards:

resource "aws_cloudwatch_dashboard" "cache" {
  dashboard_name = "cache-performance"

  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        x      = 0
        y      = 0
        width  = 12
        height = 6

        properties = {
          metrics = [
            ["AWS/Lambda", "CacheHits", "FunctionName", aws_lambda_function.get_product.function_name],
            ["AWS/Lambda", "CacheMisses", "FunctionName", aws_lambda_function.get_product.function_name]
          ]
          period = 300
          stat   = "Sum"
          region = "us-east-1"
          title  = "Cache Performance"
        }
      }
    ]
  })
}

3. Complete Terraform Workflow

For a production setup, add a CI/CD pipeline:

resource "aws_codepipeline" "deploy_pipeline" {
  name     = "products-api-deployment"
  role_arn = aws_iam_role.codepipeline.arn

  artifact_store {
    location = aws_s3_bucket.artifacts.bucket
    type     = "S3"
  }

  stage {
    name = "Source"
    action {
      name             = "Source"
      category         = "Source"
      owner            = "ThirdParty"
      provider         = "GitHub"
      version          = "1"
      output_artifacts = ["source_output"]

      configuration = {
        Owner      = "your-github-org"
        Repo       = "products-api"
        Branch     = "main"
        OAuthToken = var.github_token
      }
    }
  }

  stage {
    name = "Terraform"
    action {
      name             = "Apply"
      category         = "Build"
      owner            = "AWS"
      provider         = "CodeBuild"
      input_artifacts  = ["source_output"]
      version          = "1"

      configuration = {
        ProjectName = aws_codebuild_project.terraform.name
      }
    }
  }
}

Results and Observations

After implementing this architecture, you should see:

  • Average latency reduction from ~100ms (DynamoDB) to ~5ms (Redis) for cache hits
  • Reduced DynamoDB RCU consumption (and costs) by 80-90% for frequently accessed items
  • More consistent performance under load

Final Thoughts

This cache-aside pattern is just one of many ways to optimize DynamoDB performance. For production workloads, consider:

  • Adding write-through caching for data modifications
  • Implementing cache invalidation strategies
  • Monitoring Redis memory usage and eviction policies
  • Considering DAX for alternative DynamoDB caching

The complete Terraform code is available in this GitHub repo.

Have you implemented similar caching patterns? Share your experiences in the comments!