TTL and Archiving

A common anti-pattern in database management is keeping data forever “just in case.” This leads to bloated tables, slower backups, and massive storage bills.

Time To Live (TTL) is DynamoDB’s built-in garbage collector. It allows you to define a specific timestamp when an item is no longer needed, and DynamoDB will delete it for you—for free.

1. How TTL Works

1. Attribute: You designate a specific attribute (e.g., expireAt) as the TTL attribute.
2. Value: The value must be a Number containing a standard Unix Epoch Timestamp (in seconds).
3. Process: A background process scans the table. If current_time > expireAt, the item is marked for deletion.
4. Deletion: The item is eventually removed from storage.

The “48-Hour” Rule

TTL is not real-time. DynamoDB guarantees that the item will be deleted within 48 hours of expiration.

• Querying: Filter expressions do not automatically hide expired items that haven’t been deleted yet. Your application logic should check if (item.expireAt < Now) just to be safe.
• Cost: The deletion consumes 0 WCUs. It is completely free.

---

2. Interactive: The Timeline of Death

Set an expiration time and watch how the item moves through the TTL lifecycle.

Created

Expires

Deleted

Item Status

Active

Visible to apps

---

3. The Archival Pipeline

Deleting data is good, but usually, businesses want to keep records for compliance (e.g., 7 years).

The Pattern: TTL + DynamoDB Streams + Lambda + S3.

1. TTL deletes the item.
2. DynamoDB Streams captures the deletion.
   • The stream record has a special userIdentity field.
   • If userIdentity.type == "Service" and principalId == "dynamodb.amazonaws.com", it was a TTL deletion.
3. Lambda filters for these specific records and writes the JSON content to an S3 Bucket (Glacier class for lowest cost).

---

4. Code Implementation

How to enable TTL on a table programmatically.

Java SDK v2 Go SDK v2

import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.*;

public class EnableTTL {
    public static void main(String[] args) {
        DynamoDbClient client = DynamoDbClient.create();

        try {
            // Enable TTL on the 'Sessions' table
            // using the attribute 'expireAt'
            client.updateTimeToLive(UpdateTimeToLiveRequest.builder()
                .tableName("Sessions")
                .timeToLiveSpecification(TimeToLiveSpecification.builder()
                    .enabled(true)
                    .attributeName("expireAt")
                    .build())
                .build());

            System.out.println("TTL Enabled. Background cleanup started.");
        } catch (DynamoDbException e) {
            System.err.println(e.getMessage());
        }
    }
}

package main

import (
	"context"
	"fmt"
	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

func main() {
	cfg, _ := config.LoadDefaultConfig(context.TODO())
	client := dynamodb.NewFromConfig(cfg)

	// Enable TTL
	_, err := client.UpdateTimeToLive(context.TODO(), &dynamodb.UpdateTimeToLiveInput{
		TableName: aws.String("Sessions"),
		TimeToLiveSpecification: &types.TimeToLiveSpecification{
			Enabled:       aws.Bool(true),
			AttributeName: aws.String("expireAt"),
		},
	})

	if err != nil {
		panic(err)
	}
	fmt.Println("TTL Enabled.")
}

[!TIP] Epoch Calculation: In Java, use Instant.now().getEpochSecond() + 3600 (for 1 hour). In Go, use time.Now().Unix() + 3600. Do not use milliseconds!

---

5. Summary

• Cost: Free deletion.
• Latency: Up to 48 hours.
• Archive: Use Streams to backup deleted data to S3.
• GSIs: TTL deletes items from GSIs automatically, also for free.