Amazon ElastiCache Overview

• Definition: Fully managed implementations of two popular in-memory data stores: Redis and Memcached.
• Makes it easy to deploy and run Memcached or Redis protocol-compliant server nodes in the cloud.
• Purpose: Significantly improve latency and throughput for read-heavy or compute-intensive application workloads.
• Best for scenarios where the DB load is based on Online Analytics Processing (OLAP) transactions.
• Offers push-button scalability for memory, writes, and reads.
• In-memory key/value store: Not persistent in the traditional sense (though Redis offers persistence).
• Billing: By node size and hours of use.
• Can be On-Demand or Reserved Instances (but not Spot Instances). (Exam Tip!)
• Can be used for storing session state.

Core Concepts

• Node: Fixed-sized chunk of secure, network-attached RAM; smallest building block. Each node has its own DNS name and port. Failed nodes are automatically replaced.
• Cluster: Collection of one or more nodes using the same caching engine.
• Endpoint: A node or cluster’s unique address, used by applications to connect.
• Maintenance Windows: Can be defined for software patching.

ElastiCache Engine Comparison (Memcached vs. Redis)

Key Use Cases for ElastiCache (General):

• Web session store (e.g., storing user session info for load-balanced web servers).
• Database caching (offload reads from RDS/DynamoDB).
• Leaderboards (live leaderboards for mobile apps).
• Streaming data dashboards (landing spot for sensor data).
• Caching results of computationally intensive calculations.

Memcached Specifics

• Simplest model, widely adopted memory object caching system.
• Multi-threaded.
• Scales out/in by adding/removing nodes (horizontal scaling).
• Scales up/down by changing node family/type (vertical scaling).
• Ideal front-end for data stores (RDS, DynamoDB).
• Use Cases: Cache DB contents, dynamically generated web pages, transient session data, high-frequency counters.
• Limits: Max 100 nodes per region, 1-20 nodes per cluster (soft limits).
• Integrates with SNS for node failure/recovery notifications.
• Supports auto-discovery for nodes.
• No native multi-AZ failover or replication. Data loss on node failure.
• Does not support snapshots or backup/restore.
• Nodes in a cluster can span AZs, but this is for distribution, not failover.

Redis Specifics

• Open-source in-memory key-value store.
• Supports more complex data structures (sorted sets, lists, hashes).
• Data can be persistent (can be used as a datastore).
• Not multi-threaded.
• Scales by adding shards (in cluster mode), not just nodes.
• Supports master/replica replication and multi-AZ for cross-AZ redundancy.
• Supports automatic failover and backup/restore.
• A Redis shard is a subset of the cluster’s keyspace, containing a primary node and 0+ read replicas.
• Supports automatic and manual snapshots (to S3). Backups include cluster data and metadata.
• You can restore data by creating a new Redis cluster from a backup.
• During backup, no CLI/API operations on the cluster.
• Automated backups are enabled by default (deleted with Redis deletion).
• Snapshots can be moved between regions by exporting them before moving.

Clustering Mode Disabled (Redis)

• Only one shard.
• One shard can have one read/write primary node and 0-5 read-only replicas.
• Replicas can be distributed over multiple AZs in the same region.
• Replication from primary is asynchronous.

Clustering Mode Enabled (Redis)

• Can have up to 15 shards.
• Each shard can have one primary node and 0-5 read-only replicas.
• Snapshots are best taken from read replicas to minimize impact on primary.

Multi-AZ Failover (Redis)

• ElastiCache detects failures and automatically promotes the replica with the lowest replica lag.
• DNS records remain the same but point to the new primary's IP.
• Other replicas sync with the new primary.
• Enabling both cluster mode and multi-AZ failover provides a fully automated, fault-tolerant Redis implementation.

Caching Strategies

• These strategies define how data is loaded into and updated in the cache.

Lazy Loading (Cache Aside)

• Loads data into the cache only when necessary (on a cache miss).
• Flow: Application checks cache → if miss, fetches from DB → writes to cache → returns data.
• Benefit: Avoids filling cache with unrequested data.
• Drawback: Data in cache can become stale if not combined with TTL or other strategies. Initial requests for data will have higher latency (cache miss).

Write-Through

• Cache is updated whenever a new write or update is made to the underlying database.
• Flow: Application writes to cache → cache writes to DB → returns success.
• Benefit: Cache data remains up-to-date.
• Drawback: Can add wait time to write operations. Without TTL, can end up with a lot of cached data that is never read.

Dealing with Stale Data – Time to Live (TTL)

• Specifies the number of seconds until a key (data) expires in the cache.
• Mitigates drawbacks of Lazy Loading and Write-Through by ensuring data is periodically refreshed.
• Lazy Loading treats an expired key as a cache miss, prompting retrieval from DB and writing back with a new TTL.

Monitoring and Reporting

• All ElastiCache actions are logged by AWS CloudTrail (auditing).
• CloudWatch provides metrics for performance insight.

Memcached Metrics (CloudWatch)

• CPUUtilization: Host-level (can be high due to multi-threading). Scale up/out if too high.
• SwapUsage: Host-level (should not exceed 50 MB). Increase ConnectionOverhead if high.
• Evictions: Cache engine metric. Scale up/out if high.
• CurrConnections: Cache engine metric. Investigate application behavior if increasing.

Redis Metrics (CloudWatch)

• EngineCPUUtilization: Redis engine thread CPU (since Redis is single-threaded).
• MemoryFragmentationRatio: Memory allocation efficiency (recommended > 1.0).
• CacheHits: Successful read lookups.
• CacheMisses: Unsuccessful read lookups.
• CacheHitRate: Usage efficiency (if low, keys are evicted/expired).
• CurrConnections: Number of client connections (excluding replicas).

Logging and Auditing

• All Amazon ElastiCache actions are logged by AWS CloudTrail.
• Log entries provide identity information (root, IAM user, role, other AWS service).

Authorization and Access Control

• Access controlled by VPC security groups and subnet groups (when deployed in a VPC).
• Subnet Groups: Collection of subnets designated for your ElastiCache Cluster. Required for VPC deployment.
• Cannot move an existing ElastiCache Cluster from outside VPC into a VPC.
• When not in VPC (legacy), Cache Security Groups control access (link to EC2 Security Groups).
• Use identity-based policies (IAM) for fine-grained access.
• Redis supports Redis Auth (requiring a token/password for clients).

Charges

• Pricing per Node-hour consumed. Partial hours billed as full.
• No charge for data transfer between EC2 and ElastiCache within the same Availability Zone.

High Availability for ElastiCache

• Memcached:
  • Does not support replication; node failure results in data loss.
  • Minimize data loss by using multiple nodes and launching them across multiple AZs.
• Redis:
  • Use multiple nodes in each shard and distribute across multiple AZs.
  • Enable Multi-AZ on the replication group for automatic failover if primary nodes fail.
  • Schedule regular backups of your Redis cluster.

Important Exam Tips & Tricks

• Managed Service: ElastiCache is fully managed; you don't manage the underlying servers.
• In-Memory: Primarily for caching, not persistent storage (though Redis has persistence options).
• No Spot Instances: Cannot use Spot Instances for ElastiCache. (Exam Tip!)
• Memcached vs. Redis (Key Differences!): This is a very common exam topic.
  • Memcached: Simpler, multi-threaded, scales horizontally (add/remove nodes), no replication/persistence/HA.
  • Redis: More features (complex data types, Pub/Sub, persistence, encryption), single-threaded, master/replica replication, Multi-AZ failover, snapshots.
• Redis Persistence: Remember Redis can persist data to S3 via snapshots, making it usable as a durable in-memory datastore.
• Redis Scaling: Redis scales by adding shards (in cluster mode) and read replicas.
• Access Control: Access is primarily via Security Groups and Subnet Groups within a VPC. IAM policies control who can manage ElastiCache resources.
• Caching Strategies (Lazy Loading vs. Write-Through):
  • Lazy Loading (Cache Aside): Read-through, good for avoiding filling cache with unused data, but initial read latency is higher, and data can become stale.
  • Write-Through: Writes to cache and DB simultaneously, ensures cache is always fresh, but adds write latency.
  • TTL mitigates stale data issues for both.
• DAX vs. ElastiCache: DAX is a specific caching service for DynamoDB only, supports write-through caching only (no lazy loading). ElastiCache is general-purpose. (Exam Tip!)
• Data Loss (Memcached): Be aware that Memcached node failure leads to data loss due to no replication.
• CloudTrail: All ElastiCache API calls are logged in CloudTrail.