Skip to main content
PreparationZone Logo
PrepZone

S3 Object Storage

Design blob storage with durability, versioning, lifecycle policies and multipart uploads.

Pass checkpoint below

Why this matters

  • S3-compatible object stores underpin StreamHub video uploads, user avatars, static assets, and ML training datasets.
  • Understanding multipart upload, versioning, and lifecycle policies is essential for designing any blob-heavy system.
  • Object storage optimises for throughput and durability, not low-latency random access — choose the right tier.

S3 core concepts

  • Bucket — top-level container with regional placement and access policies.
  • Object key — unique identifier within a bucket (videos/2026/04/clip_8f2a.mp4).
  • Metadata — user-defined key-value pairs and system metadata (content-type, etag).
  • Versioning — retain previous object versions on overwrite; protect against accidental deletion.
  • Lifecycle rules — auto-transition to cheaper tiers or expire objects after N days.

Request flow

S3 object storage architecture

PUTCLIENT
Client upload
COMPUTE
EKS APIpresigned URL
STORAGE
Amazon S3Std → IA → Glac…
NETWORK
CloudFrontdelivery
CLIENT
Downloaders
Presigned uploads → lifecycle tiers → CloudFront delivery → Glacier archive.
Java
PUT /streamhub-media/videos/2026/04/clip_8f2a.mp4
Host: s3.us-east-1.amazonaws.com
Content-Type: video/mp4
Content-Length: 524288000
x-amz-storage-class: STANDARD

<binary body>
Java
GET /streamhub-media/videos/2026/04/clip_8f2a.mp4
Host: s3.us-east-1.amazonaws.com

→ 200 OK
  ETag: "d41d8cd98f00b204e9800998ecf8427e"
  Content-Type: video/mp4

Multipart upload for large objects

Objects over 100 MB should use multipart upload: split into parts, upload in parallel, then assemble.

Java
POST /streamhub-media/videos/large_clip.mp4?uploads
→ <UploadId>upload_xyz</UploadId>

PUT /streamhub-media/videos/large_clip.mp4?partNumber=1&uploadId=upload_xyz
PUT /streamhub-media/videos/large_clip.mp4?partNumber=2&uploadId=upload_xyz
...

POST /streamhub-media/videos/large_clip.mp4?uploadId=upload_xyz
<CompleteMultipartUpload>
  <Part><PartNumber>1</PartNumber><ETag>"abc"</ETag></Part>
  <Part><PartNumber>2</PartNumber><ETag>"def"</ETag></Part>
</CompleteMultipartUpload>

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 5 sections

  • Durability and consistency
  • Lifecycle policy example
  • Presigned URLs for secure access
  • Scale estimates
  • Event-driven processing

Test yourself

Answer these before moving on — recall is what makes it stick.