Why this matters
- S3-compatible object stores underpin StreamHub video uploads, user avatars, static assets, and ML training datasets.
- Understanding multipart upload, versioning, and lifecycle policies is essential for designing any blob-heavy system.
- Object storage optimises for throughput and durability, not low-latency random access — choose the right tier.
S3 core concepts
- Bucket — top-level container with regional placement and access policies.
- Object key — unique identifier within a bucket (
videos/2026/04/clip_8f2a.mp4). - Metadata — user-defined key-value pairs and system metadata (content-type, etag).
- Versioning — retain previous object versions on overwrite; protect against accidental deletion.
- Lifecycle rules — auto-transition to cheaper tiers or expire objects after N days.
Request flow
S3 object storage architecture
PUT /streamhub-media/videos/2026/04/clip_8f2a.mp4
Host: s3.us-east-1.amazonaws.com
Content-Type: video/mp4
Content-Length: 524288000
x-amz-storage-class: STANDARD
<binary body>
GET /streamhub-media/videos/2026/04/clip_8f2a.mp4
Host: s3.us-east-1.amazonaws.com
→ 200 OK
ETag: "d41d8cd98f00b204e9800998ecf8427e"
Content-Type: video/mp4
Multipart upload for large objects
Objects over 100 MB should use multipart upload: split into parts, upload in parallel, then assemble.
POST /streamhub-media/videos/large_clip.mp4?uploads
→ <UploadId>upload_xyz</UploadId>
PUT /streamhub-media/videos/large_clip.mp4?partNumber=1&uploadId=upload_xyz
PUT /streamhub-media/videos/large_clip.mp4?partNumber=2&uploadId=upload_xyz
...
POST /streamhub-media/videos/large_clip.mp4?uploadId=upload_xyz
<CompleteMultipartUpload>
<Part><PartNumber>1</PartNumber><ETag>"abc"</ETag></Part>
<Part><PartNumber>2</PartNumber><ETag>"def"</ETag></Part>
</CompleteMultipartUpload>
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 5 sections
- Durability and consistency
- Lifecycle policy example
- Presigned URLs for secure access
- Scale estimates
- Event-driven processing
Test yourself
Answer these before moving on — recall is what makes it stick.