Webhooks and Async Callbacks

Notify external systems reliably with signed payloads, retries and delivery guarantees.

AdvancedTheory10 min read

Read these first

Event-Driven Architecture

StreamHub partners (OBS plugins, analytics dashboards, clip bots) register webhook URLs. When a stream goes live or a clip is created, StreamHub POSTs a signed JSON payload to each registered endpoint — with retries, backoff, and dead-letter handling when partners are down.

Webhooks vs polling

Aspect	Webhooks (push)	Polling (pull)
Latency	Near real-time delivery	Bounded by poll interval
Efficiency	Events sent only when they happen	Empty polls waste bandwidth
Complexity	Retry, signing, idempotency on sender	Simple client loop
Partner downtime	Missed deliveries without retry/DLQ	Partner catches up on next poll
StreamHub default	Primary for registered integrations	Fallback + public API sync

Latency
Webhooks (push)Near real-time delivery
Polling (pull)Bounded by poll interval
Efficiency
Webhooks (push)Events sent only when they happen
Polling (pull)Empty polls waste bandwidth
Complexity
Webhooks (push)Retry, signing, idempotency on sender
Polling (pull)Simple client loop
Partner downtime
Webhooks (push)Missed deliveries without retry/DLQ
Polling (pull)Partner catches up on next poll
StreamHub default
Webhooks (push)Primary for registered integrations
Polling (pull)Fallback + public API sync

Webhook delivery architecture

MSK event → dispatcher → HTTPS callback with SQS DLQ on failure.

Delivery pipeline

Event trigger: Internal event (stream.started) enters webhook dispatch queue.
Subscription lookup: Find all registered endpoints for this event type + streamer.
Delivery worker: HTTP POST with signed payload, timeout, and retry policy.
Delivery log: Record attempt, status code, latency for partner debugging.
DLQ: After max retries, park failed deliveries for manual replay.

Payload and security

Java

POST /hooks/streamhub HTTP/1.1
Host: partner.example.com
Content-Type: application/json
X-StreamHub-Signature: sha256=abc123def456...
X-StreamHub-Delivery-Id: del_7f8e9a0b
X-StreamHub-Timestamp: 1727314860

{
  "event": "stream.started",
  "data": {
    "stream_id": "live_9912",
    "streamer_id": "sh_4420"
  }
}

Security checklist

HMAC signature: Partner verifies X-StreamHub-Signature with shared secret.
Timestamp tolerance: Reject payloads older than 5 minutes to prevent replay attacks.
HTTPS only: Never deliver to plain HTTP endpoints in production.
Delivery ID: Unique per attempt; partner deduplicates with idempotency store.

Java

import hmac, hashlib

def sign_payload(secret: str, timestamp: str, body: bytes) -> str:
    message = f"{timestamp}.{body.decode()}"
    digest = hmac.new(secret.encode(), message.encode(), hashlib.sha256).hexdigest()
    return f"sha256={digest}"

Retry strategy

Attempt	Delay	Action on failure
1	Immediate	Retry on 5xx or timeout
2	1 minute	Exponential backoff
3	5 minutes	Exponential backoff
4	30 minutes	Exponential backoff
5	2 hours	Move to DLQ; alert partner

1
DelayImmediate
Action on failureRetry on 5xx or timeout
2
Delay1 minute
Action on failureExponential backoff
3
Delay5 minutes
Action on failureExponential backoff
4
Delay30 minutes
Action on failureExponential backoff
5
Delay2 hours
Action on failureMove to DLQ; alert partner

Cap at 5 attempts over ~2.5 hours — then DLQ and notify partner dashboard.

Partner best practices

What partners should implement

Return 2xx within 5 seconds — process async if work is heavy.
Verify signature before processing; reject stale timestamps.
Deduplicate by X-StreamHub-Delivery-Id — at-least-once delivery is guaranteed.
Expose a health endpoint; StreamHub pauses delivery to unhealthy endpoints.

Quick recall

Everything you need if you only revisit this box.

Webhooks push events to partner URLs; polling pulls on an interval — webhooks are lower latency.
Sign payloads with HMAC; include delivery ID and timestamp for verification and dedup.
Retry with exponential backoff; DLQ after max attempts; circuit-break unhealthy endpoints.
Partners must respond 2xx quickly and process idempotently by delivery ID.
HTTPS only; reject replayed payloads via timestamp tolerance window.
Offer polling as fallback for partners who cannot receive inbound HTTP.

Test yourself

Answer these before moving on — recall is what makes it stick.