Skip to main content
RoutMe
All posts
Engineering3 min read

Your webhook will be delivered twice. Plan for it.

Duplicate deliveries, out-of-order events, and the 20-second timeout are not edge cases on the WhatsApp Cloud API. They are Tuesday. Here is the handling that keeps them boring.

JJ

Jesbin James

Engineering

Every messaging integration starts with a webhook endpoint that assumes each event arrives once, in order, and is processed instantly. Every one of those assumptions is wrong, and the bugs they produce are the kind that only appear under load, which is to say in front of customers.

Acknowledge first, work later

Meta expects a 200 quickly, treat your budget as a couple of seconds, not the full timeout, and it retries anything slower. If you call a model, hit your CRM, and send a reply before responding, you will breach that on a slow day. The retry then arrives while the first attempt is still running, and now you have two agents answering the same customer.

@app.post("/webhooks/whatsapp")
async def receive(request: Request, tasks: BackgroundTasks):
    body = await request.body()
    if not verify_signature(request.headers["X-Hub-Signature-256"], body):
        return Response(status_code=403)

    tasks.add_task(process, json.loads(body))
    return Response(status_code=200)   # ack in milliseconds, not seconds

Verify the signature before you parse, and verify it against the raw bytes. Re-serialising the JSON changes the payload and the HMAC will never match.

Deduplicate on the message ID

A retry is not a new message, but it looks exactly like one. Every inbound message carries an ID that is stable across delivery attempts, so use it as an idempotency key and let the database enforce it rather than checking first and inserting after.

INSERT INTO processed_messages (message_id, received_at)
VALUES ($1, now())
ON CONFLICT (message_id) DO NOTHING
RETURNING message_id;

No row returned means you have seen it before, so stop. Read-then-write loses this race under concurrent retries; a unique constraint does not.

Status events arrive out of order

Sent, delivered, and read are separate events, and they do not queue politely. A delivered event overtaking a read event will walk your status backwards if you write whatever arrived last.

Rank the statuses and only ever move forward. The same applies to failed, which should win regardless of arrival order, because a message that failed did not later become delivered.

RANK = {"sent": 1, "delivered": 2, "read": 3}

UPDATE messages
SET status = $2
WHERE id = $1
  AND rank_of(status) < rank_of($2);   -- monotonic, order-independent

Make sends idempotent too

The failure that actually hurts is on the outbound side: you send, the connection drops before the response arrives, your retry logic fires, and the customer gets the same message twice. From your side it looks like one send with a network error.

  • Generate a key per logical send, conversation ID plus step ID works, and record it before the request leaves.
  • On retry, look up the key first. If a message ID came back, the send succeeded and the error was in the response, not the delivery.
  • Back off exponentially with jitter. Synchronised retries after an outage are how you turn a blip into a rate limit.
  • Treat 429 and 5xx as retryable, and 4xx as a bug in your payload. Retrying a malformed template forever is a common way to burn a rate limit budget.

Order per conversation, not globally

Processing everything on one worker gives you ordering and no throughput. Processing everything in parallel gives you throughput and interleaved replies within a single conversation, which customers notice immediately.

Partition by conversation. Messages from one customer are handled in order by one consumer; different customers run in parallel. This is the single highest-value structural decision in a messaging backend, and it is painful to retrofit.

  • MCP
  • Integrations
  • WhatsApp Cloud API
  • Reliability
JJ

Jesbin James

Engineering

Writes about integrations, agent tooling, and the parts of the WhatsApp Cloud API that only show up in production.

See it on your own data

Connect a system, drop an agent node into a flow, and watch it answer from what is actually true in your systems right now.