Open chat

← All posts

backenddistributed-systems

Webhook idempotency and the transactional outbox

Designing for at-least-once delivery: deduplication keys, safe retries, and publishing after commit without dual-write hazards.

Inbound webhooks are at least once. Networks retry; providers replay. If “capture funds” or “activate subscription” executes twice, you have a reconciliation problem, not a logging problem. Production systems need idempotent handlers and, when side effects leave the database, a reliable outbox.

Deduplicate on provider event identity

Persist a natural key before executing side effects. On conflict, return success—duplicate delivery should be indistinguishable from first success for the caller.

CREATE TABLE webhook_events (
  id           BIGSERIAL PRIMARY KEY,
  provider     TEXT NOT NULL,
  event_id     TEXT NOT NULL,
  received_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
  payload_hash TEXT NOT NULL,
  UNIQUE (provider, event_id)
);
// Example: pg.Pool — substitute your client API.
async function acceptOnce(
  provider: string,
  eventId: string,
  payloadHash: string,
): Promise<"new" | "duplicate"> {
  const r = await pool.query(
    `INSERT INTO webhook_events (provider, event_id, payload_hash)
     VALUES ($1, $2, $3)
     ON CONFLICT (provider, event_id) DO NOTHING
     RETURNING id`,
    [provider, eventId, payloadHash],
  );
  return r.rowCount ? "new" : "duplicate";
}

Outbox for downstream messaging

If processing must emit to a queue or search index, write an outbox row in the same transaction as your domain mutation, then let a relay publish—avoiding “committed in DB, message lost before send.”

Takeaway: Model retries as normal; design handlers so repetition is safe and side effects are observable.

← Back to portfolio