Webhook idempotency and the transactional outbox
Designing for at-least-once delivery: deduplication keys, safe retries, and publishing after commit without dual-write hazards.
Inbound webhooks are at least once. Networks retry; providers replay. If “capture funds” or “activate subscription” executes twice, you have a reconciliation problem, not a logging problem. Production systems need idempotent handlers and, when side effects leave the database, a reliable outbox.
Deduplicate on provider event identity
Persist a natural key before executing side effects. On conflict, return success—duplicate delivery should be indistinguishable from first success for the caller.
CREATE TABLE webhook_events (
id BIGSERIAL PRIMARY KEY,
provider TEXT NOT NULL,
event_id TEXT NOT NULL,
received_at TIMESTAMPTZ NOT NULL DEFAULT now(),
payload_hash TEXT NOT NULL,
UNIQUE (provider, event_id)
);
// Example: pg.Pool — substitute your client API.
async function acceptOnce(
provider: string,
eventId: string,
payloadHash: string,
): Promise<"new" | "duplicate"> {
const r = await pool.query(
`INSERT INTO webhook_events (provider, event_id, payload_hash)
VALUES ($1, $2, $3)
ON CONFLICT (provider, event_id) DO NOTHING
RETURNING id`,
[provider, eventId, payloadHash],
);
return r.rowCount ? "new" : "duplicate";
}
Outbox for downstream messaging
If processing must emit to a queue or search index, write an outbox row in the same transaction as your domain mutation, then let a relay publish—avoiding “committed in DB, message lost before send.”
Takeaway: Model retries as normal; design handlers so repetition is safe and side effects are observable.