PSP Outage Playbook: The First 60 Minutes

The stakes scale with your volume, and history says don't round the risk to zero: a widely cited two-hour Stripe outage in 2019 left merchants without fallback routing unable to transact at all, with one startup publicly estimating roughly $100k in lost sales from that single incident (as recounted by the Merchant Risk Council). More recently, Stripe's own failover resource cites 2025 survey data: 92% of enterprise e-commerce businesses experienced payment outages or disruptions within two years. This playbook exists because "rare" and "inevitable" are both true.

Minutes 0-10: Confirm and classify

Trust your metrics over their status page. Provider status pages lag, sometimes significantly. Your first signal should be your own: authorization success rate dropping, API error spikes, latency, timeout clusters. If you don't have those metrics independently, that's the first post-incident action item. (For the self-reported view across every major provider at once, we maintain a live provider status radar, with that caveat attached.)

Classify the failure. It changes everything downstream:

• Full outage: API unreachable or erroring across the board
• Partial degradation: elevated declines or latency on specific methods, regions or endpoints (more common and more confusing than full outages)
• Async-only failure: payments succeed but webhooks/events stall. Checkout looks fine while billing state silently drifts (see why webhooks fail silently)
• It's actually you: your deploy, your DNS, your expired credential. Check the last change on your side before blaming the provider.

Open an incident. Named commander, a dedicated channel, a scribe from minute one. Timestamps you record now are the postmortem, and for regulated entities, potentially reportable-incident evidence.

Minutes 10-25: Contain

Goal: stop making it worse while you decide what to do.

• Pause aggressive retries. Hammering a degraded provider with retry storms extends their recovery and can trip your own rate limits or fraud flags. Cap or pause automated retry jobs.
• Defer batch work. Scheduled billing runs, subscription renewals, payout jobs. Postpone anything that can wait an hour rather than feeding it into a failing pipeline. Failed renewals during an outage become dunning noise and involuntary-churn risk later.
• Queue, don't drop. Where architecture allows, capture intents/orders and queue them for later processing instead of hard-failing checkout. Sales deferred beat sales lost.
• Snapshot state. Note in-flight volumes and the exact degradation window; reconciliation will need the boundaries.

Minutes 25-40: Communicate

Silence costs more trust than the outage. Three audiences, in order:

1. Support first. They're already seeing tickets. Ship the pre-drafted macro: what's happening, what customers should(n't) do, where updates appear.
2. Customers, honestly and briefly. Status-page entry or banner: payments are affected, cards were not charged where checkout failed (verify before promising), retry guidance, next-update time. Never speculate about the provider's root cause.
3. Internal stakeholders. One-line updates on a fixed cadence to leadership and finance. Revenue is at stake and they'll ask; better on your schedule.

If you're in scope for DORA or similar regimes, loop compliance now: payment incidents can carry reporting clocks (DORA guide).

Minutes 40-60: The failover decision

The hardest call of the hour, so make it mechanical:

Fail over if, and only if, you have a backup path that has been validated under test, the failure classification matches what the backup covers (a backup that can't charge vaulted tokens doesn't fix a renewals-window outage), the trigger criteria you wrote in calm conditions are met, and the named decision-owner says go.

Do not fail over if the backup has never carried live traffic, its config parity is unknown, or webhook consumers for it are half-built. An unvalidated switch during an incident converts one outage into two: the provider's, then yours, which means duplicate charges, missed events and a reconciliation crater that outlives the outage by weeks. Queuing transactions and waiting out a 90-minute outage is often the genuinely better decision; make it consciously, not by paralysis.

If you fail over: shift gradually if tooling allows, watch backup auth rates against baseline in real time, and log every routed transaction. You'll reconcile by provider later.

After the first hour: recovery is where quiet damage happens

• Drain deliberately. Release queued work in controlled batches; a recovery thundering-herd can re-degrade a fragile provider.
• Reconcile the window. Diff provider records against your ledger for the full degradation window, both providers if you switched. Refunds, duplicates and orphaned intents cluster at the edges of the window.
• Replay events. Pull missed webhooks via the provider's event API/dashboard and re-process; billing state that drifted during the outage won't fix itself.
• Return traffic gradually (if you failed over), then re-run the parity diff. Incident-mode changes have a way of becoming permanent config drift.
• Postmortem within a week: time-to-detect, time-to-decide, what the playbook got wrong. Every criterion you wish you'd had becomes a line in the runbook.

The playbook before the playbook

Everything above goes better if five things existed yesterday: independent payment-health metrics, quantified failover trigger criteria, pre-drafted comms, a tested backup path (or the explicit, documented decision not to have one, the framework), and a current map of what depends on the affected provider. That last one is the difference between "Stripe is down" and "these three flows carrying this much MRR are down", which is the sentence your incident commander actually needs. Maintaining that live map, with revenue attached, is what PayRes does; it's how the first ten minutes of your next incident start from knowledge instead of archaeology.