In this article
Before you start
This page is one step of the system design learning path.
Why interviewers ask this
A payment system is the standard test of whether a candidate designs for failure first. Money moves between parties that do not share a database, any call can time out, and a wrong answer shows up as a customer charged twice or a book that does not balance. Interviewers use it to see if you reach for idempotency, a ledger and reconciliation without prompting.
A weak answer draws a client, a server and a database and says "use a transaction." A strong answer says how a retry is made safe, how the books are kept consistent, and what happens when the outcome of a call is unknown. A Staff-level answer separates the moving of money from the recording of it, and states which steps run exactly once and which run at least once, made safe.
A payment system from first principles
- Double-entry ledger
- A double-entry ledger is an append-only record of money movements in which every transaction is written as two or more entries whose amounts sum to zero, so money is moved between accounts and never created or destroyed.
- Saga
- A saga is a way to run a business operation that spans several services as a sequence of local transactions, where a failed step is handled by compensating transactions that undo the steps already done.
- Reconciliation
- Reconciliation is the routine comparison of your own records with an outside party's records, such as the ledger against the PSP's settlement reports, to find and fix differences.
In plain words
Think of a shop with a till and a stack of receipts. Every sale writes a receipt with a number, so if the customer comes back saying "I paid" the shop looks up the number instead of charging again. The owner also writes each sale in a book as two lines, money in and goods out, so the book always balances. At closing time the owner compares the till with the bank's statement. The analogy breaks in one place. A real payment system cannot see the bank's side live. Its calls can vanish without a reply. It must be built for outcomes it cannot observe.
In one precise sentence: a payment system makes each payment happen once in effect. It combines a retry-safe request, a balanced record, a tracked state and an outside check. Stripe's idempotency article (2017) describes the retry-safe request, and Fowler's accounting transaction (2005) describes the balanced record.
Step 1: Accept each payment once with an idempotency key
The payment application programming interface (API) checks an idempotency key before doing any work. The client generates the key and the server saves the result under it, so a retry gets the saved result. Stripe's reference for its version 1 API (as of October 2026) says it saves the status and body of the first request made with a key, returns them for retries, and errors if a retry's parameters differ. A request that arrives while the first is still running conflicts and saves no result.
Claiming a key must be one atomic step. A unique index (a database rule that rejects a second row with the same key) does it: the PostgreSQL 18 documentation describes a conflict-handling clause for unique indexes. Input: key and payment details. Output: "new, proceed", the saved result, or a conflict (Hypertext Transfer Protocol 409) if the key is claimed but has no result yet, so the client retries later. In the simulation, 1,973 of 100,000 payments were charged more than once without a key and none with one.1
Step 2: Record the money in a double-entry ledger
The ledger records the payment as entries that sum to zero. For a payment of 100.00 units with a fee of 2.90, the entries are minus 100.00 on the customer's funds, plus 97.10 on the merchant's payable account and plus 2.90 on the fee account, and they sum to zero.1 Amounts are stored as whole numbers of the smallest currency unit, such as 10000, never as floating-point numbers.
The ledger is append-only: a mistake is corrected by a new reversing transaction, not by editing a row. Fowler (2005) notes that a transaction is posted only when its entries balance to zero, and that no entries can be added after posting. The steps on this page are components, not a strict timeline: when rows are written is a design choice. A safe choice is to write the balanced rows in one database transaction when the PSP confirms an authorization or capture, and to keep a pending entry that is reversed if the attempt fails. Input: the payment, the PSP's confirmation and the fee rules. Output: balanced rows. Balances are derived by summing entries.
Step 3: Move the payment through a state machine and call the provider
A payment moves through states (created, authorized, captured, settled, failed, refunded), and only legal moves are allowed. That is a state machine: a fixed set of states with rules for which move is allowed. An authorization is the issuer's promise to hold the funds, a capture is the request to collect them, and settlement is the later transfer of the money to the merchant. The issuer is the cardholder's bank, and it is the party that approves or declines. The card network (such as Visa or Mastercard) routes the request between banks. A payment service provider (PSP) sits between your service and the networks, and it is the party your service calls.
Calling the PSP is the step that can fail in the middle: it may have authorized the card even if no reply came back. When the outcome is unknown, the state is "pending", never "failed". The service retries with the same idempotency key or reference, which PSPs such as Stripe accept, or asks the PSP for the payment's status. Waits grow by exponential backoff with jitter: each wait is about double the last, with a random part so clients do not retry together, as the Stripe article describes. A multi-service payment is a saga (Richardson): local transactions with compensating transactions, and no isolation.
Step 4: Reconcile against the provider's reports
Reconciliation catches what the live path missed. Each day a job loads the PSP's settlement report and compares it with the ledger, line by line, using the payment reference. A difference has more than one possible cause, so the job first waits out the settlement window (often one or more business days, so a recent payment may legitimately be missing from the report). A payment in the report but not in the ledger can mean a missed status or a lost reply that was never recorded. A ledger entry with no report line can mean the payment has not settled yet, an authorization that was never captured, or a real error. Each difference becomes a ticket or a correcting entry.
The PSP also pushes status changes to your service as webhook calls, which arrive at least once and need the same idempotent handling. Input: the report file and the ledger. Output: a list of mismatches. This turns "we think it worked" into "the books agree with the provider", and it answers the interviewer's question about failures that nothing in the live path can detect. Keep card data out of your own systems where possible; the Payment Card Industry Data Security Standard applies to organisations that handle it.
Step-by-step description
- The API checks the idempotency key first.
- Balanced ledger entries are written.
- A state machine calls the PSP.
- The reconciliation job checks books against reports.
- The payment API checks an idempotency key before doing any work.
- The ledger database stores balanced entries that sum to zero.
- The payment service moves each payment through states and calls the PSP.
- An reconciliation job compares the ledger with the PSP's reports.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
Take an illustrative load of 10,000,000 payments a day.1 The average rate is 10,000,000 divided by 86,400 seconds, which is 115.7 payments a second, and a peak ten times the average is 1,157.4 a second. Each payment writes three ledger rows, so the peak is about 3,472 ledger rows a second and the ledger grows by 30,000,000 rows a day; at 200 bytes a row that is 6.0 gigabytes a day, or 2.19 terabytes a year.1 Keeping idempotency records for 24 hours at 600 bytes each is another 6.0 gigabytes.1
| Quantity | Value (illustrative load, computed) |
|---|---|
| Payments per day | 10,000,000 |
| Average payments per second | 115.7 |
| Peak payments per second (10 times average) | 1,157.4 |
| Ledger rows per payment | 3 |
| Ledger rows per second at peak | 3,472.2 |
| Ledger growth per day (200 bytes a row) | 6.0 GB |
| Ledger growth per year | 2.19 TB |
| Idempotency records held 24 hours (600 bytes each) | 6.0 GB |
| Payments charged more than once in 100,000, no key (2,019 extra charges, 46 payments charged three times) | 1,973 |
| Payments charged more than once in 100,000, with key | 0 |
Now trace one payment through a lost reply. The client sends "pay 100, key K1"; the service saves K1, the PSP authorizes once and approves; the reply to the client is lost; the client retries with the same key K1; the service finds K1 and returns the saved approval without a second charge.1
Step-by-step description
- Payment and key arrive; the key is saved.
- The PSP authorizes the card once.
- The client times out and cannot know the outcome.
- Same key, saved result, no second charge.
- The client sends the same idempotency key on every attempt.
- The service saves the first result under that key.
- A retry with the same key returns the saved result and does not charge again.
Text description of the diagram
- In the simulation 1,973 of 100,000 payments were charged more than once without a key.
- With a key, 0 payments were charged more than once.
The simulation is illustrative: a 2% reply-loss rate and three attempts are assumptions, not measurements of any provider.1
Trade-offs and edge cases
Payments choose consistency over availability: when the system cannot tell whether money moved, it refuses to guess and shows "pending". That is why the state machine has an unknown state and why reconciliation exists. The usual claim of "exactly once" delivery is better stated as at-least-once delivery made safe by idempotency, which is what Step 1 does.
Sagas trade atomicity for availability across services. They have no automatic rollback, so each step needs a compensating action designed in advance, such as a refund for a charge, and concurrent flows can read half-finished data (Richardson). A single database transaction is simpler and should be used whenever the ledger and the payment state live in one database.
Other cases to name: refunds and chargebacks (a chargeback is a dispute in which the issuer takes the money back from the merchant) are new ledger transactions that reverse earlier ones, not edits; currency conversion needs a quoted rate stored with the transaction; fees and tax are separate entries; and tokenisation (swapping a card number for a token that is useless elsewhere) keeps raw card numbers out of most of your systems to reduce the scope of the card-data standard. A fraud check is a model call in the path, which the machine learning system design hub covers. For the broader pattern of retries and loads, the data engineering hub discusses idempotent loads.
How to answer it in an interview
Design a payment system.
I would first ask about volume, currencies and payment methods. Then the core: every request carries an idempotency key, claimed with a unique index, so a retry returns the saved result. The payment service runs a state machine and calls the payment provider, which routes to the card networks and the issuing bank; an unknown outcome stays pending and is resolved by a status check. Money is recorded in an append-only double-entry ledger whose entries sum to zero. A daily reconciliation job compares the ledger with the provider's settlement report. The main trade-off is that I accept pending states to avoid double charges.
The spoken structure: clarify scale, then key, ledger, state machine and reconciliation, and end on the trade-off.
Common mistakes
- "Use a database transaction and the payment is safe." (wrong) The PSP call is outside your database, so a transaction cannot cover it. (right) "A transaction covers my ledger; the external call needs idempotency and a status check."
- "If the call times out, mark the payment failed and let the user retry." (wrong) The provider may have authorized already. (right) "Mark it pending and resolve it by status check or reconciliation."
- "Store the balance as one number I update." (wrong) A single updated number loses the history and the proof. (right) "Append balanced entries and derive balances."
- "Use floating-point numbers for money." (wrong) Binary fractions cannot hold amounts like 0.10 exactly. (right) "Whole numbers of the smallest currency unit."
Follow-up questions
How do you design a bank account transfer with a focus on concurrency?
Write the debit and credit as one ledger transaction and protect the balance check with a lock or a version check, so two transfers cannot both spend the same funds. Take locks in a fixed account order to avoid deadlock, and retry a transaction that fails on a version conflict.
How would you design international money transfers?
Treat it as a multi-leg transaction: store the quoted exchange rate and fee with the transfer, write ledger entries per currency that each balance, and run the steps as a saga with compensating actions. The settlement report from each partner is reconciled separately.
How do you design a banking system for reporting and auditing?
Keep the ledger append-only and derive balances and statements from it, so every figure traces back to entries. Feed a read-optimised copy to reporting, and keep the reconciliation job that compares the ledger with outside records.
What do you do when the payment provider times out?
Record the payment as pending, retry with the same key or query the provider for its status, and let the daily reconciliation catch any case the live path cannot resolve. Never mark it failed without a definite answer.
What changes at 100 times the volume?
One database no longer fits. The peak becomes about 115,700 payments a second and the ledger grows to about 219 terabytes a year, so shard the ledger by account, so that each account's entries stay together and a transfer between shards becomes a saga with compensating entries.1 Keep the key store sharded the same way and keep reconciliation running per shard.
How do you make retries safe?
Send an idempotency key with every request, save the first result under it, and use exponential backoff with random jitter so retries do not arrive together. The interview loops hub shows where this question appears in a full design loop.
- Make every payment request idempotent with a key, claimed atomically by a unique index.
- Record money in an append-only double-entry ledger whose entries sum to zero.
- An unknown outcome is pending, never failed; resolve it by status check and reconciliation.
- Sagas need compensating actions designed in advance and give up isolation.
- At 10,000,000 payments a day the peak is about 1,157 payments a second (3,472 ledger rows); storage grows about 2.19 TB a year.
Sources
- Idempotent requests (Stripe API reference) — Stripe (date = read date), 24 Oct 2026
- Designing robust and predictable APIs with idempotency — Stripe (Leach), 22 Feb 2017
- Accounting Transaction — martinfowler.com (Fowler), 22 Jan 2005
- Pattern: Saga — microservices.io (Richardson; date = read date), 24 Oct 2026
- INSERT, PostgreSQL 18 documentation — PostgreSQL Global Development Group (date = read date), 24 Oct 2026
- PCI Data Security Standard (PCI DSS) — PCI Security Standards Council (date = read date), 24 Oct 2026