6.2 KiB
title: Identifying and Mitigating Twelve Critical Errors in Payment System Design: A Comprehensive Interview Preparation Guide created: 2026-03-04 updated: 2026-03-04 type: summary tags: ["design", "notes", "payment", "system", "telegram"] external: http://go/ui/posts/2111/
Common Errors & Issues Candidates Make When Designing a Payment System
Based on well-established software engineering and system design knowledge, here are the most common mistakes and pitfalls interviewees encounter when tackling a payment system design question:
1. Ignoring Idempotency
This is arguably the #1 mistake. Candidates often fail to address:
- What happens when a payment request is retried (network timeout, client retry, etc.)
- Not introducing an idempotency key to ensure the same payment isn't processed twice
- Double-charging a customer is one of the worst outcomes in a payment system
✅ What interviewers expect: Every mutating API call (especially charge/pay) should accept a client-generated idempotency key, and the server should deduplicate based on it.
2. Not Handling Distributed Transactions / Consistency
- Treating the system as if a single database transaction can cover everything (payment gateway call + DB update + ledger entry)
- Not discussing eventual consistency and saga patterns
- Ignoring the "what if the payment gateway succeeds but our DB write fails?" scenario
✅ What interviewers expect: Discussion of the two-phase approach — first persist the intent (pending state), then call the external gateway, then reconcile.
3. Overlooking Reconciliation
- Candidates design the "happy path" only and forget that discrepancies between internal records and external payment providers are inevitable
- No mention of batch reconciliation jobs that compare internal ledger with bank/gateway statements
✅ What interviewers expect: A reconciliation service that periodically compares records and flags mismatches.
4. Poor or Missing Ledger Design
- Using simple
balancefields that get updated in place (mutable state) - Not designing a proper double-entry bookkeeping ledger
- Missing the concept that every transaction should have a debit and a credit entry
✅ What interviewers expect: Append-only ledger entries with double-entry accounting principles. Balances derived from the ledger, not stored as a single mutable field.
5. Neglecting Failure Handling & Retry Logic
- Not distinguishing between retriable errors (network timeout, 5xx) and non-retriable errors (insufficient funds, invalid card)
- No exponential backoff or dead-letter queue for failed operations
- Not discussing circuit breakers when calling external payment providers
6. Ignoring Security Concerns
- Storing raw credit card numbers (violating PCI-DSS compliance)
- Not mentioning tokenization of sensitive payment data
- Missing encryption at rest and in transit
- Not discussing fraud detection at all
✅ What interviewers expect: The system should never store raw card data. Use tokenization via providers like Stripe, Adyen, or a PCI-compliant vault.
7. No State Machine for Payment Lifecycle
- Treating a payment as a binary success/fail
- Not modeling the full lifecycle:
CREATED → PROCESSING → SUCCESS / FAILED → REFUNDED / DISPUTED - Missing edge cases like partial refunds, chargebacks, and disputes
✅ What interviewers expect: A clear state machine diagram showing valid transitions.
8. Tightly Coupling to a Single Payment Provider
- Designing the system around one specific gateway (e.g., Stripe) without an abstraction layer
- Not considering multi-provider routing for redundancy, cost optimization, or regional requirements
✅ What interviewers expect: A Payment Service Provider (PSP) abstraction layer that allows plugging in different gateways.
9. Underestimating the Importance of Async Processing
- Making everything synchronous, blocking the user while waiting for a bank response
- Not using message queues (e.g., Kafka, SQS) for downstream processing (notifications, analytics, ledger updates)
10. Missing Non-Functional Requirements
- Availability: Payment systems need extremely high availability (often 99.99%+)
- Latency: Users expect payment confirmation within seconds
- Auditability: Every action must be logged and traceable for compliance
- Scalability: Not discussing how to handle peak loads (e.g., Black Friday)
11. Not Clarifying Requirements Upfront
- Jumping straight into architecture without asking:
- What types of payments? (one-time, subscriptions, marketplace payouts?)
- What scale? (transactions per second)
- What regions/currencies?
- Who are the users? (B2C, B2B, P2P?)
12. Forgetting About Currency & Money Handling
- Using floating point numbers for monetary values (classic bug)
- Not handling multi-currency scenarios
- Ignoring exchange rate management and rounding rules
✅ What interviewers expect: Use integer cents (or the smallest currency unit) and a separate currency code field.
Summary Cheat Sheet
| Rank | Common Mistake | Severity |
|---|---|---|
| 1 | No idempotency | 🔴 Critical |
| 2 | No distributed transaction handling | 🔴 Critical |
| 3 | No reconciliation | 🔴 Critical |
| 4 | Poor ledger design | 🟠 High |
| 5 | No retry/failure strategy | 🟠 High |
| 6 | Security / PCI violations | 🔴 Critical |
| 7 | No payment state machine | 🟠 High |
| 8 | Tight coupling to one provider | 🟡 Medium |
| 9 | All-synchronous design | 🟡 Medium |
| 10 | Ignoring non-functional requirements | 🟡 Medium |
| 11 | Not clarifying requirements | 🟡 Medium |
| 12 | Floating-point money | 🟠 High |
These points are drawn from widely recognized best practices in payment system engineering (as practiced at companies like Stripe, Square, PayPal, and in resources like Designing Data-Intensive Applications by Martin Kleppmann and System Design Interview by Alex Xu). Would you like me to dive deeper into any of these areas?