Low latency trading system design

In this article

1. Before you start
2. Why interviewers ask this
3. A low-latency trading system from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one step on the system design learning path.

Why interviewers ask this

A low-latency question tests whether you know which ordinary engineering habits become faults when time is measured in microseconds. A microsecond is one millionth of a second. In a web service a garbage-collection pause or a network hop goes unnoticed. In a trading path it can decide who gets the trade. Reported in interviews at quantitative trading firms and large tech companies, the prompt is usually "design a low-latency trading platform" or "design the message flow between an exchange and a trading system".

A weak answer says "use a fast language and a cache". A strong answer names the stages from packet to order, prices each one, repairs the feed when packets are lost, makes the code replayable and talks about the tail, not the average. It also covers failure, because a fast system that sends a duplicate order is worse than a slow one.

A low-latency trading system from first principles

Tick-to-trade latency

Tick-to-trade latency is the time from the arrival of a market-data update at a trading system's network card to the departure of the order it triggers, measured as a distribution and not as one number.

Kernel bypass

Kernel bypass is a networking technique in which a program reads packets directly from the network card in user space, by polling, instead of going through the operating system's network stack and interrupts.

Sequence gap

A sequence gap is a break in the numbering of a message stream, such as receiving message 105 after message 100, which tells the receiver that messages 101 to 104 were lost or delayed.

In plain words

A trading system is a relay runner who must start when the baton appears. Any wait loses the race: a door that opens on a schedule, a stranger in the lane, a stop to tie a shoe. So the team removes the doors, clears the lane and ties the shoes beforehand. The analogy breaks in one place. A runner may be fast once. A trading system must be fast every time, so the slowest run in a thousand matters more than the average.

In one precise sentence: the design keeps the hot path in memory on dedicated threads, replaces waiting with polling, and records every input so the run can be replayed.

Step 1: Poll the network card, do not wait for interrupts

The input is a stream of market-data packets; the output is those packets in the program's memory. By default the operating system raises an interrupt when a packet arrives, then runs its network stack, then wakes your thread. Each hand-off adds delay and variation. Busy polling spends central processing unit (CPU) time to save latency: the Linux kernel documentation says it "trades off CPU cycles for lower latency", and warns that a polling kernel thread keeps its core at 100% usage (Linux kernel documentation, read October 2026).

Kernel bypass goes further. The Data Plane Development Kit (DPDK) ships poll mode drivers that are "designed to work without asynchronous, interrupt-based signaling mechanisms" (DPDK programmer's guide, release 26.11.0-rc1). The price is a core that does nothing else, so give it a dedicated core. What you now know: the first saving is to stop waiting for the operating system.

Step 2: Use sequence numbers to find and repair gaps

Market data usually arrives by User Datagram Protocol (UDP) multicast, which is fast and does not promise delivery. The Nasdaq MoldUDP64 specification describes a lightweight layer on top of UDP in which each packet carries the sequence number of its first message, so a listener can detect and re-request missed packets (Nasdaq, MoldUDP64 v1.00). The sequence number is an 8-byte field. Heartbeat packets with a message count of zero let a receiver sense loss even when trading is quiet.

When the next sequence number is not the expected one, the receiver holds the later packets and sends a request, naming the first missing number and a count, to a re-request server. Sequence numbers reveal a gap, which triggers a re-request. The input is packets that may be missing or swapped; the output is one gap-free, ordered stream. Wait a few packets before asking, because swaps are common and harmless. What you now know: a lost packet becomes a counted, repairable event instead of a silently wrong book.

Step 3: Keep the order book in memory, in one thread

The input is the ordered stream of orders and cancels; the output is the current book. Every instrument's book lives in pre-allocated memory, owned by one thread, so no lock is ever taken. The book keeps buy and sell orders sorted by price and, within a price, by arrival, which is the priority an exchange uses for matching.

Pre-allocation matters: the Disruptor report notes that entries allocated at startup leave the garbage collector little to reclaim (Thompson et al., 2011). Choose structures with predictable cost, such as fixed-size arrays, not ones that resize mid-burst. Keep one writer per book. What you now know: the book is a single-writer, no-lock, no-allocation structure.

Step 4: Pass work between threads without locks

The strategy often runs on its own thread, so book updates must cross from one thread to another. An ordinary bounded queue is a poor way to do it: the Disruptor report says queues suffer write contention on the head, tail and size variables, and that head and tail often share a cache line, so two threads writing them collide (false sharing) (Thompson et al., 2011).

The alternative is a pre-allocated ring buffer, in which each consumer tracks its own position. In the report's three-stage pipeline test, mean latency per hop was 52 nanoseconds for the Disruptor against 32,757 nanoseconds for a bounded queue, three orders of magnitude lower; these are the authors' own benchmark figures. The input is an update; the output is the strategy's decision, with no lock on the way. What you now know: the hand-off between threads is part of the latency budget.

Step 5: Check risk, then send with a client order id

Before an order leaves, a pre-trade risk check compares it with limits: size, price band and exposure. It sits in the hot path as a few comparisons against in-memory counters, not as a call to another service. The order then goes out with a client order id. A client order id is a unique identifier chosen by the sender, which the venue uses to refer to that order in later messages.

That id is part of the answer to the failover question, but only part. Suppose the primary crashes after sending an order but before recording that it did. The backup replays the journal and cannot tell whether the order left. Three rules make the replay safe. First, derive the client order id from the journalled input, for example from the input's sequence number and the strategy's decision index, so the backup regenerates the same id and not a fresh one.

Second, before resending, reconcile with the venue: ask for the order's status, or read execution reports replayed through session resend or a drop copy. Third, rely on a venue rejecting a repeated id only where its own specification says so. The Financial Information eXchange (FIX) 4.4 definition of the client order id makes the sender responsible for uniqueness within a trading day and names no one who enforces it (FIX 4.4 dictionary, read October 2026), and a backup on a different session may not be checked against the primary's orders at all.

In the model below, 473 of 1,000 failovers crash after the send; resending with a fresh id creates 473 duplicates, resending with the same id still creates 473 when the venue does not deduplicate, and reconciling first creates none in every case.¹ See also designing a payment system. What you now know: failover safety comes from a reproducible id plus reconciliation, not from timing and not from hoping the venue rejects a repeat.

Step 6: Journal every input and measure the tail

The input to the system is the stream of messages it received; the output is the list of orders it sent. If the code is deterministic, the first can reproduce the second. The system therefore writes every input, with its sequence number, to a journal, off the hot path, and never lets the strategy read the wall clock or a random number without recording it.

The journal serves three jobs: audit, recovery after a crash, and testing. Measurement closes the loop. Record the whole distribution of tick-to-trade times, not the mean, and read the 99th and 99.9th percentiles, because a rare pause is exactly what loses the trade. What you now know: a replayable system can be tested, audited and failed over with the same log.

1 · Feed2 · Gaps3 · Book4 · Decide5 · Send6 · JournalReplay

Step-by-step description

1. Poll the network card for packets.
2. Sequence numbers expose a missing packet.
3. Update the order book in memory.
4. A deterministic strategy reads the book.
5. Risk check, then send with a client order id.
6. Branch the sequenced input into a journal, off the hot path.

• The feed is read by polling the network card, not by waiting for interrupts.
• Sequence numbers reveal a gap and trigger a re-request.
• The order book and strategy are deterministic in-memory code.
• The order leaves with a client order id after a risk check, and the journal records every input.

Worked example

Every figure here is illustrative, produced by the saved script from stated assumptions, and none is a measurement of any venue or firm.¹ First, distance. Light in glass travels at the speed of light divided by the refractive index; with an assumed index of 1.47 that is 4.903 microseconds per kilometre. A 10-metre cable adds 0.049 microseconds, 100 kilometres adds 490.3 microseconds, and 1,000 kilometres adds 4.9 milliseconds. Physics sets a floor that no software removes, which is why firms place their servers near the venue.

Second, the budget. Assume the stages below, in nanoseconds, for a path through the kernel network stack and a path with kernel bypass. In this illustrative budget the kernel path totals 15,050 nanoseconds and the bypass path totals 3,050, which is 3.05 microseconds, because the 12,000-nanosecond stack stage disappears.¹

Text description of the diagram

• The kernel path totals 15.05 microseconds in the illustrative budget.
• The bypass path totals 3.05 microseconds.
• The network stack stage is 12 microseconds, 79.7% of the kernel path.

Third, the feed. A stream of 100,000 messages in 25,000 packets of four loses 1% of packets at random: 255 packets, 1,020 messages. A receiver that applies whatever arrives is missing all 1,020. A receiver with sequence numbers sent 255 re-requests and delivered all 100,000 messages in exact order.¹

Fourth, determinism. A toy matching engine with price-time priority processed 200,000 random orders and produced 155,357 trades. Replaying the same input gave the same output digest both times. Swapping two adjacent input orders, at positions 1,001 and 1,002, changed the digest, so order of arrival is part of the state and the journal must keep it.¹

Fifth, the tail. In a simulated distribution of one million samples, with one sample in a thousand a long pause, the median is 4.0 microseconds and the mean 4.39, but the 99th percentile is 8.13 and the 99.9th is 111.25.¹

Text description of the diagram

• The median is 4.0 microseconds and the mean is 4.39.
• The 99th percentile is 8.13 microseconds.
• The 99.9th percentile is 111.25 microseconds.

Trade-offs and edge cases

Every saving above has a price. State it when you propose the saving.

• Polling against power and cores. Busy polling and poll mode drivers burn a full core, even when nothing arrives. The Linux documentation says so directly. Pay it for the hot path only.
• Speed against safety. Risk checks add nanoseconds, but a missing check can lose far more than a slower path; keep them few, simple and in memory.
• Single writer against parallelism. One thread per book is lock-free and simple, but one thread's capacity bounds an instrument. Shard by instrument, never by operation.
• Determinism against convenience. Reading the clock or a random number inside the strategy breaks replay. Inject both as recorded inputs.
• Garbage-collected languages. A collector pause lands in the tail. Pre-allocate and reuse objects, or use a language with manual memory control for the hot path.
• Co-location against flexibility. Placing servers in the same building as the venue removes the distance term of the worked example, but it ties you to one site and one set of rules. Decide which instruments justify it.
• Overload. In a burst the feed can outrun the book. Decide the policy before it happens: drop the slowest consumer, shed non-essential symbols, or halt trading, and always log the choice.
• Feed arbitration. A feed may be published on two redundant lines. Take whichever copy of each sequence number arrives first and discard the second, so one lost packet on one line costs nothing.
• Clocks. To compare timestamps across machines you need synchronised clocks, usually through a hardware-assisted time protocol, and you must record the clock error.
• Machine-learning signals. A model in the path needs its features precomputed and its inference bounded; see the machine learning system design overview and a feature store for the training-serving problem.

The cache side of the problem, keeping hot data close, is covered in designing a distributed cache.

How to answer it in an interview

The spoken structure: the path first, then the feed repair, then the single-writer core, then failure, then how you measure.

Common mistakes

• "Average latency is under ten microseconds, so we are fine." (wrong) The mean hides the rare pause that loses the trade. (right) "I track the 99th and 99.9th percentiles."
• "Use a thread-safe queue between stages." (wrong) A locked queue adds contention and cache-line sharing at every hop. (right) "A pre-allocated ring buffer with one writer."
• "On a gap, wait for the next packet." (wrong) The book is silently wrong until the missing messages arrive. (right) "Detect the gap by sequence number and re-request after a short window."
• "On failover, resend anything unacknowledged." (wrong) The primary may have sent it, creating a duplicate. (right) "Replay with the same derived client order id and reconcile with the venue before resending."
• "Make the strategy read the current time." (wrong) A replay then differs from the live run. (right) "Time is an input and is journalled."

Follow-up questions

Where do the microseconds go?

In the illustrative budget, 12,000 of 15,050 nanoseconds are the kernel network stack and interrupt, and the rest is decode, book, strategy, risk and the two card crossings. Measure your own path stage by stage with hardware timestamps before optimising anything.

How do you fail over without sending an order twice?

Derive each client order id from the journalled input so the backup regenerates the same one, then reconcile with the venue (order-status request, or execution reports through session resend or a drop copy) before resending. Count on a duplicate rejection only if the venue's specification promises it. In the model, 473 failovers crashed after sending; reconciling first left none, while resending without it left 473 wherever the venue did not deduplicate.¹

How do you test latency changes?

Replay the same recorded input through the old and new builds and compare the full distribution, including the 99.9th percentile, not the mean. A deterministic system makes the comparison exact; see deployment strategies for rolling a change out safely.

What if the strategy runs 100 km from the venue?

Co-location comes first. In the worked example 100 km of fibre adds 490.3 microseconds each way, against a 3.05-microsecond bypass budget, so no software change can make up the difference.¹

How do you design a low-latency trading system that uses order latency as a parameter for backtesting?

Replay recorded market data through the same code and delay each simulated order by a latency drawn from a measured distribution before it reaches the simulated book. Use the tail of the distribution as well as the typical value, because the late orders decide the result.

Sources

1. ClOrdID (tag 11), FIX 4.4 dictionary — OnixS (FIX 4.4 field definition; date = read date), 24 Oct 2026
2. MoldUDP64 Protocol Specification, V 1.00 — Nasdaq (last version-control entry, formatting update; read 2026-10-24), 2 Aug 2024
3. Disruptor: High performance alternative to bounded queues for exchanging data between concurrent threads — LMAX (Thompson, Farley, Barker, Gee and Stewart), 1 May 2011
4. NAPI (Linux kernel documentation) — The Linux kernel project (date = read date), 24 Oct 2026
5. Overview, Data Plane Development Kit programmer's guide — DPDK project (release 26.11.0-rc1; date = read date), 24 Oct 2026