Execution replay: auditing our own fills
There is a particular silence that falls over a performance conversation when someone asks: "And how were the fills?" Backtests assume execution. Live records contain it. The difference between the two is where a large share of paper alpha goes to die — and in listed derivatives, where spreads can be wide and depth can be thin, it can be most of it.
We decided early that execution quality would be treated as a first-class research input, with the same tooling rigor as signal research. The mechanism is simple to describe and unforgiving in practice: every live fill is replayed against recorded market data.
The mechanics
Our capture layer records full-depth market data for everything we trade, timestamped at the source. When an order fills, we can reconstruct the exact state of the book around that moment — what was quoted, at what size, on both sides — and ask precise questions:
- What was the quoted midpoint and spread at decision time, at order placement, and at fill?
- How much of the spread did we pay, relative to the best case available?
- Did the market move between signal and execution — and was that move adverse selection, latency, or noise?
- How did fill quality vary by time of day, instrument, order size, and volatility state?
Because the replay comes from our own recorded feed rather than a vendor's end-of-day summary, the audit has tick-level resolution and no survivorship in it. The market as it actually was, against the trades as they actually happened.
What the audit changes
Three things happen when execution replay is routine rather than forensic.
Slippage becomes a budget, not a surprise. Every strategy carries an explicit execution-cost model, estimated from its own live fills. When realized costs drift from the model, that drift is an alert — investigated like any other anomaly, because it usually means the market's liquidity profile changed or our behavior did.
Backtests inherit honesty. Measured fill behavior feeds back into simulation assumptions. A strategy is evaluated against the execution costs we have actually demonstrated we can achieve — not against midpoint fills nobody gets. Paper edges that exist only at the midpoint die in this step, which is exactly where they should die: before capital touches them.
Sizing respects the market's actual depth. Fill data is the ground truth for capacity. How much size moved the market, and by how much, is measured — so scaling decisions rest on observed market mechanics rather than optimistic extrapolation.
The uncomfortable part
A replay audit will, occasionally, tell you something you do not want to hear: that a strategy's headline return was flattered by an execution assumption, or that live fills are systematically worse than simulation in a specific regime. We have had audits catch exactly this class of gap — a strategy whose replayed, cost-honest performance diverged materially from its idealized simulation.
That result is not a failure of the process. It is the process. An edge that only exists before costs is not an edge; it is a story. We would rather find that out ourselves, at research scale, than have an allocator find it out at capital scale.
Why this belongs in diligence
If you are evaluating any systematic manager — including us — we think fill-level execution evidence belongs on the standard diligence checklist, next to the track record itself. Ask how execution costs are measured, from what data, and how the answer feeds back into simulation and sizing. The quality of the answer tells you a great deal about the quality of everything else.
Our answer: from our own tick-level records, continuously, for every fill — and the results are part of the record we show.