Concurrent Order Book
- Published on
- 11 January 2026-2 min read
Why Shared_Mutex Failed: 15.7M ops/s
Benchmark two lock strategies on identical code. The assumption (RW-locks are faster for reads) met reality (8× latency explosion at 8 threads).
Benchmark: Myth vs Reality
| Scenario | std::mutex | std::shared_mutex | Winner |
|---|---|---|---|
| 1 thread, balanced | 15.7M ops/s | 5.4M ops/s | mutex 2.9x |
| 8 threads, read-heavy (95/5) | 8.2M ops/s | 1.5M ops/s | mutex 5.5x |
| p99 latency (8 threads) | 30 μs | 99 μs | mutex 3.3x |
The verdict: std::mutex won every configuration. RW-locks aren't magic.
What Actually Happened
Built a multithreaded order book in C++17 and benchmarked two locking strategies across 12 configurations.
The hypothesis was wrong. Expected shared_mutex to win in a 90% read workload. Measured the opposite — mutex was 3.6x faster. Traced it to three causes: critical sections too short (~20ns) for shared locking to pay off, atomic reader counters causing cache-line contention at 8 threads, and macOS os_unfair_lock being significantly lighter than pthread_rwlock.
Lock policy swapped at compile time. Used template parameters instead of virtual dispatch so benchmarks measure pure lock difference with zero runtime overhead.
Found a dangling pointer bug in cancel logic. remove_if freed a list node, but the code still dereferenced the pointer afterward. Intermittent failures — sometimes the freed memory still held the old value. Fixed by copying price to a local variable before deletion.
View on GitHub →