Most firms' Trade-Surveillance (TS) controls rely on large libraries of static, scenario-based rules to detect behaviours such as spoofing, layering, wash trading, and marking-the-close.
These rule engines are costly to maintain, generate excessive false positives, and often fail to capture novel manipulation patterns.
Prospect 33 has already proven the effectiveness of AI-driven anomaly detection within Transaction-Reporting (TR) at Tier-1 banks. By learning what "normal" looks like across complex data lifecycles, our models uncover previously undetected errors and completeness issues.
The same scientific discipline (lifecycle modelling, unsupervised learning, and explainability) can be applied upstream to orders rather than executions, providing a quantitative, regulator-aligned measure of trading behaviour and control effectiveness.
Regulators require firms to monitor for potential market abuse, manipulation, and insider dealing to protect market integrity and investor confidence.
Key regulations include:
| Behaviour | Description | Why Regulators Care |
|---|---|---|
| Spoofing | Large, non-bona-fide orders placed then cancelled to mislead market depth | Distorts supply/demand; breaches MAR Art. 12 & CFTC § 4c(a)(5)(C) |
| Layering | Multiple price-stepped spoof orders on one side with fills on the other | Creates artificial price pressure; precursor to spoofing |
| Wash Trading | Same entity buys and sells identical instrument with no risk transfer | Fabricates volume and liquidity; false market |
| Quote Stuffing | Rapid order/cancel bursts to overload systems | Impedes fair access; market-disruption offence |
| Momentum Ignition | Aggressive orders to trigger price move and profit from reaction | Induces volatility and price instability |
| Front Running | Trading ahead of known client or proprietary flow | Misuse of confidential information; fiduciary breach |
| Marking the Close | Manipulating closing or reference price through late activity | Distorts benchmarks, valuations, and NAVs |
| Cross-Market Manipulation | Coordinated orders across correlated instruments or venues | Artificial price formation spanning markets |
Each pattern leaves a distinct temporal and structural fingerprint in order data—precisely what our lifecycle-based anomaly-detection framework is designed to expose.
The new data requirement is therefore access to detailed order-event feeds from the firm's Order Management System (OMS) or Execution Management System (EMS), or from venue drop-copy feeds.
(OMS and EMS are internal systems that capture, route, and manage client and proprietary orders through their lifecycle.)
Learn normal order-lifecycle transitions and timing patterns per trader–instrument–venue and detect statistically significant deviations that may indicate manipulation or control weaknesses.
Collect all order events—new, modify, cancel, partial fill, full fill, expire—with timestamps, price, size, venue, and trader identifier.
Model transitions such as:
New → Modify → (Partial Fill)* → [Full Fill | Cancel | Expire]
Learn transition probabilities and dwell-time distributions per entity.
Compute anomaly scores (e.g., Jensen–Shannon divergence) when observed behaviour diverges from the baseline.
Order-to-Trade Ratio (OTR): number and notional of orders vs. trades.
Many venues publish member-level or instrument-level OTR statistics under RTS 9; these can be ingested to benchmark internal activity against market norms.
Cancel-on-Approach Rate: probability of cancelling as price nears best bid/offer.
Short-Rest Ratio: share of orders cancelled unusually quickly.
Quote Imbalance: persistent skew between buy and sell quoting.
Each anomaly is accompanied by a concise rationale ("Cancel-on-Approach +3 σ; dwell time 120 ms vs 590 ms baseline").
Surveillance SMEs review results and feed back classifications to refine baselines.
After order-only value is proven, progressively incorporate:
Trader limits, desk mandates.
Volatility, halts, benchmark events.
Chat, voice, and public sentiment for intent inference.
Depending on client maturity:
| Situation | Prospect 33 Role |
|---|---|
| Existing TS platform in place | Quantitatively test and enhance current alerts, reduce false positives, validate coverage using anomaly metrics. |
| Partial implementation | Provide order-data analytics layer that complements vendor systems and bridges surveillance, reporting, and risk controls. |
| No current TS capability | Build end-to-end foundation—data ingestion, lifecycle baselines, anomaly scoring—and expand iteratively. |
Applying anomaly-detection science to order-book behaviour enables firms to:
A Markov model describes a process that moves between a set of states, where the probability of the next state depends only on the current state, not on the full past history.
Mathematically, it assumes memorylessness:
In simple terms, once you know the current state, the next move is predictable by historical transition probabilities.
A Semi-Markov model extends this idea by including time, modelling how long the process tends to stay in each state before transitioning.
Each transition has both a probability (which next state) and a dwell-time distribution (how long until it happens).
For order-book data, that "how long" dimension is critical — abusive behaviour often reveals itself not just in what sequence of events occurs, but how fast they occur (e.g., cancellations after 50 ms instead of 500 ms).
Each order moves through well-defined lifecycle states:
New → Modify → Partial Fill → Full Fill / Cancel / Expire
A Markov model naturally represents these transitions as probabilities between states.
Every trader, desk, or strategy develops a distinctive transition pattern and rhythm.
Markov models learn these normal patterns quantitatively, so deviations (e.g., sudden rise in "New→Cancel" transitions) stand out immediately.
Transition matrices and dwell-time histograms are visual, interpretable artefacts: easy to show in audit or model-risk review.
They provide why something was flagged ("Cancel-after-Modify transitions +250% vs baseline").
No "black box" deep learning required at the start.
Only sequence and timing of order events are needed — no complex features, labels, or contextual enrichment — making them ideal for an initial deployment while establishing data pipelines and governance.
Markov and Semi-Markov computations scale linearly with event count; that's essential when processing order books that are 10–1000× larger than trade datasets.
These models form the quantitative benchmark against which advanced sequence-learning models (autoencoders, transformers) can later be validated and tuned.
For a typical trader–instrument pair:
| Transition | Baseline Probability | Recent Probability | Deviation |
|---|---|---|---|
| New → Modify | 0.35 | 0.33 | –0.02 |
| Modify → Cancel | 0.18 | 0.45 | +0.27 (↑) |
| New → Cancel (short rest < 200 ms) |
0.04 | 0.21 | +0.17 (↑) |
The spike in Modify→Cancel and short-rest cancels signals behaviour drift potentially associated with spoofing or quote-stuffing activity.
Semi-Markov analysis confirms the average dwell time before cancel dropped from 520 ms to 80 ms.
| Feature | Why it matters |
|---|---|
| Discrete state transitions | Naturally mirrors order lifecycles. |
| Timing sensitivity | Semi-Markov captures "speed of action," key in HFT abuse. |
| Explainable output | Transition probabilities and dwell times are transparent. |
| Low data prep requirement | Uses existing OMS/EMS order events directly. |
| Quantitative baseline | Foundation for advanced anomaly-detection and model validation. |
Markov and Semi-Markov models are the most logical and defensible starting point for AI-enhanced Trade Surveillance.
They allow firms to move from anecdotal, rule-based reasoning to quantitative behavioural baselines—a necessary first milestone before progressing to deep-learning or multi-venue correlation techniques.