Quantitative Trading with Machine Learning

Define an experiment before choosing an algorithm

1.1 Prediction time determines usable information

Specify the decision timestamp, universe, holding horizon and available action. Predicting tomorrow’s close-to-close direction is not the same as earning that return: a signal formed after today’s close cannot necessarily trade at today’s close. Define the first executable price and the rules for missing, suspended and delisted securities. Keep unsuccessful securities in historical universes where appropriate; a dataset containing only today’s survivors overstates what a historical investor could have selected.

1.2 A target must match an executable decision

A supervised target can be a future return, direction or event. Record exactly which future interval creates each label. Features might include lagged returns, volumes, volatility or dated disclosures, but their values must have been available at prediction time. Distinguish an economic period-end from the publication date of its results. For news sentiment, preserve the original release timestamp, language, revisions and duplicate stories before claiming a text-based signal anticipates price.

Step-by-step method

1. Write the decision, action and target interval explicitly.
2. Build a point-in-time universe with data lineage.
3. Verify release times and the first realistic execution opportunity.

Engineer features and choose proportionate models

2.1 Transformations can leak information

Fit scaling, imputation, feature selection and text vocabularies on training data only. A pipeline helps keep those operations inside each training fold. Lag a rolling indicator consistently with the decision timestamp; a full-day high is unavailable at midday. Avoid interpolation that uses future observations to fill past values. Join macro releases by their actual availability and preserve revised versus originally published data when the distinction affects the experiment.

2.2 Complexity should answer a research need

Start with a no-change or majority-class baseline and a simple regularised model. Random forests combine decision trees and can represent nonlinear interactions, but correlated noisy features still invite overfitting. LSTM networks model sequences with learned state; they require justified sequence construction and enough relevant data. Reinforcement learning chooses actions based on a reward and simulated environment, so unrealistic fills or rewards can teach exploitation of the simulator rather than a usable strategy.

Step-by-step method

1. Create a feature-availability audit for every input.
2. Fit all learned transformations within the training window.
3. Compare advanced models with a reproducible simple baseline.

Validate chronologically and isolate model selection

3.1 Walk forward into genuinely later observations

Randomly mixing financial rows can train on the future or place near-duplicate observations on both sides of a split. Walk-forward evaluation trains on an earlier window and tests on a later one, repeating with either expanding or rolling training history. Keep an untouched final period for a final research decision. Hyperparameters and feature choices belong inside a nested training-and-validation process, not in repeated adjustments to the final holdout after viewing results.

3.2 Purge overlapping labels and impose a justified gap

Labels covering several future days can overlap a test interval even when their feature dates are earlier. Purging removes training observations whose label intervals overlap the protected evaluation interval. A gap or embargo separates adjacent dependent observations where justified; its length must follow the label horizon and information dependencies, not an arbitrary universal number. A library’s chronological splitter and gap option are useful building blocks, but do not automatically audit label overlap, cross-asset grouping or all forms of leakage.

Step-by-step method

1. Draw training, label and evaluation intervals on a timeline.
2. Purge intersecting label intervals and justify any gap.
3. Freeze choices before using the final holdout once.

Evaluate probabilities and economics together

4.1 Accuracy alone can conceal failure

Compare class balance, confusion matrices, probability calibration and suitable loss measures. A model can achieve high accuracy by always predicting the majority outcome. A well-calibrated probability means outcomes occur at approximately the stated frequency across comparable predictions, not that any individual prediction is certain. Assess performance across market regimes, securities and dates rather than allowing a few exceptional observations to dominate the conclusion. Report uncertainty and the number of experiments attempted.

4.2 Costs, turnover and capacity determine feasibility

Translate predictions into a predeclared position rule, then simulate fees, relevant taxes, spread, slippage, borrow constraints, participation limits and realistic latency. Teaching example: an assumed gross edge of12 basis points per trade becomes9,5,1 and−3 basis points when round-trip costs are3,7,11 and15 basis points. Increasing turnover can erase a small signal. Unavailable shorting or insufficient order-book depth cannot be solved by assuming frictionless execution.

Step-by-step method

1. Evaluate baseline-relative statistical performance and calibration.
2. Apply an executable decision rule to unseen observations.
3. Stress costs, delays, turnover and position capacity.

Document, monitor and stop a failed experiment

5.1 Reproducibility is part of the evidence

Save source provenance, licences, timestamp rules, code version, parameters, random seeds and split dates. Record unsuccessful experiments so that the research history does not become a catalogue of winners. Explain why the feature should contain relevant information and test its sensitivity to small design changes. An apparently strong result that vanishes with a different start date or realistic fill assumption needs investigation before any operational consideration.

5.2 Deployment changes the risk problem

Begin with paper evaluation and independent review, not automatic orders. Any later live use requires authorised broker access, applicable permissions, position controls and a tested operational process. Monitor data completeness, prediction distribution, calibration, realised net outcomes and changing market conditions. Define a pause rule for broken inputs, excessive drawdown, abnormal orders or model drift. Retraining does not automatically repair deterioration; it creates another model that needs appropriate validation and version control.

Step-by-step method

1. Package the experiment so another reviewer can reproduce it.
2. Run a forward paper test with fixed rules and realistic costs.
3. Set monitoring thresholds, authorised overrides and a retirement rule.

A small assumed edge can disappear after costs

Reference table

Glossary of key terms

Calibration

Agreement between predicted probabilities and observed frequencies over comparable cases.

Embargo

A deliberately excluded separation interval used to reduce information dependence around evaluation.

Feature

An input available when a prediction is made.

Label

The outcome a supervised model is trained to estimate.

Leakage

Use of information unavailable at the decision time or improperly shared across evaluation boundaries.

Overfitting

Learning incidental sample patterns that fail to generalise.

Purging

Removing observations whose outcome intervals contaminate a protected evaluation interval.

Walk-forward evaluation

Repeated training on earlier data and testing on later data under fixed chronological rules.

Frequently asked questions

Common pitfalls