Statistics & Validation
This stream is for students who like breaking results. The job is to find out whether a signal still looks real after costs, new samples, different regimes and less flattering assumptions.
Example work
A few examples of what this stream could pick up inside SSI Labs.
01
Rerun a strategy with wider costs, delayed execution and stricter fill assumptions.
02
Split results by volatility, trend, liquidity or funding regime and explain what changes.
03
Check whether a tuned parameter works because of the market or because it was fitted to one sample.
Work
What this stream owns
- Separate in-sample and out-of-sample results.
- Design the checks a backtest has to pass before anyone trusts it.
- Look for look-ahead bias, survivorship bias, data leakage and overfitting.
- Summarise drawdown, turnover, exposure, volatility and transaction-cost sensitivity.
- Call out when a result breaks instead of dressing it up.
Preparation
What to build toward
- Probability, inference, regression, hypothesis testing and time-series basics.
- Python for analysis, especially pandas, NumPy, plotting and reproducible notebooks.
- Understanding of backtest traps: look-ahead, survivorship, multiple testing and regime dependence.
- Clear communication: what changed, why it matters and whether the original claim survives.
Fit
Who should look here
- You like finding the assumption that makes a result fall apart.
- You would rather be accurate than impressive.
- You like turning messy output into a clear yes, no or not-yet answer.
Reading
Further reading
A short list for understanding the work behind this stream.
- CFA Institute: diligence and reasonable basis
A clear standard for checking model assumptions, limitations and quantitative outputs before relying on them.
- How backtest overfitting leads to false discoveries
Bailey and Lopez de Prado on why historical strategy searches produce fragile results.
- A data science solution to the multiple-testing crisis
Lopez de Prado on selection bias, multiple testing and the Deflated Sharpe Ratio.
