Home/Systematic Trading/Manager Selection and Due Diligence

Manager Selection and Due Diligence

model-buildingLevel 3 — Advanced

Prerequisites

Quant Research ProcessBacktesting DisciplineStrategy Lifecycle Management

What It Is

Manager selection and due diligence is the process of evaluating systematic and quantitative investment managers — including stress-testing their frameworks, interpreting return dispersion, assessing robustness, and distinguishing genuine edge from data-mined backtests.

Correct Execution

  • Stress test parameters and look-back windows: a robust strategy should show similar performance across a wide range of parameter choices
  • Evaluate dispersion within a category (especially managed futures) as a diagnostic tool — large dispersion means implementation details matter, not just the style
  • Demand attribution of returns: what percentage comes from each model specification, signal type, and portfolio construction choice
  • Prefer the simple model that has survived stress testing to the complex model with more moving parts
  • Assess client behavior management as a genuine performance attribute — the manager who helps investors stay invested earns more than the attribution suggests

Progression Levels

Diagnostic Tree

Coaching Cues

  • "I stress test every parameter, variable, and market in my program to try to break it. If I can't break it, I trust it." — Eric Crittenden, FWM S3E7
  • "Stand ten feet away from two equity curves. If you can't tell which is which, maybe you don't need the one with 16 moving parts." — Eric Crittenden, FWM S3E7
  • "Dispersion in managed futures is both the problem and the solution — it means you can build uncorrelated portfolios within the category." — Adam Butler, FWM S1E8

Common Errors

  1. Selecting managers on most recent 3-year returns: Recent performance in any strategy type is mostly regime luck → Evaluate on the entire history, compare to expected behavior for regime, not just raw returns
  2. Ignoring parameter sensitivity: A strategy with 15 parameters can always look good on the training set → Demand to see how the strategy performs with parameters shifted ±30%; robust strategies survive this
  3. Treating client behavior management as soft/unquantifiable: The manager who keeps investors invested during drawdowns is generating real alpha → Include behavioral/communication quality in manager evaluation, not just return statistics

Edges

🔑 Hidden Causal Lever

Managed Futures Dispersion Is a Feature — You Can Build Uncorrelated Portfolios Within the Category

model-buildingmanager-selection-due-diligence →

Managed futures is unusual in that two managers with identical philosophical approaches (trend-following) can have near-zero live return correlation. This is because signal parameters (look-back, smoothing), markets traded, and portfolio construction choices create different return streams from the same underlying philosophy. Most allocators see this dispersion as confusing and try to pick the "best" CTA. Sophisticated allocators use it intentionally: blending two uncorrelated CTAs with the same philosophy is a free diversification benefit within a single asset class allocation.

What most people do
Select a single "best" managed futures manager; treat dispersion as evidence of manager quality differences rather than implementation diversity.
What the best do
Select two or three CTAs with genuinely different implementation approaches (different look-backs, different market universes); blend them at the portfolio level to reduce idiosyncratic manager risk while maintaining the category-level benefit.
Why it's an edge: The diversification benefit available within a single asset class category is rarely understood by allocators who think of "managed futures" as a monolith. You can improve portfolio Sharpe by blending within the category.
How to exploit: Request full disclosure of signal parameters (look-back periods, smoothing methods, markets traded) from any CTA you evaluate. Compute the expected correlation between managers with different parameters using academic factor models. Build a 2-3 manager CTA portfolio that maximizes parameter diversity, not just recent return difference.
Cross-domain parallel
In sports betting, using multiple models with different input weightings and blending their predictions produces better calibration than any single model — the same principle applies to blending systematic trading programs.
Adam Butler, "Liquidity Premium," FWM S1E8, 2021-04-10; Eric Crittenden, FWM S3E7, 2021-04-10
⚡ Conventional Wisdom Is Wrong

Removing a Manager After a 2-Year Drawdown Is the Worst Possible Decision

model-buildingmanager-selection-due-diligence →

The institutional response to manager underperformance is to remove the allocation and reallocate to better-performing managers. For systematic strategies, this is backward: a 2-year drawdown in a trend-following strategy that has a 3-year expected drawdown cycle is a buy signal, not a sell signal. The strategy is closest to its expected recovery; recent underperformance has removed the crowded long positions from the strategy; the next period should, in expectation, be better than average. Removing at the bottom locks in the loss AND misses the recovery.

What most people do
Establish redemption triggers based on absolute performance ("if strategy underperforms for 24 consecutive months, redeem"); remove manager during the worst phase; reallocate to a manager that has recently outperformed.
What the best do
Establish redemption triggers based on behavioral deviation from the strategy — "if the strategy no longer exhibits the characteristics I expected (trend-following behavior, appropriate drawdown profile), I consider redemption." Recent poor performance within expected distribution is explicitly NOT a trigger.
Why it's an edge: The investors who hold managed futures through 2009-2013 underperformance positioned themselves for the 2014-2019 recovery. Those who redeemed in 2012 after 3 years of underperformance missed the entire recovery. This pattern repeats in every systematic strategy category.
How to exploit: Before investing in any systematic manager, document in writing: "Expected drawdown range: X% to Y%. Expected drawdown duration: 12-36 months. Redemption trigger: behavioral deviation from strategy philosophy, NOT relative performance vs benchmark or absolute drawdown within expected range." Review this document at every board meeting where the manager is discussed.
Cross-domain parallel
In sports betting model management, abandoning a positive-EV model during a losing streak (within statistical expectations) destroys the edge. The Kelly Criterion explicitly addresses this: the edge is in the model, not in the recent results.
Eric Crittenden, FWM S3E7, 2021-04-10 — "removing a trend-follower in 2012 because it has underperformed since 2008 is the classic investor error."
💎 Elite-Only Behavior

Parameter Sensitivity Testing Is the Real Due Diligence — Everything Else Is Marketing

model-buildingmanager-selection-due-diligence →

Manager presentations showcase the best-performing parameter set and the most flattering time period. The only way to distinguish genuine edge from backfit is to stress-test the parameters: shift look-back windows ±30%, change rebalancing dates, alter threshold levels. A robust strategy looks similar across this parameter space — a "1940s jeep" that survives all conditions. A backfit strategy produces sharp peaks in parameter space that disappear with small perturbations. Most allocators never run this test; they evaluate on the manager's preferred presentation of data.

What most people do
Evaluate managers on their presented equity curve, live track record, and Sharpe ratio; accept the manager's choice of parameters as "well-researched."
What the best do
Replicate the manager's strategy (approximately) and run parameter sensitivity analysis before any due diligence meeting; use the results to identify whether performance is concentrated in a narrow parameter regime.
Why it's an edge: Managers who know allocators run sensitivity analysis maintain robust strategies rather than backfit ones; managers who know allocators don't run it have less incentive to be robust. Being known as a rigorous allocator improves the quality of managers who seek your capital.
How to exploit: For any systematic manager under consideration, request the backtested performance at the parameters used AND at ±30% of each key parameter (look-back, vol target, position limit). If they cannot or will not provide this, treat it as a red flag. Build the sensitivity table from their data; evaluate whether the equity curve at median parameters looks similar to the presented one.
Eric Crittenden, FWM S3E7, 2021-04-10 — "I stress test every parameter, variable, and market in my program to try to break it."
🔑 Hidden Causal Lever

Client Behavior Management Is Real Alpha — The Manager Who Keeps Investors Invested Earns More

model-buildingmanager-selection-due-diligence →

A manager who helps clients maintain discipline through a 3-year drawdown generates genuine excess return compared to the mathematically superior manager whose clients exit at the bottom. This "behavioral alpha" is real, measurable, and systematically underpriced in manager evaluation because it doesn't appear in return attribution.

What most people do
Evaluate managers purely on risk-adjusted returns. Choose the highest Sharpe ratio. Ignore the manager's communication style and client management.
What the best do
Evaluate behavioral alpha explicitly: How does the manager communicate during drawdowns? What is their client retention rate through the worst 2-year period? A manager with 0.7 Sharpe and 95% client retention may deliver more lifetime return than a 1.0 Sharpe manager with 60% retention through drawdowns.
Why it's an edge: The gap between "return generated" and "return experienced by investors" is enormous in volatile strategies. The manager who closes this gap through behavioral management delivers more actual wealth to clients — which is what return is for.
How to exploit: Add a behavioral alpha screen to manager evaluation: request client retention data through worst drawdown, review investor letters during crisis periods, and ask about proactive communication frequency. Weight this alongside quantitative metrics.
From Progression Level 4 and Common Errors #3 — behavioral alpha in manager evaluation

Sources

  • Eric Crittenden, "All-Weather Portfolios with Trend Following," Flirting with Models S3E7, 2021-04-10 — manager evaluation framework; parameter stress testing; simplicity as robustness; crisis alpha limitations
  • Adam Butler, "Liquidity Premium," Flirting with Models S1E8, 2021-04-10 — signal shape diversification; using managed futures dispersion constructively
  • Adam Butler, "Questioning the Quant Orthodoxy," Flirting with Models S5E13, 2022-10-03 — factor attribution in manager evaluation