How to Evaluate an Automated Trading Strategy's Real Performance

A single profitable week or month tells you almost nothing about whether a strategy has a real, repeatable edge — markets are noisy enough that even a strategy with no real edge at all will occasionally produce a good short-term result purely by chance. Genuine evaluation requires a large enough sample of trades, across a range of different market conditions, to distinguish skill from noise. Beyond raw profit, a serious review looks at the profit factor (gross profit divided by gross loss — a value comfortably above 1 across a large sample is a healthier sign than a single lucky month), maximum drawdown and how long it took to recover from it, the consistency of results across different periods rather than one outsized month carrying the whole track record, and how the strategy performed specifically during periods that were difficult for its style — a trend-following system's results during a choppy, range-bound stretch, for instance. It's also worth separating results into in-sample performance (during the period the strategy was developed and tuned) versus out-of-sample or live-forward performance (after the rules were locked and the strategy was simply run going forward). A large gap between the two — a beautiful backtest and a much weaker live result — is one of the clearest warning signs of overfitting. Finally, real evaluation accounts for realistic costs: actual spread, commission, and typical slippage for the broker and account type being used, not idealized frictionless fills. A strategy that only looks profitable under generous backtest assumptions and loses that edge once real trading costs are applied was never actually profitable — the backtest was simply understating what trading really costs.