A strong backtest can be the result of a useful idea, a favorable historical period, an implementation mistake, or repeated tuning until a convincing pattern appears. The final chart rarely reveals which explanation is most plausible. Alpha research therefore needs a record of the process that produced the result, not only the result itself.
This article proposes a practical workflow for evaluating an altcoin strategy. It does not present a profitable model or imply that following the workflow will generate returns. The goal is to make evidence easier to challenge. Begin with the Altcoin Hedge Alpha Strategies page for the distinction between an investment hypothesis and a demonstrated outcome.
1. State the hypothesis before opening the results
Explain why the proposed strategy might earn a return. Is it compensation for supplying liquidity, bearing a particular risk, or reacting to information that is not fully reflected in prices? A rule that can be described only as “the parameters that looked best” does not yet have an economic explanation.
Specify the universe, signal, holding period, portfolio constraints, and benchmark before evaluating the final test sample. Record what evidence would contradict the hypothesis. A strategy designed to reduce market exposure should be judged differently from one designed to capture broad market gains.
The research paper Deep Reinforcement Learning for Cryptocurrency Trading: Practical Approach to Address Backtest Overfitting examines overfitting in a specific crypto trading research setting. Its experiment does not validate every model or establish future profitability. It illustrates why evaluation must consider whether apparent historical success reflects fitting the sample rather than a relationship that generalizes.
2. Build a point-in-time view of the universe
At each simulated decision date, include only information and assets that would actually have been available. A universe made from today's surviving tokens can exclude failures and delistings that a historical investor could have held. That can make a strategy appear more resilient than a contemporaneous implementation would have been.
Track listings, removals, ticker changes, redenominations, and changes in contract specifications. A price series that silently joins different economic instruments can create returns that no trader could earn. Record the data source and transformation rules so that another researcher can reproduce the series.
Availability includes publication time. An announcement date, data timestamp, or later corrected observation may not be the time at which the information became usable. Distinguish the event, the publication, and the decision. When timing cannot be verified, label the limitation and test a conservative delay rather than assuming immediate knowledge.
3. Prevent information from leaking into the past
A signal should use only information available before the simulated order. Computing a signal from a day's closing price and filling at that same close can be unrealistic when the closing observation was not yet known at order time. The necessary execution delay depends on the actual strategy and venue.
Preprocessing can also leak information. Normalizing data with statistics calculated over the full sample lets future observations influence earlier decisions. Fit those transformations on the permitted training data, then apply them to later data without using future values. Treat missing data and revised observations with the same care.
A simple leakage example
A hypothetical example makes the issue visible: a rule selects tokens using their returns over the next seven days and reports buying the winners today. That is not forecasting; it is using the outcome to choose the trade. Less obvious forms of the same error can hide in labels, filters, and data-cleaning steps.
4. Separate exploration from evaluation
Divide the research process into stages with different purposes. An exploratory sample supports idea development. A validation process helps compare candidate designs. A genuinely untouched test period evaluates the final locked design. Once a test result has influenced another change, that period is no longer untouched evidence.
Chronological evaluation matters because a trading strategy operates forward through time. A walk-forward process can repeatedly fit on earlier information and evaluate on later intervals. Any labels or positions overlapping a boundary require careful handling so that the separation is meaningful rather than just a change in file names.
Keep a trial ledger. Record not only the final settings but also discarded indicators, universes, lookback windows, and filters. Trying many variants creates more opportunities to find an impressive result by chance. Reporting a single winner without its selection history hides an important part of the evidence.
5. Model the trade rather than the idealized signal
Translate the signal into orders, positions, cash flows, and costs. Include fees, bid-ask spreads, financing or funding, contract sizes, and turnover. For short positions, investigate whether the exposure could actually be established and maintained. A negative portfolio weight in code is not proof that a corresponding trade was available.
Use conservative execution assumptions appropriate to the data's resolution. Daily closing prices do not reveal the exact intraday depth available for a large order. A backtest should not claim precision that the input data cannot support. Where execution is unknown, show sensitivity to a range of clearly labeled assumptions.
For example, an invented strategy earning 0.10% per completed round trip before costs would lose money if realistic round-trip costs were 0.15%. A smooth gross equity curve cannot resolve that arithmetic. The cash-and-carry analysis shows a similar distinction between a quoted spread and a net economic result.
6. Inspect where returns and losses come from
Compare the strategy with an appropriate baseline using the same period, currency, and cost treatment. A model with broad directional exposure should not be called alpha merely because it gained during a rising market. Analyze whether the intended mechanism, rather than an incidental exposure, explains the result.
Review returns across periods, assets, and market conditions without turning every subgroup into a new optimization exercise. Check whether a small number of trades accounts for most of the profit. Inspect drawdowns, turnover, capital requirements, and the time spent underwater, not just an annualized average.
Use parameter sensitivity to look for fragility. If a minor, reasonable change destroys the result, investigate whether the original setting fitted noise or exploited a data artifact. Stability is useful evidence, but it is not a profitability guarantee. A broad family of similar-looking strategies can all share the same flawed assumption.
7. Move from historical testing to observable operations
A paper-trading period can reveal differences between modeled and observed signals, fills, funding, and data availability without committing live capital. Define in advance what is being measured and how discrepancies will be investigated. Do not quietly alter the benchmark or fill assumptions to make the paper record agree with the backtest.
Keep deployment separate from proof. A technically functioning strategy can still lose money, and a small live sample does not establish a durable edge. Any decision to use capital requires its own risk limits, appropriate permissions, and consideration of the investor's circumstances. Research evidence does not replace those decisions.
The portfolio stress-testing framework adds scenarios that a historical sample might not contain. Combine empirical observations with explicit failure cases such as unavailable data, broken execution, rapidly changing costs, and funding needs. Both kinds of evidence have limitations; together they expose more assumptions than either alone.
Conclusion: preserve the evidence trail
A credible research package includes the hypothesis, point-in-time data rules, trial history, validation design, cost model, results, and known limitations. It distinguishes hypothetical performance from actual results and labels every assumption that materially changes the conclusion. The final chart is only one page of that package.
The purpose of this discipline is to reduce avoidable self-deception, not to manufacture certainty. An idea that fails a realistic test has produced useful information. The strongest research decision may be to reject a strategy, simplify it, or gather better data rather than turn an attractive backtest into an unsupported promise of alpha.
