Research method and evidence discipline
Financial strategy research has an unusually large researcher-degrees-of-freedom problem: assets, windows, indicators, thresholds, costs, labels, splits, and metrics can all be varied. White's Reality Check was developed to test whether the best result in a family beats a benchmark after data snooping, while Bailey and López de Prado's Deflated Sharpe Ratio adjusts for selection bias, non-normal returns, and the effective number of trials. The practical conclusion is not that one statistic makes a strategy valid; it is that the complete search path must be recorded and the final period must remain untouched. Freqtrade separately warns that full-dataframe backtests can leak future information and provides lookahead analysis. Its recursive analysis checks whether indicator values depend materially on the available startup history. These tools are necessary falsification checks, not proof of tradability.
Point-in-time market data, liquidity, and survivorship
Current exchange endpoints are snapshots of today's market structure, not historical membership. Binance documents that its depth endpoint returns bid/ask price and quantity plus an update ID, and that symbol status can be TRADING, HALT, or BREAK. That is enough to record current quoted liquidity, but it does not reconstruct what was listed, tradable, or visible at an earlier date. The crypto-specific survivorship study by Ammann et al. used 3,904 assets and found that omitting failed/delisted assets can materially distort results, especially equal-weighted portfolios. A current 50-asset list therefore cannot support historical cross-sectional claims without dated membership and delisting returns. Order-book research also shows that volume alone is not an execution-cost measure; spread, depth at the intended notional, venue fragmentation, and feed delay matter. Displayed depth is cancellable and is not a fill guarantee.
Technical analysis, trend, momentum, reversal, and baselines
Simple moving-average and trading-range rules have a long empirical literature. Brock, Lakonishok, and LeBaron found signal differences in a long Dow sample using bootstrap tests, but that result is neither crypto evidence nor a license to reuse their parameters. Crypto studies report trend and momentum effects in some periods, while local CryptoKingpin work already shows large regime dependence and no family robust across both tested eras. Published effects should therefore define small theory families, not create a parameter-mining menu. A valid TA test asks whether a low-complexity, lagged rule produces incremental net expectancy against buy/hold, cash, and an always-enabled version of the same portfolio. A complex ML model must also beat the corresponding rule-only signal; otherwise the model has not demonstrated incremental value.
Bounded martingale, position sizing, stops, and tail risk
A martingale-style size increase does not create predictive edge. It changes the distribution of outcomes: many small recoveries are exchanged for a rarer, larger loss. Finite capital, exchange minimums, gaps, correlated losses, and maximum-bet constraints break the infinite-bankroll story. Kelly's growth framework sizes exposure from a positive informational advantage; it does not make a non-positive-expectancy gamble profitable. Because estimated crypto edge is uncertain and non-stationary, full Kelly would itself be too aggressive for this project. Freqtrade warns that every position adjustment adds fees, may behave differently in backtests because callbacks run once per candle, and needs an explicit maximum number of entries. A stop-limit can also fail to fill through a fast move. The local one-recovery design must therefore be tested as a separate risk overlay against the same signals with no recovery order.
Paper execution, order lifecycle, and reconciliation
A signal is not an order, an order is not a fill, and a quoted price is not the realized execution price. Binance's user-data stream emits execution reports for order state changes. Freqtrade's order objects retain side, type, exchange order ID, status, cancellation reason, and their parent trade. Those event identities make exact reconciliation possible, but dry-run fills remain simulations and must not be represented as venue fills. Freqtrade's quickstart explicitly notes that backtests assume fills. Position-adjustment callbacks can cancel/replace open orders and behave more frequently in dry/live mode than in candle-based backtests. A credible paper ledger must therefore join the intended signal with every subsequent order version, partial fill, fee, cancel, stop, and final position state.
Performance measurement and backtest validity
No single metric is sufficient. Sharpe measures average differential return per unit of variability, but its interpretation depends on horizon, benchmark, serial dependence, and return distribution. Crypto returns and martingale-like payoffs can be skewed and heavy-tailed, making raw Sharpe and win rate especially incomplete. Deflated Sharpe explicitly addresses non-normality and selection across many trials. Expected shortfall focuses on losses beyond a chosen quantile and is more informative about rare damage than volatility alone. Freqtrade's backtesting documentation can export trades and signals for detailed analysis, but its quickstart warns that modeled fills differ from dry/live operation. The paper record must therefore report both expected backtest metrics and reconciled realized paper metrics under the same formulas.
Machine-learning validation, calibration, and model risk
FreqAI supports periodic retraining and large feature sets, but feature volume is not evidence of economic signal. Its labels intentionally use future outcomes; strict separation is therefore required so label information never enters features, preprocessing, thresholds, or neighboring folds. Scikit-learn's TimeSeriesSplit preserves order and provides a gap, while probability calibration documentation warns that calibration data must be independent from model fitting. Financial relationships drift. NIST's AI RMF organizes model risk around govern, map, measure, and manage, with ongoing monitoring and documented validation throughout the lifecycle. For this bot, the model is a candidate filter only: it must beat linear and rule-only baselines after costs, stay calibrated, and abstain when inputs are missing, stale, or dissimilar.
Stablecoin stress and cross-asset regime controls
Stablecoins are part of the market's funding and quote infrastructure, so a peg deviation can be a liquidity/risk event rather than a directional predictor. Research on high-frequency depegs reports larger jump and co-jump probabilities in non-stable crypto assets after Tether instability. Federal Reserve and BIS work emphasizes run risk, reserve/intermediary structure, and transmission across crypto and traditional funding markets. These mechanisms support testing a stablecoin stress gate, but not a simplistic rule that “USDT moves, therefore buy BTC/ETC.” Crypto assets also share market-wide exposures. Cross-asset features can accidentally restate the target's own contemporaneous move or create leakage if timestamps are not synchronized. Their best initial use is therefore exposure control: reduce or abstain during depeg, broad volatility, correlation, spread, or depth stress.
Paper readiness, operations, security, and human review
Freqtrade recommends dry-run before production and documents that API credentials are unnecessary for dry-run. Its database guidance separates dry-run and live records. NIST's Secure Software Development Framework calls for protected artifacts, recorded provenance, testing, vulnerability response, and continuous improvement. NIST's AI RMF adds continuous measurement and management of model risk. Together these support a staged readiness case, not automatic promotion. Operational reliability is part of strategy evidence. A profitable simulation that cannot recover from stale data, authentication loss, process failure, database corruption, or stop-order failure is not ready. Signed local Git history, root-owned services, watchdog state, append-only logs, and tested rollback are evidence only when restoration is actually exercised. Paper validation is not a global research lock. The active paper candidate must remain frozen, but separate discovery, challenger-ML, data-quality, and operational-proof lanes should continue. New candidates cannot silently replace or retune the active paper candidate.
