Reinforcement-learning trading
Reinforcement-learning (RL) trading treats trading as a sequential decision problem: an agent observes the market state, takes an action (buy / sell / hold, size a position), and receives a reward (typically a function of returns), learning a policy that maximizes cumulative reward over an episode. It is the learned-strategy approach to algorithmic-trading — the decision layer is trained, not hand-coded.
The component decomposition
TensorTrade is the founding example and makes the standard RL-for-trading decomposition explicit:
- Action scheme — the agent’s move set.
- Reward scheme — the training signal (e.g. position-based returns); the benchmark to beat is usually buy-and-hold.
- Observer — what the agent perceives each step. This is where the lookahead discipline bites: the observation must contain only information available at that step.
- Exchange simulation + data feed — the simulated market (fills, commissions) the agent trains against, played as an event-driven episode.
Training uses general RL infrastructure (TensorTrade trains via Ray RLlib) over a Gym-compatible environment.
Two scales of implementation
- Library — tensortrade: composable Gym components you assemble into an agent. Lightweight.
- Platform — trademaster (NTU): a full ecosystem around the same idea — bundled market simulators, 8 datasets, a 13+ trading-specific algorithm zoo (DeepScalper, EIIE, SARL, DeepTrader… plus PPO/DQN/ SAC baselines), and a six-axis/17-measure evaluation toolkit — covering portfolio management, order execution, HFT, and market making, not just a single strategy. The heavyweight end of RL-for-trading.
Contrast with the other two layers
- vs prediction (Kronos): a forecaster outputs where the market goes; an RL agent outputs what to do — and can in principle learn execution and sizing a pure forecaster never expresses.
- vs rule-based execution (banbot): a bot runs a fixed strategy the human wrote; an RL agent discovers the strategy from reward. Both still need an event-driven, no-lookahead market to be trustworthy.
Caveats
RL on noisy, non-stationary financial data is notoriously hard — overfitting to a backtest and distribution shift between training and live markets are the standing risks (the founding source is a framework, not an independent performance result).
Related
tensortrade · algorithmic-trading · event-driven-trading · backtesting · kronos-financial-foundation-model · synthesis