# What changed in tianshou

> 57 changes in tianshou 1 to 2 from its official release notes, upgrade guides and advisories: 38 breaking, 14 new APIs and options. Each one cited. Sources checked 2026-10-29. HTML: https://unstale.sh/pypi/tianshou/

To see which of these apply to a repository, run `npx @unstale/cli init` in it.

## tianshou 2

### Breaking changes

- Compatibility was adjusted to require pandas versions below 3. (2.0.1) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.1)
- Interfaces are more well-defined and usable, parameters are extensively documented, and some parameter names are made more consistent and intuitive. (2.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.0)
- The revised hierarchy separates on-policy, off-policy, and offline algorithms at the type level and makes inheritance relationships meaningful. (2.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.0)
- Replaces the former BasePolicy abstraction; instantiate an Algorithm and pass it a Policy. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed to Actor. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed to max_grad_norm for consistency. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed from continuous.Actor. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed from continuous.ActorProb. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Removed as a member and constructor argument from the listed network and module classes. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed to gamma. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed to n_step_return_horizon. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Replaces trainer iterator semantics; manual trainer use should call reset followed by execute_epoch. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Removed from algorithm constructors; schedulers are handled through the factory abstractions. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Training parameters are now passed through this dedicated configuration object instead of directly as keyword arguments. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed from OffpolicyTrainer; trainers now use dedicated configuration objects. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Training parameters are now passed through this dedicated configuration object instead of directly as keyword arguments. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed from OnpolicyTrainer; trainers now use dedicated configuration objects. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Training parameters are now passed through this dedicated configuration object instead of directly as keyword arguments. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Algorithms now require optimizer factories rather than torch.optim.Optimizer instances; the factory creates optimizers internally. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Agents persisted with earlier versions cannot be loaded in v2. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Introduced as the policy abstraction optimized by an Algorithm. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Renamed from reset_buffer in run and reset for clarity. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Replaces run's reset_prior_to_run parameter; run now replicates reset's parameters. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Replaced by TrainingConfig and on-policy/off-policy subclasses with aligned parameter names. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Replaced by set_eps_training and set_eps_inference. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- The default changed from True to False, avoiding assumptions that collection-step scores are meaningful for early stopping. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- The package formerly named tianshou.policy was renamed to tianshou.algorithm, reflecting the separation between learning algorithms and policies. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Atari helper modules atari_network and atari_wrapper moved into this library package. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Replaces SamplingConfig as the high-level training configuration abstraction. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Replaces the trainer parameter update_per_step with a clearer name. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)

### New APIs and options

- A separate Algorithm abstraction establishes a clear separation between learning algorithms and policies. (2.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.0)
- Discrete Q-learning policies accept this parameter to set epsilon for inference separately from training. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Discrete Q-learning policies accept this parameter to set epsilon for training separately from inference. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Adds an option to customize the collector factory. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Factory abstraction for creating learning-rate schedulers through the optimizer factory setup. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- Introduced as the trainer specialization for offline learning, with its own configuration class. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)
- A separate Policy abstraction establishes a clear separation between learning algorithms and policies. (2.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.0)
- Lets an Algorithm directly initiate the learning process. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)

### Fixes and other changes

- Experiment.run now returns an ExperimentResult with trainer_result set. (2.0.1) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.1)
- Original policy parameters are now restored when line search fails. (2.0.1) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v2.0.1)
- Tianshou v2 is compatible with Python 3.12. (2.0.0) — [upgrade guide](https://github.com/thu-ml/tianshou/blob/master/CHANGELOG.md)

## tianshou 1

### Breaking changes

- The internal attribute critic1 was renamed to critic. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Support for gym environments is dropped; only Gymnasium environments are supported. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Gymnasium environments are now the only supported environment type. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Several signatures no longer accept **kwargs, including policy constructors. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Trainer functions such as offpolicy_trainer are removed in favor of the OffpolicyTrainer class and its run method. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Use the OffpolicyTrainer class and its run method in place of trainer functions such as offpolicy_trainer. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Python versions below 3.11 are no longer supported. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- Training methods return dataclasses instead of dictionaries. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)

### New APIs and options

- A method computes an action directly from a policy's observation and can be used for unrolling. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- BatchPrototypes covers fields needed and returned by methods relying on batches in a backwards-compatible way. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- CalQL is supported as part of CQL. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- High-level interfaces for experiments are available. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- ReplayBuffer supports custom keys. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)
- The multiprocessing context can be set explicitly for SubprocEnvWorker. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)

### Behavior and default changes

- critic2 no longer needs to be explicitly constructed and passed when it should use the same network as critic. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)

### Fixes and other changes

- The implementation uses dist.mode instead of inferring loc or argmax from the dist_fn input. (1.0.0) — [release notes](https://github.com/thu-ml/tianshou/releases/tag/v1.0.0)