123456789101112131415161718192021222324252627 |
- Building Custom Policy Classes
- ------------------------------
- .. currentmodule:: ray.rllib
- .. warning::
- As of Ray >= 1.9, it is no longer recommended to use the ``build_policy_class()`` or
- ``build_tf_policy()`` utility functions for creating custom Policy sub-classes.
- Instead, follow the simple guidelines here for directly sub-classing from
- either one of the built-in types:
- :py:class:`~policy.eager_tf_policy_v2.EagerTFPolicyV2`
- or
- :py:class:`~policy.torch_policy_v2.TorchPolicyV2`
- In order to create a custom Policy, sub-class :py:class:`~policy.policy.Policy` (for a generic,
- framework-agnostic policy),
- :py:class:`~policy.torch_policy_v2.TorchPolicyV2`
- (for a PyTorch specific policy), or
- :py:class:`~policy.eager_tf_policy_v2.EagerTFPolicyV2`
- (for a TensorFlow specific policy) and override one or more of their methods. Those are in particular:
- * :py:meth:`~policy.policy.Policy.compute_actions_from_input_dict`
- * :py:meth:`~policy.policy.Policy.postprocess_trajectory`
- * :py:meth:`~policy.policy.Policy.loss`
- `See here for an example on how to override TorchPolicy <https://github.com/ray-project/ray/blob/master/rllib/algorithms/ppo/ppo_torch_policy.py>`_.
|