1234567891011121314151617 |
- Transformer Kernels
- ===================
- The transformer kernel API in DeepSpeed can be used to create BERT transformer layer for
- more efficient pre-training and fine-tuning, it includes the transformer layer configurations and
- transformer layer module initialization.
- Here we present the transformer kernel API.
- Please see the `BERT pre-training tutorial <https://www.deepspeed.ai/tutorials/bert-pretraining/>`_ for usage details.
- DeepSpeed Transformer Config
- ----------------------------
- .. autoclass:: deepspeed.DeepSpeedTransformerConfig
- DeepSpeed Transformer Layer
- ----------------------------
- .. autoclass:: deepspeed.DeepSpeedTransformerLayer
|