Distillation

We provide the user with multiple classes to distil a teacher model into a student one. There are some already implemented distillation logic. However, if you want to use a custom technique you simply need to extend the BaseDistiller class.

bert_squeeze.distillation

Distillation Utilities

• Distillation Utilities