bertsqueeze

Project

Bert-squeeze is a repository aiming to provide code to reduce the size of Transformer-based models or decrease their latency at inference time.

It gathers a non-exhaustive list of techniques such as distillation , pruning, quantization, early-exiting, … The repo is built using PyTorch Lightning and Transformers.

Content

Getting started

• Installation
• Training
  • Training a simple Transformer model
  • Training FastBert
  • Training TheseusBert
• Distillation + Pruning + Quantization

Bertsqueeze

• Overview
• Concepts
• Assistants
• Distillation
• Models

API References

• API
  • Assistants
  • Models
  • Distillation
  • Data
  • Inference
  • Utilities
• GitHub Repository

Indices and tables

• Index

• Module Index