A smaller student model is trained to imitate a larger teacher, using the teacher's outputs (text or probability distributions) as training targets.
It gives much of the teacher's quality at a fraction of the latency and cost, and is widely used to create small task-specific models.
Check the licensing terms for using a model's outputs as training data, and evaluate the student on your real tasks.
Loading…