Knowledge Distillation: A technique used in transfer learning that is based on the idea of training a smaller model to mimic the behavior of a larger pre-trained model.