This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# deit-base-distilled-patch16-224_a13567891499773952587369
该模型是一个蒸馏版的 Vision Transformer (ViT)。它除了类 token 外，还使用了蒸馏 token，以便在预训练和微调期间有效地从教师模型（CNN）中学习。蒸馏 token 通过自注意力层与类 token ([CLS]) 和图像块 token 交互，并通过反向传播进行学习。