Abstract: In this study, we explore the potential of utilizing transformers in a limited data setting. We propose a teacher-student framework to train transformers for classifying lung diseases using ...