Few-shot Learning Using Orthogonal Convolutions

This thesis explores the potential of orthogonal convolutions in few-shot learning models, a field where limited data availability imposes significant challenges. Orthogonal convolutions are known for their ability to reduce the Lipschitz constant of a network, leading to more stable training and improved robustness. Additionally, orthogonal convolutions have the potential to enhance a model's generalization capabilities, which is particularly important in scenarios such as few-shot learning where models must make predictions based on a small amount of data. This study compares and analyzes three specific methods for imposing orthogonality on convolutional layers: regularization of convolutions parameterized as doubly block-Toeplitz matrices, the use of skew orthogonal convolutions, and Jacobian regularization based on Hutchinson‘s estimator of Frobenius norm. The primary goal of this thesis is to investigate how orthogonalization techniques affect the performance of few-shot learning models. The novelty of this study lies in its investigation of the impact of different methods for orthogonalizing convolutions, which has never been done before in few-shot learning models, and the subsequent comparison of these methods with each other. Through this analysis, the thesis aims to contribute to the field of machine learning by providing insights into effective strategies for optimizing few-shot learning pipelines and advancing our understanding of this area of research.