Development of a Method for Decomposition of Embedding Vectors Using a Gaussian Mixture

Static embeddings represent connection between words, but high dimension of the vector spaces complicates interpretation of the encoded semantic relations. In this paper, we introduce a method for vector space decomposition using Gaussian mixture models, which create interpretable topic clusters of words and allow to analyze the obtained components. To assess the method, we used pretrained Word2Vec models. Gaussian mixtures provided a plausible space decomposition where the components were close to human judgment. The analysis included visualization of components on a plane, comparison of the resulting gaussians covariance matrices and the rating of the dimensions by their variance. It reveals that Gaussian mixtures are an effective instrument for building interpretable semantic components as it allowed us to obtain a set of interpretable features which could describe the lexical relations. The question of scaling the approach, though, remains open.