Convolutional Models for Audio Source Separation in Streaming Setting

The research project was dedicated to the creation of a deep neural network for online target source separation. In order to solve this problem, we adapted one of the state-of-the-art convolutional neural networks for the offline task, SpexPlus, to the online mode. We developed several memory mechanisms for the architecture to improve its quality by expanding the captured context during sequential processing of audio chunks. The proposed methods were tested in experiments. We continued to work with the model with the most suitable memory mechanism, conducted a study on how much we can compress this architecture structurally (by changing its hyperparameters) and in case of applying standard techniques for model size reduction (pruning and quantization) to it. We investigated how network's compression affects its quality, weight and speed of inference. This paper describes the ideas from other studies that were used as a basis for creating memory mechanisms, the course and results of our research, the problems and features of models' training that we faced.