Automated Simplification of Texts and their Classification by Language Proficiency

The aim of the study is to create transformer-based models for text complexity classification and text simplification. The work considers existing datasets, metrics and approaches to classifying texts by a language proficiency level and to simplifying them. Using agile methodology, web scraping, exploratory data analysis, visualization, fine-tuning and automated evaluation, the author achieves the following results. A dataset for text complexity classification and text simplification is collected. Fine-tuning is performed for eight encoder models for text complexity classification and eighteen encoder-decoder models for text simplification. The developed application estimates reading time, extracts keywords, analyzes vocabulary and grammar, determines the complexity of the text, simplifies it and generates exercises.