HSE Student to Present at Association for Computational Linguistics Conference

© Essentials/ iStock collection

An article by HSE Faculty of Computer Sciences senior Artyom Gadetsky was accepted for presentation at the Association for Computational Linguistics international conference, the only A* level conference on computational linguistics. According to the CORE system, which ranks large conferences in computer science, the category A* is the highest level for a conference.

The ACL conference has taken place every year since 1962. It features research related to natural language processing, including dialogue systems, machine translation, and various generative text models. On average, around 20% of all works submitted are accepted for presentation at the conference, with 384 works making it in out of 1,544 submissions.

Artyom prepared his paper together with Faculty of Computer Science Professor Dmitry Vetrov and Ilya Yakubovsky, a researcher at the company Joom. The paper, ‘Conditional Generators of Words Definitions,’ will be presented at the conference, which is set to take place in Melbourne, Australia, on July 15-20.

About the Article

Text oftentimes serves as data in tasks involving machine learning. Such text is usually broken up by words and then transformed into a vector – a set of numbers with which machine learning algorithms can already work comfortably. Artyom Gadetsky’s work researches this sort of vector representation of words. One vector is usually used for a single word, but since there are polysemantic words, it is not obvious that this representation stores information on all meanings of a word. The paper presents a model for generating word meanings. With the help of this model, various definitions can be generated for different meanings of the same word, with an example of word usage serving as the foundation. See below for an example of how this works with simple words such as star, sentence, and head:

 

Word	Usage Example	Generated Definition
star	she got star treatment	a person who is very important
star	bright star in the sky	a small circle of a celestial object or planet that is seen in a circle
sentence	sentence in prison	an act of restraining someone or something
sentence	write up the sentence	a piece of text written to be printed
head	the head of a man	the upper part of a human body
head	the leader of organization	with the highest rank or position

The research also showed that one such representation contains information about numerous possible meanings. In addition, there is reason to believe that separate components of the word vectors could be responsible for parts of speech and various other word properties.

Artyom Gadetsky, fourth-year student in the Applied Mathematics and Information Science programme

Building interpretable models is an important field that is still developing, as a full understanding of how such methods work allows you to find limitations and fix them. Our work is a step towards understanding how neural networks learn.

 

Dmitry Vetrov, Head of the Samsung-HSE Laboratory

This work is an excellent experimental confirmation that modern neural networks extract much more information on data than previously believed. A neural network not only understands the meanings of words depending on context, but is also able to formulate its own understanding in human language. This opens up new opportunities to build interpretable AI models, where a neural network, aside from decision-making, immediately generates a textual description of why it made this decision in particular. The technology proposed in our article presents both a concrete practical use in the financial sphere, for example, as well as a more fundamental use. It does this by showing that even relatively primitive (as far as fully fledged AI is concerned) machine learning technologies have unexpected properties that are traditionally attributed to higher nervous activity. This means that another step has been taken towards creating a complete artificial mind.