Specialists from the HSE Institute of Education Confirm GigaChat’s Erudition in Social Sciences
A multimodal neural network model by Sber, under the supervision of HSE University’s expert commission, has successfully passed the Unified State Exam in social studies. GigaChat completed all exam tasks and scored 67 points.
This result exceeds the minimum score for applying to university (45 points) and the average score for the subject in 2023 (56.4 points).
Denis Filippov, Vice President of Salyut Digital Surfaces, Sberbank, spoke about this at the AIJ 2023 conference.
The tests were held to check the capabilities of the updated version of GigaChat, which is based on a highly advanced model of the Russian language with 29 billion parameters.
To conduct the experiment, the chosen subject was social studies—a school discipline that covers the fields of economics, law, and social sciences. Thus, the model’s ability to successfully pass the exam demonstrates a high degree of maturity in its awareness of social norms, as well as economic and legal laws.
To test GigaChat’s knowledge, the team used only relevant test tasks for 2024, which were posted on the website of the Federal Institute of Pedagogical Measurements. Before the experiment, the team made sure that these tasks were not used to pre-train the model. GigaChat’s answers were checked first by an independent expert from HSE University, and then by an expert commission from HSE University’s Institute of Education. The correctness of the task setting and the reliability of GigaChat’s factual answers were assessed, as was the quality of performing creative (open) tasks.
Denis Filippov, Vice President of Salyut Digital Surfaces, Sberbank
‘It is important for us to evaluate the effectiveness of GigaChat not only by technical metrics, but also from the viewpoint of an ordinary person: whether the service is able to help in any particular area of knowledge and how smart and creative the model is. Tests used in the educational system, including the Unified State Exam, are well suited for such an assessment. The exam results show that GigaChat is “well-read” in the field of social sciences. This means that our artificial intelligence “understands” the basic laws of society and has a good sense of morality issues. This is further evidence that our service can be used to solve real-life fact-related problems: just ask a question in a natural way, and GigaChat will give you an accurate answer or help you understand a complex topic.’
Evgeniy Terentev, Director of HSE University’s Institute of Education
‘Our experts assessed GigaChat’s knowledge independently of Sber research and engineering teams. We checked the answers in the same way as if they were given by an ordinary high school graduate. The results show that the neural network model not only has a sufficient level of factual knowledge, but is also capable of thinking logically and choosing the best solution possible.’
Soon, anyone interested will be able to conduct an experiment similar to that conducted jointly with HSE University: GigaChat developers are preparing a special script for publication on GitHub. It will allow users to test how Sber’s neural network model passes the Unified State Exam with ‘one button’, without the need to manually enter the texts of tasks.
See also:
Neural Network Developed at HSE Campus in Perm Will Determine Root Cause of Stroke in Patients
Specialists at HSE Campus in Perm and clinicians at Perm City Clinical Hospital No. 4, have been collaborating to develop a neural network capable of determining the root cause of a stroke. This marks the world's first attempt to create such a system, the developers note.
HSE Researchers Teach Neural Networks to Better Detect Humour
A group of scientists from the HSE Faculty of Computer Science has conducted a study on the ability of neural networks to detect humour. It turns out that for more reliable recognition, it’s necessary to change the approach to creating datasets on which neural networks are trained. The scientists presented these results at one of the world's most important conferences on natural language processing — EMNLP 2023.
Neural Networks of Power: AI Unravels Knots and Tangles in Relationships between Humans, Elves and Hobbits
One of the most popular writers of the last century, John Ronald Reuel Tolkien, was born on January 3rd. Researchers from HSE University, AIRI and MISSIS have used machine learning to explore the social connections between the characters of his Middle-earth universe. The algorithm managed to create an accurate picture of the social structures and dynamics of the characters' relationships, providing a unique map of interactions in the epic world. The results of the work were published in IEEE Xplore.
‘In the Future, I Expect Rapid Development of Professions Related to Prompt Engineering’
The English-language programme of HSE Online ‘Master of Computer Vision’ will change its name to ‘Artificial Intelligence and Computer Vision’ in 2024. Andrey Savchenko, the programme academic supervisor, shares how the new name will affect the programme semantics, why AI has become the main federal trend in the field of information technology, and what tasks graduates will solve.
Artificial Intelligence as a Driver of Digital Transformation
In December, the HSE Institute for Statistical Studies and Economics of Knowledge and the HSE AI Research Centre participated in UNCTAD eWeek to discuss the future of the emerging digital economy. One of the topics discussed during the conference was artificial intelligence and its applications in driving the digital transformation of industry sectors. The session was co-organised by HSE University.
HSE University Receives Highest Grant under Priority 2030 Programme
HSE University has proved its leading position in the first group of the ‘Research Leadership’ field under the Priority 2030 programme. The university has also received the highest grant for teaching digital competencies to students, demonstrating its educational leadership in the fields of digital technologies and AI.
‘The Future Lies with AI Technologies and HSE University Understands That’
At the AI Journey 2023 international conference in Moscow, a ranking of Russian universities that train the best AI specialists was published. HSE University entered the A+ leadership group, taking first place according to such criteria as ‘Demand for hiring graduates’, ‘Quality of educational environment’, and ‘Activities for the development of school education’. Ivan Arzhantsev, Dean of HSE University’s Faculty of Computer Science, spoke to the HSE News Service about how AI specialists are trained at HSE University and what plans the university has in this area.
‘Every Article on NeurIPS Is Considered a Significant Result’
Staff members of the HSE Faculty of Computer Science will present 12 of their works at the 37th Conference and Workshop on Neural Information Processing Systems (NeurIPS), one of the most significant events in the field of artificial intelligence and machine learning. This year it will be held on December 10–16 in New Orleans (USA).
HSE University Students Win in the AIJ Science Competition at AI Journey 2023
The International Sber Conference of Artificial Intelligence, ‘AI Journey 2023’ recently took place in Moscow. Alexander Rogachev, doctoral student of the HSE Faculty of Computer Science, and Egor Egorov, an HSE 4th-year undergraduate student became the winners of the AIJ Science competition for scientific articles on artificial intelligence that was held as part of the event. The research was carried out under the umbrella of the HSE's Laboratory of Methods for Big Data Analysis (LAMBDA).
HSE University Hosts Fall into ML 2023 Conference on Machine Learning
Over three days, more than 300 conference participants attended workshops, seminars, sections and a poster session. During panel discussions, experts deliberated on the regulation of artificial intelligence (AI) technologies and considered collaborative initiatives between academic institutions and industry to advance AI development through megaprojects.