Multi-Agent Systems with LLM-Based Agents

The main focus of the research is on the method of ensembling LLM agents and enhancing its performance by introducing a Deciding-Agent. This agent analyzes the solutions provided by the primary multi-agent system and selects the most optimal response. The work consists of several key parts: Analysis of Multi-Agent System Methods: This includes reviewing existing sampling and voting-based methods and analyzing their effectiveness on history and mathematics tasks. Proposing a New Approach: Introducing a Deciding-Agent to improve the quality of solutions. This agent consolidates and structures information from multiple agents, leading to a more well-founded and accurate final answer. Application to Context-Based Question Answering Tasks: Experiments demonstrating how multi-agent systems can improve the quality of responses based on the analysis of several contextual passages. Using GPT-4 to Solve a "What? Where? When?" Dataset: Analysis and results of applying multi-agent systems to this task. The research showed that using a Deciding-Agent with a Chain of Thought significantly improves solution quality. The quality increase amounted to 12 percentage points in mathematics tasks and 8 percentage points in history tasks. The method also demonstrated its versatility and effectiveness in various tasks, including context-based question answering. In conclusion, the proposed approach enhances the performance of multi-agent systems, ensuring high accuracy and flexibility in solving a wide range of tasks. Further research may include increasing the number of agents, applying the method to fine-tuned models, and fine-tuning the final agent to improve analysis and decision-making.