Solving Complex Problems with a Dynamic Multi-Agent LLM Framework

In today’s rapidly evolving world, the ability to solve complex problems in a transparent, traceable, and adaptable way is more crucial than ever. In this blog post, we explore a dynamic multi-agent framework that leverages large language models (LLMs) to collaboratively reason through problems, determine roles, and generate robust solutions. This architecture is especially useful for problems where multiple perspectives and collective decision-making are necessary.

The Motivation Behind Multi-Agent Architectures

Traditional single-agent systems—where one model generates a chain-of-thought—are powerful but often opaque. They lack the modularity to separately reason about different aspects of a problem. Our multi-agent framework addresses this by decomposing the problem-solving process into specialized roles:

• Government Agents (Ministers): These agents take charge of proposing solution strategies. They synthesize insights from their subagents and produce a detailed proposal.

• Departmental Subagents: Acting as the specialists, these agents analyze specific aspects of the problem and provide detailed insights.

• Domain Expert Agents: They offer expert-level reasoning, drawing on historical knowledge and domain-specific techniques.

• Opposition Agents: Their role is to critically evaluate proposals, highlight potential pitfalls, and suggest improvements.

• Speaker/Moderator: This agent oversees the entire process, ensuring that discussions remain coherent and that all perspectives are logged for transparency.

Dynamic Role Determination

One of the most innovative aspects of this framework is its ability to dynamically determine which agent roles are required based on the problem description. When a problem is input, an LLM is first queried with a prompt like:

"Based on the following problem, provide a JSON specification of the agent roles required to solve it. Include keys such as 'government_agents', 'domain_experts', 'opposition_agents', and 'subagents_per_government'."

The LLM returns a JSON object that dictates how many of each agent to instantiate. This makes our system incredibly flexible—it can adapt to a wide variety of problems without hardcoding role numbers.

Example:For the classic counterfeit coin bag puzzle, the LLM might suggest:

{
  "government_agents": 2,
  "domain_experts": 1,
  "opposition_agents": 2,
  "subagents_per_government": 2
}

Alternatively, users can manually specify these values if they wish to override the dynamic decision.

How Does the Framework Solve a Problem?

Let’s walk through the key steps:

1. Problem Input and Role Specification

• User Input: The user describes a complex problem. For example, consider the following puzzle:

  "You have 10 bags full of coins. Each bag contains infinite coins, but one bag has forged coins that weigh 1.1 g each while genuine coins weigh 1 g. Using a digital weighing machine and a strategy of taking a different number of coins from each bag, determine which bag contains the forged coins with minimal readings."

• Dynamic Role Determination:The system either uses the LLM to determine the required agent roles or lets the user manually specify them.

2. Agent Instantiation

Based on the role specification, the framework creates:

• Government Agents with their Departmental Subagents.

• Domain Experts and Opposition Agents.

• A Speaker/Moderator to oversee the session.

3. LLM-Based Proposal Generation

Each agent generates its output by constructing an LLM prompt tailored to its role:

• Government Agents combine context from their subagents and ask the LLM for a detailed, step-by-step proposal.

• Domain Experts provide their own expert-level analysis and propose a solution.

• Opposition Agents generate critiques and alternative proposals.

4. Voting and Synthesis

Rather than a simple yes/no vote, every agent casts a vote by returning its proposed solution along with a rationale. These votes are aggregated, and an LLM prompt is then used to synthesize a final, coherent solution that reflects the majority reasoning.

5. Hierarchical Discussion and Transparency

The framework builds a discussion tree where each agent’s contribution is logged. This hierarchical discussion is stored in a Right To Information (RTI) component for complete traceability. Users can inspect every chain-of-thought, decision, and critique that led to the final solution.

6. Validation

To ensure consistency, the aggregated reasoning is passed to the LLM for validation. This step compares the final solution against expected outcomes (when applicable) and checks if the logic is consistent.

7. Final Answer

After all the reasoning, synthesis, and validation, the final answer is printed, along with a detailed transparency report.

An Example: The Counterfeit Coin Bag Puzzle

Imagine you’re trying to solve the counterfeit coin puzzle described earlier. Here’s how the framework would work:

1. Input:The user inputs the coin bag puzzle. The framework detects keywords like "coin" and "bag" and simulates a digital weighing reading (e.g., 55.4 grams, suggesting that bag 4 is counterfeit).

2. Dynamic Role Determination:The LLM determines that 2 government agents, 1 domain expert, 2 opposition agents, and 2 subagents per government agent are required.

3. Proposal Generation:

   • Government Agents gather insights from their subagents and generate proposals (via LLM) describing the weighing strategy.

   • Domain Experts reinforce the reasoning with domain-specific expertise.

   • Opposition Agents critically analyze the proposals and generate their own solution proposals.

4. Voting:Each agent votes on the final answer, with votes being aggregated by the Voting Module. The LLM then synthesizes these votes into a final solution.

5. Validation and Discussion:The framework validates the reasoning and presents a hierarchical discussion tree that shows every step of the thought process.

6. Final Output:The final answer (e.g., "Bag 4 is the counterfeit bag") is clearly printed along with the most compelling rationale and a full transparency report.

Customizing the Framework

The beauty of this framework lies in its modularity and adaptability:

• Custom Prompts: You can edit the LLM prompts used in any agent’s methods (e.g., in prepare_policy, vote, or discuss) to tailor the behavior to your domain.

• Additional Roles: You can easily add new agent types or modify existing ones to handle specific aspects of your problems.

• Dynamic vs. Manual: Choose whether to use dynamic role determination via LLM or manually specify the roles, giving you flexibility in deployment.

Conclusion

This dynamic multi-agent framework represents a robust and transparent approach to solving complex problems. By leveraging LLMs at every stage—from role determination and proposal generation to synthesis and validation—the system provides a powerful toolkit for collaborative problem-solving.

Whether you’re dealing with puzzles like the counterfeit coin bag problem or tackling multifaceted real-world issues, this framework offers a flexible, modular, and transparent solution-building process. The ability to inject your own prompts and customize the behavior further makes it an excellent foundation for research and practical applications.

Feel free to fork this framework, modify the prompts, and extend the architecture to suit your needs. Happy problem solving!

Note: This blog post is based on a dynamic multi-agent architecture that uses LLMs (such as OpenAI’s GPT models) to generate and validate solutions. The complete source code is available in this repository.