Mixture of Agents (MoA)

Mixture of Agents (MoA) is an inference-time technique in which several large language models (LLMs) answer the same prompt and one or more aggregator models read all of those answers and write a single, improved response. Junlin Wang and colleagues at Together AI introduced it in the 2024 paper Mixture-of-Agents Enhances Large Language Model Capabilities. It rests on an effect the authors call collaborativeness: an LLM tends to write a better response when it can see other models' responses to the same question, even when those models are weaker than it is. MoA needs no training or fine-tuning, only API calls and prompts, so it can combine models from different providers.

How Mixture of Agents Works#

MoA arranges models in layers. In the first layer, several proposer models each answer the prompt independently. Each later layer receives the original prompt together with every response from the layer before it, through an aggregate-and-synthesize prompt that tells the model to evaluate those responses critically, recognizing that some may be biased or incorrect, and to write one refined answer. The final layer holds a single aggregator whose output is the system's answer. Models differ in these roles: in the paper's experiments, WizardLM wrote excellent proposals but aggregated poorly, and GPT-4o and Qwen1.5 performed well in both.

The paper's main configuration used three layers of six open-source models, Qwen1.5-110B-Chat, Qwen1.5-72B-Chat, WizardLM-8x22B, Llama-3-70B-Instruct, Mixtral-8x22B, and DBRX-Instruct, with Qwen1.5-110B-Chat as the final aggregator. It reached a 65.1% length-controlled win rate on the AlpacaEval 2.0 instruction-following benchmark, ahead of GPT-4o's 57.5% at the time, and a cheaper two-layer MoA-Lite variant with Qwen1.5-72B-Chat as aggregator scored 59.3%. The authors released a reference implementation of the method.

Mixture of Agents vs. Related Techniques#

• Mixture of Experts (MoE): An architecture inside a single neural network, where a learned router activates a few expert sublayers for each token and all experts are trained together. MoA combines separate, complete models at inference time through prompts.
• Model ensembles: An ensemble averages or votes on its members' predictions. An MoA aggregator writes new text conditioned on all candidates, so it can combine the correct parts of answers that are each partly wrong.
• Model routing: A router sends each query to the one model most likely to answer it well at the lowest cost. MoA queries several models for every prompt.
• AI debate: In multi-agent debate, models read each other's answers and revise their own over several rounds. MoA passes information forward through layers and ends with one aggregator.
• AI agent orchestration: Orchestration coordinates agents that usually handle different subtasks. MoA is one concurrent pattern within it, with every model working on the same prompt.

Real-World Applications#

• Text generation: Open-ended writing, analysis, and instruction following, the tasks AlpacaEval measures, gain the most from synthesizing several drafts.
• Agentic coding: Several models propose solutions to the same programming problem, and an aggregator merges the strongest ideas, with unit tests deciding between conflicting proposals.
• Synthetic data generation: Teams use MoA outputs as high-quality targets for training smaller models, where the extra inference cost is paid once.

The same pattern extends to multimodal AI, where several vision-language models describe the same image and an aggregator reconciles their descriptions. The Ultralytics LLM interface calls OpenAI and OpenAI-compatible providers such as DeepSeek, Kimi, Z.AI, and OpenRouter through the same LLM() class, so one script can send an image to several proposer models and pass their answers to an aggregator.

Strengths and Limitations#

MoA trades compute for quality. The main configuration makes more than a dozen model calls for every prompt, and because the aggregator cannot write its first token until the last layer finishes, the paper notes a high time to first token, which hurts interactive use. Inference latency and cost therefore limit MoA to tasks where answer quality matters more than speed.

Mixing different models also matters less than it first appeared, because adding weaker proposers lowers the average quality of what the aggregator reads. The Self-MoA study by Wenzhe Li and colleagues found that aggregating several sampled outputs from only the single best model beat a standard mixture of different LLMs by 6.6% on AlpacaEval 2.0 and by 3.8% on average across benchmarks including MMLU, CRUX, and MATH.