← Back to publications
2025 · Conference paper · Published

RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs

Zhongzhan Huang, Guoming Ling, Yupei Lin, Yandong Chen, Shanshan Zhong, Hefeng Wu, Liang Lin
Findings of the Association for Computational Linguistics: EMNLP 2025
Figure 1 — RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs
Figure 1: The Overview of Routing LLMs. For each given input, the router distributes it to the appropriate LLM to achieve specific objectives, such as high accu- racy, low computational cost, reduced hallucinations, etc. We find that Routing LLMs is a promising paradigm for LLMs to achieve model-level scaling up.
Overview

Introduces a large-scale benchmark for LLM routing, with more than 200 million performance records across 12 evaluations and over 8,500 language models.