Meet LLMRouter: An Intelligent Routing System designed to Optimize LLM Inference by Dynamically Selecting the most Suitable Model for Each Query

This seems to be an interesting approach! Probably it can be combined/integrated with mixture of experts (MoE) approaches.

"LLMRouter is an open source routing library from the U Lab at the University of Illinois Urbana Champaign that treats model selection as a first class system problem. It sits between applications and a pool of LLMs and chooses a model for each query based on task complexity, quality targets, and cost, all exposed through a unified Python API and CLI. The project ships with more than 16 routing models, a data generation pipeline over 11 benchmarks, and a plugin system for custom routers. ..."

"... To achieve intelligent routing, it defines:

• Smart Routing: Automatically routes queries to the optimal LLM based on task complexity, cost, and performance requirements.
• Multiple Router Models: Support for over 16 routing models, including KNN, SVM, MLP, Matrix Factorization, Elo Rating, Graph-based routers, BERT-based routers, Hybrid probabilistic routers, transformed-score routers, multi-round routers, and many additional advanced strategies.

..."

Meet LLMRouter: An Intelligent Routing System designed to Optimize LLM Inference by Dynamically Selecting the most Suitable Model for Each Query - MarkTechPost

LLMRouter: An Open-Source Library for LLM Routing