AIRouter: Intelligent Routing Tool for Calling Multiple Models with Unified API Interface

AIRouter is an intelligent routing tool designed for Large Language Models (LLMs), connecting multiple LLM providers (e.g., OpenAI, Anthropic, Google, etc.) through a unified API interface for efficient task distribution. It supports intelligent load balancing, selecting the optimal model based on response time, cost, and success rate, while providing real-time health monitoring and API key management.AIRouter also supports multimodal inputs (e.g., text+image) and function calls, making it suitable for developers who need to operate across models. The project is based on Python development, supports Docker containerized deployment, and is ready to use out-of-the-box, suitable for rapid integration into production environments.

Function List

• Unified API interface: Access to multiple LLM providers, including OpenRouter, DeepInfra, TogetherAI, etc., through a single interface.
• Intelligent Load Balancing: Dynamically assign tasks to the optimal model based on response time, cost, and success rate.
• Real-time health monitoring: Automatically check API status and block unavailable or high-cost models.
• Efficient API key management: Optimize key usage, improve performance, and automatically avoid invalid keys.
• multimodal support: Handles text and image inputs and function calls to meet diverse task requirements.
• Pareoptimality: Intelligently select the best model that balances performance and cost from multiple models.
• Cost optimization: Reduce the cost of using high-cost models with the health-check blocking feature.
• Containerized Deployment: Supports Docker deployment, simplifying the environment configuration and startup process.

Using Help

Installation and Setup

AIRouter is an open source Python project that supports deployment via Python packages or Docker. Here are the detailed installation and usage steps:

1. Environmental preparation

• system requirements: Install Python 3.7 or above, Linux or macOS is recommended.
• Dependent tools::
  • Install Git: for cloning code repositories.
  • Install Docker (optional): for containerized deployment.
  • Install MySQL: for storing API key usage logs.
• Checking the Python version::

  python --version
  

  Make sure the version is 3.7 or above.

2. Cloning of warehouses

• interviews https://github.com/THESIS-AGENT/AIRouter, copy the repository URL.
• Runs in the terminal:

  git clone https://github.com/THESIS-AGENT/AIRouter.git
  cd AIRouter
  

3. Installation of dependencies

• Way 1: Install as a Python package (recommended)::

  pip install -e .
  

• Way 2: Direct installation of dependencies::

  pip install -r requirements.txt
  

4. Configuration projects

• Configuring the API Key::
  • Copy the sample configuration file:

    cp ew_config/api_keys.example.py ew_config/api_keys_local.py
    

  • compiler ew_config/api_keys_local.py, fill in the real API key of each LLM provider. For example:

    API_KEYS = {
    "openai": "sk-xxxxxxxxxxxxxxxxxxxx",
    "anthropic": "sk-ant-xxxxxxxxxxxxxxxx",
    "google": "AIzaSy-xxxxxxxxxxxxxxxx"
    }
    

• Setting up the database::
  • Create a MySQL database:

    CREATE DATABASE airouter;
    CREATE TABLE api_key_usage (
    request_id VARCHAR(50) PRIMARY KEY,
    api_key VARCHAR(100) NOT NULL,
    model_name VARCHAR(50) NOT NULL,
    source_name VARCHAR(50) NOT NULL,
    prompt_tokens INT,
    completion_tokens INT,
    create_time DATETIME NOT NULL,
    finish_time DATETIME NOT NULL,
    execution_time FLOAT NOT NULL,
    status BOOLEAN NOT NULL
    );
    

  • Copy the environment variable file:

    cp env.example .env
    

  • compiler .env file, fill in the database information, for example:

    DB_HOST=localhost
    DB_USER=root
    DB_PASSWORD=your_password
    DB_NAME=airouter
    DB_PORT=3306
    

5. Activation of services

• Docker deployment (recommended)::
  • Build the Docker image:

    docker build -t airouter:latest .
    

  • Start the service:

    docker-compose up -d
    

  • Check the status of the service:

    docker-compose ps
    

• manual activation::
  • Launching of the Health Screening Service:

    python CheckHealthy.py
    

  • Start the API key management service on a new terminal:

    python -m api_key_manager.main
    

Functional operation flow

1. Basic text generation

• utilization LLM_Wrapper class calls the model to generate the text:

  from LLMwrapper import LLM_Wrapper
  response = LLM_Wrapper.generate(
  model_name="gpt4o_mini",
  prompt="解释量子计算的基本原理"
  )
  print(response)
  

• The system will choose the optimal model (e.g., OpenAI's gpt4o_mini) to execute the task based on the load balancing policy.

2. Multimodal inputs

• Supports image and text input, e.g. to describe the content of a picture:

  import base64
  with open("image.jpg", "rb") as f:
  img_base64 = base64.b64encode(f.read()).decode()
  response = LLM_Wrapper.generate_mm(
  model_name="gpt4o_mini",
  prompt="描述这张图片的内容",
  img_base64=img_base64
  )
  print(response)
  

• Ensure that the model supports multimodality (e.g. gpt4o_mini) and that the image needs to be converted to Base64 format.

3. Function calls

• Configure the tool and call external functions, such as querying the weather:

  tools = [
  {
  "type": "function",
  "function": {
  "name": "get_weather",
  "description": "获取天气信息",
  "parameters": {
  "type": "object",
  "properties": {
  "location": {"type": "string", "description": "城市名称"}
  },
  "required": ["location"]
  }
  }
  }
  ]
  response = LLM_Wrapper.function_calling(
  model_name="gpt4o_mini",
  prompt="北京今天天气如何？",
  tools=tools
  )
  print(response)
  

4. Configure load balancing

• Three load balancing modes are supported:
  • fast_first: Prioritize the most responsive models.
  • cost_first: Preference is given to the model with the lowest cost.
  • balanced: Balancing speed and cost.
• Example:

  response = LLM_Wrapper.generate(
  model_name="gpt4o_mini",
  prompt="你好",
  mode="cost_first"
  )
  

5. Pareto-optimal choices

• Select the optimal result from multiple models:

  response = LLM_Wrapper.generate_fromTHEbest(
  model_list=["gpt4o_mini", "claude35_sonnet", "gemini15_pro"],
  prompt="复杂推理任务"
  )
  

• The system automatically selects the best model based on response time and cost.

6. Health monitoring and logging

• Check the health status of the service:
  • interviews http://localhost:8001/check_healthy Check the status of health screening services.
  • interviews http://localhost:8002/check_healthy Check the status of the API key management service.
• View Log:

  docker-compose logs -f airouter-health-check
  tail -f health_check.log
  

caveat

• Ensure that the API key is valid; a failed key will cause the task to fail.
• Docker deployments need to ensure that the ports (8001, 8002) are not occupied.
• Check the GitHub repository regularly for updates to the latest features and fixes.
• Database password (DB_PASSWORD) is required, and its absence will result in startup failure.

application scenario

1. Multi-model task distribution
   Developers need to use multiple LLMs (e.g., OpenAI, Anthropic) to complete tasks at the same time. AIRouter can distribute tasks through a unified interface and automatically select the optimal model to save development time.
2. Cost-sensitive projects
   With a limited budget, AIRouter's cost optimization feature prioritizes low-cost models for startups or individual developers.
3. Multimodal Application Development
   When developing image description or multimodal chat applications, AIRouter supports both text and image input to simplify the development process.
4. automated operation and maintenance (O&M)
   Ops teams can build stable AI services with health monitoring and load balancing features to reduce manual intervention.

QA

1. Which LLM providers does AIRouter support?
   Support for OpenRouter, DeepInfra, DeerAPI, TogetherAI, Google, OpenAI, Anthropic, and many other providers, with a specific list of support available in the ew_config/source.py View.
2. How to choose the optimal model?
   utilization generate_fromTHEbest method, the system selects the best model based on a Pareto-optimal algorithm that combines response time, cost, and success rate.
3. What is the difference between Docker deployment and manual deployment?
   Docker deployment through containerization to simplify the environment configuration, suitable for production environments; manual deployment needs to install dependencies one by one, suitable for development and debugging.
4. How do I handle API key failure?
   AIRouter's key management system automatically detects invalid keys and switches to an available key. api_keys_local.py Configure multiple valid keys in the