Any-LLM: Open Source Tool for Unified Interface Invocation of Multilingual Models

Any-LLM is an open source Python library developed by the Mozilla AI team, designed to invoke different Large Language Model (LLM) providers, such as OpenAI, Mistral, and Anthropic, through a single interface. it simplifies the complexity of switching between models for developers by eliminating the need to set up additional proxies or gateway servers. any-LLM uses the official SDK to ensure compatibility and maintenance reliability, while remaining framework-agnostic for a wide range of project scenarios. LLM uses the official SDK to ensure compatibility and maintenance reliability, while remaining framework-agnostic and suitable for a wide range of project scenarios. Developers only need to configure API keys or model parameters to quickly invoke different models, generate text or conduct conversations. The project is actively maintained and widely used in Mozilla's product any-agent for developers who need to flexibly test and integrate language models.

Function List

• Unified Interface: Call multiple LLM providers through a single API, supporting OpenAI, Mistral, Anthropic, etc.
• Official SDK support: Prioritize the use of the provider's official SDK to reduce the maintenance burden and ensure compatibility.
• No proxy dependency: No need to set up a proxy or gateway server to communicate directly with the LLM provider.
• Framework-independent: Compatible with any development framework, suitable for different project requirements.
• OpenAI format compatibility: Response formats follow OpenAI API standards for easy integration and migration.
• Flexible Configuration: Supports setting API keys directly via environment variables or parameters to simplify operation.
• Model Switching: Easily switch between models from different providers, suitable for testing and comparing model performance.

Using Help

Installation process

To use Any-LLM, you first need to install Python (recommended version 3.11 or higher). Here are the detailed installation steps:

1. Installing the Any-LLM Library
   Run the following command in a terminal to install Any-LLM and its dependencies:

   pip install any-llm
   

   If support is required for a specific provider (e.g. Mistral or Anthropic), you can install the corresponding module:

   pip install any-llm[mistral,anthropic]
   

   Or install all supported providers:

   pip install any-llm[all]
   

2. Configuring API Keys
   Any-LLM requires the provider's API key. It can be configured in either of the following two ways:
   • environment variable: Store the key in an environment variable. for example:

     export MISTRAL_API_KEY='your_mistral_api_key'
     export OPENAI_API_KEY='your_openai_api_key'
     

   • Setting in the code: pass directly on the call api_key parameter (not recommended, less secure).
     Make sure the key is valid, otherwise the call will fail.
3. Verify Installation
   Once the installation is complete, you can run the following command to check for success:

   python -c "import any_llm; print(any_llm.__version__)"
   

   If the version number is output, the installation was successful.

Main Functions

The core functionality of Any-LLM is to call models from different LLM providers through a unified interface to generate text or conduct conversations. Here is how it works:

1. Basic text generation

Any-LLM offers completion function for generating text. The following is an example of a call to a Mistral model:

from any_llm import completion
import os
# 确保已设置环境变量
assert os.environ.get('MISTRAL_API_KEY')
# 调用 Mistral 模型
response = completion(
model="mistral/mistral-small-latest",
messages=[{"role": "user", "content": "你好！请介绍一下 Python 的优势。"}]
)
print(response.choices[0].message.content)

• Parameter description::
  • model: the format is <provider_id>/<model_id>e.g. mistral/mistral-small-latestThe
  • messages: A list of dialog contents containing role(Roles such as user maybe assistant(math.) and content(message content).
• exports: The text returned by the model is stored in the response.choices[0].message.content Center.

2. Switching model providers

Any-LLM supports easy switching of models in code. For example, switching to OpenAI's model:

response = completion(
model="openai/gpt-3.5-turbo",
messages=[{"role": "user", "content": "什么是机器学习？"}],
api_key="your_openai_api_key"  # 可选，直接传递密钥
)
print(response.choices[0].message.content)

Simply change the model parameter without modifying other code structures.

3. Configuring advanced parameters

Any-LLM allows setting the temperature (temperature), max. token Number (max_tokens) and other parameters to control the style and length of the generated text. For example:

response = completion(
model="anthropic/claude-3-sonnet",
messages=[{"role": "user", "content": "写一首短诗"}],
temperature=0.7,
max_tokens=100
)
print(response.choices[0].message.content)

• temperature: Controls the randomness of the generated text, the lower the value the more deterministic it is (default 1.0).
• max_tokens: Limit the output length to avoid overly long responses.

4. Error handling

Any-LLM throws an exception if the API key is invalid or the model is not available. It is recommended to use the try-except Catch errors:

try:
response = completion(
model="mistral/mistral-small-latest",
messages=[{"role": "user", "content": "你好！"}]
)
print(response.choices[0].message.content)
except Exception as e:
print(f"错误：{e}")

Featured Function Operation

1. Model comparison and testing

The biggest advantage of Any-LLM is that it supports fast model switching, which is suitable for developers to compare the performance of different models. For example, testing the difference in responses between Mistral and OpenAI:

models = ["mistral/mistral-small-latest", "openai/gpt-3.5-turbo"]
question = "解释量子计算的基本原理"
for model in models:
response = completion(
model=model,
messages=[{"role": "user", "content": question}]
)
print(f"{model} 的回答：{response.choices[0].message.content}")

This helps the developer to choose the most suitable model for a particular task.

2. Integration into existing projects

Any-LLM's framework-agnostic nature makes it easy to integrate into web applications, command line tools, or data analysis scripts. For example, integration in a Flask application:

from flask import Flask, request
from any_llm import completion
app = Flask(__name__)
@app.route('/chat', methods=['POST'])
def chat():
data = request.json
response = completion(
model=data.get('model', 'mistral/mistral-small-latest'),
messages=[{"role": "user", "content": data['message']}]
)
return {"response": response.choices[0].message.content}
if __name__ == '__main__':
app.run()

This code creates a simple chat API that takes user input and returns a model-generated response.

caveat

• API Key Security: Avoid hard-coding keys in code and prioritize the use of environment variables.
• network connection: Any-LLM requires a network connection to call the cloud model and ensure network stability.
• Model Support: The models and parameters supported by different providers may vary, so please refer to the official documentation.
• performance optimization: For high-frequency calls, it is recommended to batch process requests to reduce API overhead.

application scenario

1. Rapid Prototyping
   Developers can use Any-LLM to quickly test the performance of different LLMs on specific tasks such as text generation, Q&A or translation, shortening the development cycle.
2. Model Performance Comparison
   Data scientists or AI researchers can use Any-LLM to compare the output quality of multiple models on the same task and select the optimal model.
3. Education and learning
   With Any-LLM, students or beginners can experience the capabilities of different LLMs and learn how the models work and how the API calls are made.
4. Enterprise Application Integration
   Enterprises can integrate Any-LLMCHF

M models are integrated into business systems to quickly build AI-driven functionality such as intelligent customer service or content generation tools.

QA

1. What language models does Any-LLM support?
   Supports models from major providers such as OpenAI, Mistral, Anthropic, etc. For specific models, please refer to the provider's documentation.
2. Is there any additional server setup required?
   No, Any-LLM is called directly from the official SDK, no proxies or gateway servers are needed.
3. How are API keys handled?
   It is recommended to set the key via an environment variable, or you can set the key in the completion Passed directly in the function (not recommended).
4. Does Any-LLM support local models?
   The current version mainly supports cloud models, which need to be called from the Internet, while local model support needs to refer to other tools such as llamafile.
5. How to debug call failures?
   Check that the API key, network connection, and model name are correct using the try-except Capture error messages.