Langroid: Easily Navigating Large Language Models with Multi-Intelligent Body Programming

Langroid is a Python framework for researchers and developers designed to simplify the process of building applications driven by Large Language Models (LLMs). Developed by researchers from Carnegie Mellon University and the University of Wisconsin-Madison, its core idea is to harness the power of LLMs through multi-intelligent body programming. In this framework, developers can set up multiple agents, equip them with large language models, vector databases, and tools, and then assign tasks to allow these agents to collaborate on solving complex problems by exchanging messages.Designed to be intuitive, lightweight, and scalable, Langroid is independent of other LLM frameworks (e.g., LangChain) and can work with virtually any type of large-scale LLM. It does not depend on other LLM frameworks (e.g. LangChain) and can be used with almost any type of large-scale language model, providing developers with a flexible and powerful development environment.

Function List

• Multi-Intelligence Collaboration. Supports the creation of multiple intelligences that can collaborate via messaging to accomplish complex tasks together.
• Task organization. pass (a bill or inspection etc)Taskclasses to package intelligences, set explicit instructions and goals for them, and enable task delegation in a hierarchical, recursive manner.
• Extensive LLM support. In addition to supporting OpenAI's family of models, hundreds of other providers' LLMs are supported through agent libraries and local servers (e.g., ollama, LiteLLM), including open source, locally deployed, and commercial models.
• Tools/Function Calls. supports OpenAI's function call functionality and provides its ownToolMessagemechanism that is applicable to any LLM. utilizing Pydantic for definitions simplifies the development process and can handle incorrectly formatted JSON generated by LLMs.
• Retrieval Augmentation Generation (RAG). internally installedDocChatAgentIt can easily interact with document collections, including local files and URLs, and supports document chunking, embedding, vector storage and retrieval.
• Multiple vector database support. Support Qdrant, Chroma, LanceDB, Pinecone, pgvector and many other vector databases for efficient RAG.
• Caching mechanism. Supports caching of LLM API responses using Redis or Momento to reduce the cost and time of repeated calls.
• Observability and logging. Generate detailed logs of multi-intelligence interactions and maintain the source and along of messages for developers to trace and debug.

Using Help

Langroid is a powerful Python framework that allows you to build complex applications by combining different "intelligences". Below, we'll explain in detail how to install and use Langroid to perform some basic and advanced operations.

Installing Langroid

Langroid requires Python 3.11 or higher. It is highly recommended to install it in a virtual environment to avoid dependency conflicts with other projects.

1. Foundation Installation
   The installation of the very core is very simple and requires only one command:

   pip install langroid
   

   This base version contains the core functionality needed to use OpenAI models.

2. Installation of additional features
   Langroid has a modular design and many advanced features require additional dependencies to be installed. You can choose to install them as needed:
   • Document Chat Feature (RAG). If you need to let intelligences read and understand PDFs, Word documents, etc., you need to install thedoc-chatAdd-on packages.

     pip install "langroid[doc-chat]"
     

   • Database Interaction Functions. If you want the smart body to connect and query the database, you need to install thedbAdd-on packages.

     pip install "langroid[db]"
     

   • You can also install multiple add-on packages at the same time:

     pip install "langroid[doc-chat,db]"
     

   • Installation of all features. If you want to install all optional dependencies at once, you can use thealltags, but this can significantly increase installation time and size.

     pip install "langroid[all]"
     

Environment Configuration

Once the installation is complete, you need to configure the necessary environment variables, mainly your Big Language Model API key.

1. establish.envfile
   In your project root directory, copy the repository's.env-templatefile and rename it.envThe

   cp .env-template .env
   

2. Configuring the API Key
   show (a ticket).envfile, fill in your OpenAI API key. This is the most basic requirement to get Langroid up and running.

   OPENAI_API_KEY=your-key-here-without-quotes
   

Core concepts and operations

At the heart of Langroid is theAgent(Intelligentsia) andTask(Tasks). Intelligentsia are performers, and tasks are instructions that direct how intelligentsia should act.

1. Direct interaction with LLM

You can skip the intelligences and talk directly to a configured LLM. This is useful for quickly testing model responses.

import langroid.language_models as lm
# 配置要使用的模型，例如GPT-4o
# 或者使用本地模型 "ollama/mistral"
llm_config = lm.OpenAIGPTConfig(
chat_model=lm.OpenAIChatModel.GPT4o,
)
model = lm.OpenAIGPT(llm_config)
# 直接发送问题并获取回答
response = model.chat("中国的首都是哪里？", max_tokens=20)
print(response.message)

2. Dialogue using intelligences

Intelligent Body (ChatAgent) The ability to maintain dialog history and enable contextualized communication.

import langroid as lr
# 创建一个默认配置的聊天智能体
agent = lr.ChatAgent()
# 智能体能够记住之前的对话
agent.llm_response("中国的首都是哪里？")
response = agent.llm_response("那印度的呢？") # 智能体会理解“那”指代的是首都
print(response.content)

3. Managing intelligences through tasks

Task(Tasks) provide an operational framework for intelligences to define their roles, goals, and manage the cycle of interaction with users.

import langroid as lr
# 1. 创建一个智能体
agent = lr.ChatAgent()
# 2. 为智能体创建一个任务，并赋予系统消息（角色设定）
task = lr.Task(
agent,
name="Bot",
system_message="你是一个乐于助人的助手。"
)
# 3. 运行任务，这将启动一个与用户的交互循环
task.run("你好") # 以用户的 "你好" 开始对话

4. Multi-intelligence working together

This is one of the most powerful features of Langroid. You can create multiple intelligences and organize them to solve a more complex problem.

For example, we create a "teacher" and a "student" intelligence and let them engage in a question-and-answer session.

import langroid as lr
# 通用的大模型配置
llm_config = lr.ChatAgentConfig(
llm=lr.language_models.OpenAIGPTConfig(
chat_model=lr.language_models.OpenAIChatModel.GPT4o,
),
)
# 创建老师智能体和任务
teacher_agent = lr.ChatAgent(llm_config)
teacher_task = lr.Task(
teacher_agent,
name="Teacher",
system_message="""
向你的学生提出简明的数学问题，并给予反馈。
请从一个问题开始。
"""
)
# 创建学生智能体和任务
student_agent = lr.ChatAgent(llm_config)
student_task = lr.Task(
student_agent,
name="Student",
system_message="简明地回答老师的问题。",
single_round=True, # 学生回答一轮后就等待
)
# 将学生任务设置为老师任务的子任务
teacher_task.add_sub_task(student_task)
# 运行主任务，整个对话将自动进行
teacher_task.run()

In this example, theteacher_taskwould speak first (ask the question) and then hand over control to thestudent_task, the student answers and control is handed back to the teacher, and so on.

5. Chatting with Documents (RAG)

utilizationDocChatAgentYou can easily ask questions to a bunch of documents.

import langroid as lr
from langroid.agent.special import DocChatAgent, DocChatAgentConfig
# 配置文档路径，可以是网址或本地文件
config = DocChatAgentConfig(
doc_paths=[
"https://arxiv.org/pdf/2308.08155.pdf", # 一篇关于语言模型的论文
"/path/to/your/notes.txt",
],
)
# 创建DocChatAgent，它会自动处理文档的加载、切分和索引
agent = DocChatAgent(config)
# 将智能体包装在任务中，并开始交互式聊天
task = lr.Task(agent, name="DocAssistant")
task.run("请总结一下这篇论文的核心思想。")

Behind this process, Langroid automates the entire process of document download, text extraction, segmentation into chunks, generating vector embeddings, and storing them in a vector database. When you ask a question, it first retrieves the most relevant document fragment from the database, and then hands it over to LLM along with your question to generate a well-reasoned answer.

application scenario

1. Building automated research assistants
   It is possible to create one intelligence responsible for finding information using a search engine, another responsible for extracting key information from downloaded documents, and a third responsible for consolidating this information into a report.
2. Development of interactive data analysis tools
   utilizationTableChatAgentUsers can query data in CSV files or databases in natural language. For example, one could ask "What was the highest selling product last quarter?". The intelligence will automatically generate and execute the corresponding query code.
3. Create domain-specific Q&A systems
   pass (a bill or inspection etc)DocChatAgentLoad documents in specialized fields (e.g. legal texts, medical manuals, technical documents) and quickly build an intelligent Q&A robot that can accurately answer specialized questions.
4. Modeling complex systems and processes
   In business or social science research, intelligences representing different roles (e.g., consumers, suppliers, regulators) can be set up to model market dynamics or social phenomena through their interactions.
5. Code Generation and Interpretation
   One intelligence can be designed to specialize in generating code based on requirements, while the other is responsible for interpreting the logic and usage of the code, or debugging the code if it goes wrong.

QA

1. What is the difference between Langroid and LangChain?
   Langroid is a standalone framework that does not use LangChain. its design philosophy focuses more on building applications through multi-intelligence collaboration, and its core abstractions areAgentcap (a poem)TaskIn contrast, LangChain focuses on a "chain" structure that links different components (models, hints, tools). In contrast, LangChain focuses on a "chain" structure that connects different components (models, hints, tools), and Langroid believes that multi-intelligence architectures are more flexible and scalable when dealing with complex tasks.
2. Can I use locally deployed open source big models?
   Yes. Langroid makes it easy to connect to open source models running locally such as Llama, Mistral, etc. by integrating with tools such as Ollama, LiteLLM, and others. All you need to do is set thechat_modelJust point to your local modeling service address, for example"ollama/mistral"The
3. What is a Tool? How does it work?
   Tools are features that allow intelligences to execute code, call APIs, etc. beyond the capabilities of the language model itself. In Langroid, you can do this by defining an API that inherits from theToolMessageThe framework uses the Pydantic model to create a tool. When an intelligent body (the LLM) thinks it needs to use a tool, it generates a specially formatted "message", which the framework captures, calls your pre-defined Python function to perform the actual operation, and then returns the result to the intelligent body. This process is very intuitive for developers.
4. How do intelligences decide among themselves who will speak?
   attributableTaskThe scheduling logic of a task is determined. In the hierarchy, the parent task is responsible for coordinating the execution order of its children. By default, tasks are executed sequentially in turn (including interactions with the user). Developers can also set thesingle_round,done_ifand other parameters to precisely control the execution flow and termination conditions of the task.
5. Does Langroid support streaming output?
   Support. Many of the methods in Langroid have asynchronous (async) versions and support for streaming responses, which means you can see the model's responses displayed word by word, as you would in the ChatGPT interface, rather than waiting for it to generate the full response, which is important for improving the user experience.