SE-Agent: a framework for self-optimizing AI intelligences

SE-Agent is a self-evolutionary framework designed for Large Language Model (LLM) intelligences. It allows different reasoning paths to exchange information through a trajectory-level evolutionary mechanism, which breaks the limitation of a single thinking path. The SE-Agent is able to extend the search range, avoid falling into local optimal solutions, and show new capabilities in group interactions. This framework achieves the best performance to date on the SWE-bench (a software engineering benchmark), truly enabling autonomous evolution of large model intelligences in complex reasoning tasks. It enables AI intelligences to solve complex programming problems more efficiently by analyzing, combining and optimizing different solutions.

Function List

• Self-evolutionary framework: The core mechanism achieves information exchange between different reasoning paths through evolution at the trajectory level, breaking the cognitive limitations of a single path.
• Three core evolutionary operations:
  • Revision: Intelligentsia analyze failed solutions through deep self-reflection and generate entirely new and structurally different solutions.
  • Recombination: Intelligently blends the best of many different solutions to create new and better ones.
  • Refinement:: Draw on the experience of all solutions to optimize those with potential, eliminate redundant steps and provide guidance on risk avoidance based on historical failures.
• Leading performance: In industry-standard SWE-bench tests, using both open- and closed-source large-language models, SE-Agent demonstrated top performance, solving the 80% validated problem.
• Flexible scalability: Support users to create customized evolution strategies (Operator) to suit different needs and scenarios.
• Efficient track management: Includes a trajectory system for compactly storing run records (which reduces the size of 80%), accumulating knowledge across iterations, and utilizing large models for trajectory analysis and summarization.
• Batch processing capability: Support batch processing and testing of multiple instances in SWE-bench.

Using Help

The following section describes in detail how to install and use the SE-Agent framework.

Installation and Configuration

Option 1: Use Pip (recommended)

This is the easiest and most straightforward way to install.

1. First, clone the code repository from GitHub to your local computer:

   git clone https://github.com/JARVIS-Xs/SE-Agent.git
   ```2.  进入项目目录：
   ```shell
   cd SE-Agent
   

2. Use pip for editable mode installations, which will allow you to make changes to the code effective immediately:

   pip install -e .
   

Option 2: Use Conda Virtual Environment

If you want to create an isolated environment for your project and avoid conflicts with other Python projects, Conda is a good choice.

1. Clone the code repository:

   git clone https://github.com/JARVIS-Xs/SE-Agent.git
   

2. Go to the project catalog:

   cd SE-Agent
   

3. Use Conda to create a file namedSEof your new environment and specify Python version 3.12:

   conda create -n SE python=3.12
   

4. Activate this newly created environment:

   conda activate SE
   

5. Finally, install SE-Agent in this environment:

   pip install -e .
   

Verify Installation

After the installation is complete, you can run the following command to check if the installation was successful. If you see a help message or the test script runs normally, the installation is complete.

sweagent --help
python SE/test/run_operator_tests.py

Configuring the API Key

SE-Agent needs to be connected to the Big Language Modeling Service to work. You need at least one API key.

1. In the root directory of the project (SE-Agent/), create a file named.envof the document.
2. Depending on the modeling service you have or wish to use, write the corresponding key information to a file.

For example, if you use DeepSeek's API:

echo "DEEPSEEK_API_KEY=你的DeepSeek密钥" > .env

Or, if you use the OpenAI API:

echo "OPENAI_API_KEY=你的OpenAI密钥" > .env

Or, use Anthropic's API:

echo "ANTHROPIC_API_KEY=你的Anthropic密钥" > .env

The program automatically loads.envkey information in the file.

Basic operation process

1. Run the demo (no API consumption)

This is a demo mode that doesn't call any of the online big model APIs and gives you a quick overview of how the program runs.

python SE/basic_run.py --mode demo

2. Run your first evolutionary experiment

This command will initiate a complete self-evolutionary process where the intelligence will start solving a predefined problem. This process will call the Big Model API that you configured.

python SE/basic_run.py --mode execute

Upon execution, you will see an output similar to the one below, indicating that the intelligence has been successfully initialized and has begun 3 rounds of self-evolution:

✅ SE-Agent initialized successfully
🔄 Starting self-evolution with 3 iterations

3. Batch processing

If you need to test the performance of the SE-Agent on a large-scale dataset (e.g. SWE-bench), you can use therun-batchcommand. This command allows you to specify profiles, models, subsets of datasets, and processing ranges.

sweagent run-batch \
--config config/default.yaml \
--agent.model.name deepseek/deepseek-chat \
--instances.subset verified \
--instances.slice :10

This example will use thedeepseek/deepseek-chatmodel processingSWE-benchcenterverifiedsubset of the first 10 questions.

Custom Development

The power of SE-Agent is its extensibility. You can develop your own evolutionary operator (Operator) to define entirely new evolutionary strategies.

Here is a simple example showing how to create a custom operator:

1. surname CongSE.operatorsImporting Base ClassesTemplateOperatorand registration functionsregister_operatorThe
2. Creates an object that inherits from theTemplateOperatorNew classes such asMyEvolutionOperatorThe
3. Implemented in a new class_generate_contentmethod, which is your custom evolution logic.
4. Finally, use theregister_operatorfunction registers your new operator with the framework and gives it a unique name.

from SE.operators import TemplateOperator, register_operator
class MyEvolutionOperator(TemplateOperator):
def _generate_content(self, instance_info, problem_description, trajectory_data):
# 在这里实现你的自定义进化策略
return "这是由我的新算子生成的内容"
# 注册后就可以在配置文件中使用了
register_operator("my_operator", MyEvolutionOperator)

application scenario

1. Software Engineering Automation
   SE-Agent can automatically solve problems encountered in software development, such as fixing errors (bugs), optimizing code performance, or adding new features on demand. It tries multiple solutions by simulating the thought process of a human developer, and learns and evolves from them to eventually find the best solution.
2. Complex Problem Solving Research
   In the field of academic research, researchers can utilize SE-Agent as an experimental platform to explore the reasoning and learning capabilities of AI intelligences in solving complex problems. By customizing different evolutionary strategies, it is possible to investigate which method is more effective for specific types of problems.
3. AI Intelligence Body Capability Assessment
   The SE-Agent framework can be run on standardized benchmarks (e.g., SWE-bench) for evaluating and comparing the performance of different large language models on software engineering tasks. This provides an objective performance measure for model developers and users.

QA

1. What is the core idea of SE-Agent?
   The core idea of SE-Agent is "self-evolution". Instead of trying to solve a problem once, as traditional AI does, it generates multiple solutions (called "trajectories"), and then iteratively optimizes them by analyzing failures, integrating successes, and ultimately generating a high-quality solution, just like biological evolution.
2. What are the big language models supported by SE-Agent?
   The SE-Agent is available through thelitellmlibrary to support multiple modeling APIs, including DeepSeek, OpenAI, and Anthropic. All you need to do is add the.envfile to configure the appropriate API key for use.
3. Do I need specialized hardware to run SE-Agent?
   Running SE-Agent itself does not require much local hardware, as it mainly calls the large language model APIs in the cloud for computationally intensive tasks. All you need is a stable network connection and enough API quota.
4. How to contribute code to SE-Agent?
   SE-Agent is an open source project where you can report issues or make suggestions by submitting Issues on GitHub, or contribute code directly by submitting Pull Requests. Before developing, it is recommended to read the development documentation in the project.