PromptForge: open source tool to support prompt word design and testing

PromptForge is an open source prompt word engineering workbench designed to help users design, test and optimize prompt words for AI models. It provides powerful analysis tools and supports multiple AI providers such as Anthropic, OpenAI, Azure OpenAI, and native Ollama. users can quickly create efficient prompts with simple configuration and operation, suitable for developers and researchers. The project is active on GitHub with 516 stars and 56 forks with high community support. Its core goal is to improve the efficiency and quality of cue word creation and help AI application development.

Function List

• Support for multiple AI models: Compatible with AI providers such as Anthropic, OpenAI, Azure OpenAI and Ollama.
• Intelligent Cue Word Suggestions: Provides context-based suggestions for cue word improvements.
• Cue word testing and evaluation: Support systematic testing of cue word effect and provide detailed analysis.
• The environment is easy to configure: via the .env file to quickly set up API keys and runtime environments.
• Community-driven development: open source code, accepting community contributions and feedback on issues.
• Local runtime support: support for local deployment, suitable for development and testing scenarios.

Using Help

Installation process

PromptForge is easy to install for users familiar with Git and the command line. Here are the detailed steps:

1. clone warehouse
   Open a terminal and run the following command to clone the project locally:

   git clone https://github.com/insaaniManav/prompt-forge.git
   

   Once the cloning is complete, go to the project directory:

   cd prompt-forge
   

2. Configuration environment
   The project requires the user to provide an API key to connect to the AI model. Copy the example environment file:

   cp .env.example .env
   

   Use a text editor (e.g. nano maybe vim) Open .env file, add the required API key. Example:

   ANTHROPIC_API_KEY="sk-ant-api03-..."
   OPENAI_API_KEY="sk-..."
   AZURE_OPENAI_API_KEY="your-key"
   AZURE_OPENAI_BASE_URL="https://your-resource.openai.azure.com"
   OLLAMA_BASE_URL="http://localhost:11434"
   DEFAULT_AI_PROVIDER="ollama"
   

   Fill in the corresponding keys as required. If you are using a local Ollama model, make sure that the Ollama service is in the http://localhost:11434 Running.

3. Running Projects
   go into api directory and start the service:

   cd api && go run main.go
   

   Once the project is launched, users can access PromptForge's functionality through a local interface or API.

Main Functions

The core of PromptForge is the creation, testing and optimization of prompt words. Below is a detailed walkthrough of the main features:

1. Creating cues

• procedure::
  1. Open the PromptForge interface (usually local) http://localhost:8080(The specific port is confirmed by the runtime log).
  2. In the Prompt Word Input box, enter the initial prompt word, e.g., "Generate a 500-word science and technology article on the future development of artificial intelligence."
  3. Select the AI model (e.g. Anthropic or OpenAI).
  4. Click on the "Generate" button to view the output of the model.
• Featured Functions::
  PromptForge offers "Smart Suggestions". After generating the results, the system analyzes the clarity and effectiveness of the prompts and recommends improvements. For example, the system may suggest changing "Generate a scientific article" to "Generate a clearly structured scientific article with case studies" to improve the quality of the output.

2. Test and evaluation prompts

• procedure::
  1. In the "Test" module, enter multiple prompt word variants. For example:
     • Prompt Word 1: "Write an essay about AI."
     • Prompt #2: "Write a 500-word AI article that includes real-world examples."
  2. Select test parameters such as model type, output length, or number of generations.
  3. Click "Run Test", the system will batch generate the results and show the comparison analysis.
  4. Check out the analytics report for scores for each cue word (based on clarity, relevance, and output quality).
• Featured Functions::
  The system provides visual analysis tools, such as bar charts or tables, to show the differences in the effectiveness of different cues. Users can optimize the cue words based on the report.

3. Optimization of cues

• procedure::
  1. In the Optimize module, upload the tested cues.
  2. Based on the results of the test, the system will suggest adjustments to wording, add details, or simplify language.
  3. Users can directly edit the cue word and retest.
• Featured Functions::
  PromptForge supports iterative optimization, recording the historical version of each tweak, making it easy for users to go back and compare.

4. Local operation of the Ollama model

• procedure::
  1. Make sure Ollama is installed and running locally (refer to the official Ollama documentation).
  2. exist .env file to configure the OLLAMA_BASE_URL="http://localhost:11434"The
  3. Select Ollama as the default AI provider (DEFAULT_AI_PROVIDER="ollama").
  4. When testing cue words, the system calls the local model to generate the results.
• Featured Functions::
  Runs locally without an internet connection, suitable for development environments or privacy-sensitive scenarios.

caveat

• API Key Security: Don't put .env The file is uploaded to a public repository to avoid disclosing the key.
• Dependent environment: Ensure that the Go language environment (version 1.16 or higher) and Git are installed.
• Community Support: If you run into problems, you can file an Issue in the GitHub repository or check out existing Pull Requests.

application scenario

1. AI application development
   Developers use PromptForge to design and test prompts to optimize the output of a conversation system or content generation application. For example, when developing a chatbot, you can test different prompts to ensure more natural responses.
2. Research and experimentation
   The researchers used PromptForge to evaluate the performance of AI models under different cues and to explore the impact of cue design on model output. For example, they tested the effect of "short description" versus "detailed instructions" on the quality of generation.
3. Education and training
   Teachers or trainers use PromptForge to create instructional prompts to generate customized learning materials. For example, design prompts to generate student-specific practice problems or case studies.

QA

1. What AI models does PromptForge support?
   Support for Anthropic, OpenAI, Azure OpenAI, and native Ollama models is available to users via the .env File Configuration.
2. How to fix runtime API key errors?
   probe .env Make sure the key in the file is correct and that API access is activated for the account corresponding to the AI provider.
3. How much computing resources are needed to run locally?
   Running PromptForge is inherently low resource, but local Ollama models may require GPU support and at least 8GB of video memory is recommended.
4. How do I contribute code?
   Visit the GitHub repository, Fork the project, and submit a Pull Request; it is recommended that you check out the existing Issues first to get a sense of what the community wants.