AgentGuard: A Tool for Monitoring AI Agent Costs in Real Time and Preventing Overspending

AgentGuard is an open source tool designed for developers and enterprises to monitor and control the cost of using AI agents, such as large language models, in real time. With simple code integration, it helps users set budget caps to prevent high costs caused by AI agents running out of control.AgentGuard supports multiple protection modes and can automatically stop the process or issue warnings in case of cost overruns. It is designed to be easy to use and can be enabled in a project with just a few lines of code, making it suitable for a variety of AI development scenarios. AgentGuard provides reliable cost management and transparent logging for both standalone development and multi-process environments.

Function List

• Monitor the cost of API calls for AI agents in real-time, showing current spend and budget share.
• Supports setting a budget limit, with options to throw an error, send a notification, or terminate the process when it is exceeded.
• Provides a detailed history log of API calls for easy tracking and cost analysis.
• Supports multi-process budget sharing and cross-process cost tracking through Redis.
• Provides privacy protection options to automatically block sensitive data.
• Support dynamic adjustment of budget ceiling and protection mode to adapt to different mission requirements.
• Integrate webhook functionality to send real-time cost alerts to Slack or Discord.
• Sample code is provided to support fast integration with frameworks such as LangChain.

Using Help

Installation process

AgentGuard is a Node.js module with a simple installation and configuration process suitable for most AI projects. Below are detailed installation and usage steps:

1. Installing Node.js
   Make sure you have Node.js installed on your system (recommended version 18 or higher). You can download and install it from the official Node.js website.
2. Cloning AgentGuard Repository
   Clone AgentGuard's GitHub repository by running the following command in a terminal:

   git clone https://github.com/dipampaul17/AgentGuard.git
   cd AgentGuard
   

3. Installation of dependencies
   After entering the project directory, run the following command to install the necessary dependencies:

   npm install
   

4. Verify Installation
   After the installation is complete, run the following command to check for success:

   node verify-installation.js
   

   If the installation is correct, the terminal outputs a confirmation message.

5. Initializing AgentGuard
   Add the following two lines of code to your AI project to initialize AgentGuard:

   const agentGuard = require('agent-guard');
   await agentGuard.init({ limit: 50 });
   

   Here.limit: 50 Indicates setting a budget limit of $50. You can adjust this value as needed.

Configuration options

AgentGuard offers flexible configuration options. The following are descriptions of commonly used parameters:

• limit:: Set a budget ceiling (in dollars), such as 50 Indicates $50.
• mode: Protected mode, the following three are supported:
  • throw: Throw an error on overrun, for cases where manual handling is required.
  • notify: Sends a notification when it overruns (e.g., via webhook), but does not terminate the process.
  • kill:: Direct termination of the process in case of cost overruns, which lends itself to strict budgetary control.
• webhook: Configure the notification address (e.g. Slack or Discord webhook URL) for receiving alerts. Example:

  webhook: 'https://hooks.slack.com/...'
  

• redis: Configure Redis URLs for multi-process budget sharing. Example:

  redis: 'redis://localhost:6379'
  

• privacy: Set to true Automatically block sensitive data when
• silent: Set to true Hide the real-time cost update log when.

Sample initialization code:

const guard = await agentGuard.init({
limit: 100,
mode: 'throw',
webhook: 'https://hooks.slack.com/...',
redis: 'redis://localhost:6379',
privacy: true
});

Main Functions

1. View Current Costs
   utilization guard.getCost() Get the current total spend. Example:

   console.log(`已花费: $${guard.getCost()}`);
   

2. Adjustment of the budget ceiling
   The budget can be dynamically adjusted if task priorities change:

   guard.setLimit(500); // 将预算上限改为 500 美元
   

3. replacement cost
   The cost count can be reset when a new task starts:

   await guard.reset();
   

4. Viewing the Call Log
   utilization guard.getLogs() Get a detailed history of API calls for easy analysis of cost sources.
5. Addressing overexpenditures
   When the budget is overspent, AgentGuard will take advantage of the mode parameter performs the corresponding operation. For example, if the mode is throwYou can use try-catch to handle this:

   try {
   const response = await openai.chat.completions.create({...});
   } catch (error) {
   if (error.message.includes('AGENTGUARD_LIMIT_EXCEEDED')) {
   console.log('预算超支:', error.agentGuardData);
   // 切换到低成本模型或保存状态
   }
   }
   

Browser Environment Support

AgentGuard also supports browser environments. Just introduce the following script:

<script src="https://unpkg.com/agent-guard@latest/dist/agent-guard.min.js"></script>
<script>
AgentGuard.init({ limit: 50, mode: 'notify' });
</script>

sample code (computing)

Below is a complete example showing how to integrate AgentGuard in an OpenAI project:

const agentGuard = require('agent-guard');
const openai = require('openai');
(async () => {
const guard = await agentGuard.init({
limit: 25,
mode: 'notify',
webhook: 'https://hooks.slack.com/...',
privacy: true
});
try {
const response = await openai.chat.completions.create({
model: 'gpt-4',
messages: [{ role: 'user', content: 'Hello world' }]
});
console.log('响应:', response);
console.log(`当前花费: $${guard.getCost()}`);
} catch (error) {
if (error.message.includes('AGENTGUARD_LIMIT_EXCEEDED')) {
console.log('预算保护触发:', error.agentGuardData);
}
}
})();

Debugging and Testing

AgentGuard provides several examples for developers to test:

• (of a computer) run node examples/runaway-loop-demo.js Simulate infinite loop scenarios to test budget protection.
• (of a computer) run node examples/langchain-example.js Check out the integration with LangChain.
• show (a ticket) examples/test-browser.html Test browser environment support.

application scenario

1. Cost Control in AI Development
   When building AI applications, developers often incur unexpectedly high costs due to too many API calls. agentGuard ensures development costs are manageable with real-time monitoring and budget constraints for individual developers or small teams.
2. Enterprise AI Project Management
   Enterprises deploying large AI projects need to share budgets across teams, and AgentGuard's Redis support allows for multi-process collaboration to manage costs for complex enterprise-class applications.
3. Education and Research Programs
   Students or researchers experimenting with AI models can use AgentGuard to limit the cost of experiments and avoid wasting resources by running off-task.
4. Browser-side AI applications
   Developers running AI agents in the browser can monitor the cost of front-end AI calls with AgentGuard's browser support for interactive web applications.

QA

1. What AI frameworks does AgentGuard support?
   AgentGuard is a framework-agnostic tool that supports any AI project that uses API calls. It provides sample integrations with frameworks such as LangChain and can be extended by developers as needed.
2. How do you handle budget overruns?
   Depending on the configuration of the modeAgentGuard can throw errors (throw), sending notifications (notify) or terminate the process (kill). Developers can catch errors and handle them with try-catch.
3. Do I need Redis?
   Redis is optional and only needed in multi-process or distributed environments for shared budget data. Redis does not need to be configured for standalone projects.
4. How to protect sensitive data?
   commander-in-chief (military) privacy set to trueAgentGuard automatically blocks sensitive information and ensures that logs and notifications do not contain private data.