Vercel AI Gateway's intelligent fault provisioning is one of the key features that differentiates it from using AI vendor APIs directly. In the traditional model, when a developer calls an AI model's API directly, if that service fails, the application will rely entirely on the developer's error handling mechanism. AI Gateway, on the other hand, has a built-in automatic failover capability that seamlessly switches the request to an alternate model as soon as it detects that the service of the primary model is unavailable, based on a pre-configured policy.
This mechanism works as follows: the gateway monitors the health status of all upstream models in real time and switches traffic as soon as a problem is detected according to a backup priority set by the developer (which can be configured based on criteria such as cost, performance, etc.). For example, when the main model OpenAI's GPT-4 experiences a service outage, the system can automatically switch the request to Anthropic's Claude model. This not only avoids the delay of manually handling emergency failures, but also ensures that end-users barely perceive the service switch, minimizing the impact of unplanned downtime.
This answer comes from the articleVercel AI Gateway: a gateway to manage and optimize AI application requestsThe
English 
简体中文 
日本語 
Deutsch 
Português do Brasil