12-Factor Agents 12. Making Your Intelligence a Stateless Reducer

If you've been waiting for an introduction to humanlayer, then this is it. If you are practicing Element 6 - Start/Pause/Resume via a simple API and Element 7 - Contacting humans via tool calls, then you are ready to integrate this element. Allow the user to start/pause/resume from s...

Instead of building monolithic intelligences that try to do everything, it is better to build small, focused intelligences that can do one thing well. Intelligentsia are just one building block in a larger, largely deterministic system. The key insight here is the limitation of the big language model: the larger and more complex the task, the...

This is a small point, but worth mentioning. One of the benefits of an agent is "self-healing" - for short tasks, a large language model (LLM) may call a failed tool. There is a good chance that a good LLM will be able to read an error message or stack trace and...

If you are in control of your own control flow, you can implement many interesting features. Build custom control structures that fit your particular use case. Specifically, certain types of tool calls might be a reason to jump out of a loop, wait for a human to respond, or wait for another long-running task (e.g., a training pipeline)...

By default, the Large Language Model (LLM) API relies on a fundamentally high-stakes Token choice: do we return plain text content, or do we return structured data? You put a lot of weight on the first Token choice, which in the case of the weather in tokyo...

Intelligences are programs, and we expect to be able to start, query, resume, and stop them in some way. Users, applications, pipelines, and other intelligences should be able to easily start an intelligence with a simple API. When long-running operations need to be performed, intelligences and their orchestration deterministic code...

Even outside of the AI space, many infrastructure systems try to separate the "execution state" from the "business state". For AI applications, this can involve complex abstractions to keep track of information such as the current step, next step, wait state, number of retries, and so on. This separation introduces complexity, and while it may be worthwhile,...

The tool need not be complex. At its core, it's just structured output from your Large Language Model (LLM) for triggering deterministic code. For example, suppose you have two tools CreateIssue and SearchIssues. asking a Large Language Model (LLM) to "use one of the multiple tools" is really asking it to output...