The steps to use a local large language model in Sim Studio are as follows:
- Pull the model via script: run
./apps/sim/scripts/ollama_docker.sh pull <model_name>command to download the required model (e.g. LLaMA). - Start an environment that supports local models: select GPU or CPU mode depending on the hardware configuration and run the appropriate Docker Compose command.
- Select the local model in the workflow configuration and specify the mode of operation (GPU or CPU).
- Test model response to ensure that the workflow is working properly.
Note that the local model requires larger storage space and computational resources, and it is recommended to use a high-performance GPU for a better experience.
This answer comes from the articleSim: Open Source Tools for Rapidly Building and Deploying AI Agent WorkflowsThe
May not be reproduced without permission:AI productivity tools " How can I use a local large language model in Sim Studio?
English 
简体中文 
日本語 
Deutsch 
Português do Brasil