The following steps are required to integrate a local large language model using Sim:
- First pull the desired model through the script:
./apps/sim/scripts/ollama_docker.sh pull <model_name> - Select the boot method according to the hardware environment:
- GPU environment:
docker compose --profile local-gpu -f docker-compose.ollama.yml up -d - CPU environment:
docker compose --profile local-cpu -f docker-compose.ollama.yml up -d
- GPU environment:
- Selecting pulled local models in the workflow configuration
- Specify GPU or CPU mode as required
- Test model response to ensure workflow is working properly
Note that local models require larger storage space and computational resources, especially since GPU environments provide better performance.
This answer comes from the articleSim: Open Source Tools for Rapidly Building and Deploying AI Agent WorkflowsThe
May not be reproduced without permission:AI productivity tools " How can I use a local large language model in Sim?
English 
简体中文 
日本語 
Deutsch 
Português do Brasil