Efficient Deployment Guide for Android
Running the 14B parametric model on a mobile device requires special attention to the following key points:
- Version Selection Priority::
- Q4_K_M.gguf (best balance)
- IQ3_XS.gguf (Extreme Edition)
- Avoid using the F16 version
- Specific operation process::
- Download the adapted GGUF model file via HuggingFace (<8GB recommended)
- Install termux and configure the Linux environment:
pkg install clang make cmake - Compile the llama.cpp branch that adapts Android:
git clone -b android https://github.com/ggerganov/llama.cpp - utilization
--n-gpu-layers 20Parameters section to enable GPU acceleration
- Performance Optimization Tips::
- set up
--threads 4Match the number of CPU cores of the device - increase
--mlockPreventing Memory Swapping - utilization
--prompt-cacheCache Common Cue Words
- set up
- Official APK Alternative: If manual deployment is difficult, a pre-built APK can be downloaded from HuggingFace, but note that only certain model versions are supported!
This answer comes from the articleTifa-Deepsex-14b-CoT: a large model that specializes in roleplaying and ultra-long fiction generationThe
May not be reproduced without permission:AI productivity tools " What is the best way to deploy Tifa-Deepsex-14b-CoT on Android devices?
