Insufficient memory is a common problem when deploying AI models in edge devices such as the RK3588 and can be resolved with the following steps:
1. Optimizing compilation options: Limit the number of concurrent tasks when compiling FastDeploy (e.g.python setup.py build -j 4), reducing peak memory usage.
2. Adding a swap partition: Configure at least 4GB of Swap swap space for the device (specific steps:
a) sudo fallocate -l 4G /swapfile
b) sudo chmod 600 /swapfile
c) Adoptionsudo swapon /swapfilestart using
3. Enabling model quantification: Using FastDeploy's W8A16 or FP8 quantization function (model.enable_quantization()), which reduces the 50%-75% memory footprint.
4. Selective compilation: Enable only the necessary modules (e.g. RKNPU2 through theENABLE_RKNPU2_BACKEND=ON) to avoid resource consumption in irrelevant backends.
This answer comes from the articleFastDeploy: an open source tool for rapid deployment of AI modelsThe
