Solutions for insufficient video memory
The problem of insufficient video memory for ordinary computers can be solved by the following three technical solutions:
- QLoRA quantification program: Train with 4-bit precision to reduce the 7B model video memory requirement to 6GB. modify the settings.json in theload_in_4bitThe parameter is True
- Distributed training programs: Use multiple consumer graphics cards to work together and share video memory with DeepSpeed's ZeRO-2 optimizer. Installation Command:
uv pip install deepspeed - Cloud Service Alternatives: Free GPU resources from AutoDL or Kaggle are recommended, with care taken to select instances equipped with T4 (16GB) and higher graphics cards
For computers with 8GB of video memory, additional steps can be taken to 1) turn down batch_size to 2, 2) enable gradient_checkpointing, and 3) use CPU offloading techniques. Specifically, you need to add in train_sft.py--gradient_checkpointing --offload_param=cpuparameters
This answer comes from the articleWeClone: training digital doppelgangers with WeChat chats and voicesThe
May not be reproduced without permission:AI productivity tools " How to solve the problem of not being able to run WeClone due to insufficient video memory on a regular computer?
English 
简体中文 
日本語 
Deutsch 
Português do Brasil