operational readiness
First you need to download the following necessary files:
- Base model weights
- Visual Transformer Model Weights
- H-LoRA Understanding Weights
- Fusion layer weights
Parameter Configuration
modificationscom_infer.shpath variable in the script:
- MODEL_NAME_OR_PATH: base model path
- VIT_PATH: Vision Transformer Path
- HLORA_PATH: H-LoRA weighting paths
- FUSION_LAYER_PATH: Fusion Layer Path
Implementation modalities
- Executed via script:
bash com_infer.sh - or run the Python command directly:
python3 com_infer.pyand configure all parameters
Parameter description
Problem text and image paths need to be specified, with adjustments to data type (FP16), H-LoRA parameters (r=64, alpha=128), etc.
This answer comes from the articleHealthGPT: A Medical Big Model to Support Medical Image Analysis and Diagnostic Q&AThe
May not be reproduced without permission:AI productivity tools " How to use HealthGPT for medical image quiz? What are the exact steps to do so?
