The following installation and configuration steps are required to use Gaze-LLE:
Installation Process:
- Cloning Project Warehouse:
git clone https://github.com/fkryan/gazelle.git - Go to the project directory and create a virtual environment:
conda env create -f environment.yml - Activate the environment and install dependencies:
conda activate gazelle && pip install -e . - Optional installation of xformers acceleration:
pip3 install -U xformers
Model use:
Gaze-LLE offers a wide selection of pre-trained models that can be quickly loaded via PyTorch Hub:
- Base model:
torch.hub.load('fkryan/gazelle', 'gazelle_dinov2_vitb14') - Large-scale modeling:
torch.hub.load('fkryan/gazelle', 'gazelle_dinov2_vitl14')
Predictive Output:
The model generates a spatial heat map with values ranging from [0,1] indicating the probability of being gazed at at each location. Users can learn about the complete multi-person gaze prediction implementation process through the Colab demo notes.
This answer comes from the articleGaze-LLE: A Target Prediction Tool for Character Gaze in VideoThe
May not be reproduced without permission:AI productivity tools " How to install and use Gaze-LLE for gaze prediction?
