preliminary
Rankify provides 40 pre-retrieved datasets that can be downloaded and used from the Hugging Face platform.
Detailed steps for use
1. Module for importing data sets
from rankify.dataset.dataset import Dataset
2. Checking available data sets
View all preset datasets:
Dataset.available_dataset()
3. Downloading data sets
Take the BM25 retriever and the nq-dev dataset as an example:
dataset = Dataset(retriever="bm25", dataset_name="nq-dev", n_docs=100) documents = dataset.download(force_download=False)
4. Loading local data sets
If there is already a downloaded dataset file:
documents = Dataset.load_dataset('./bm25_nq_dev.json', 100)
This answer comes from the articleRankify: a Python toolkit supporting information retrieval and reorderingThe
May not be reproduced without permission:AI productivity tools " How do I use pre-searched datasets in Rankify?
