Controlling HuggingFace Upload Results Guide¶
This guide explains how to upload evaluation results to HuggingFace repositories and manage your result storage.
Overview¶
The upload of evaluation results obtained with eval-framework is completely optional and has to be enabled explicitly. When enabled, results are uploaded to a HuggingFace dataset repository for sharing and collaboration.
Upload Methods¶
Method 1: CLI Upload¶
Add the --hf-upload-repo and --hf-upload-dir arguments to your CLI command:
uv run eval_framework \
--llm-args model_name="gpt-4o-mini" \
--task-name ARC \
--num-fewshot 3 \
--num-samples 100 \
--output-dir "./eval_results" \
--hf-upload-repo "my_arbitrary_repo" # Hugging Face repository
--hf-upload-dir "my_arbitrary_name"
Repository Structure¶
Default Repository¶
By default, results are uploaded to your specified HuggingFace dataset repository.
Custom Repository¶
You can specify a different repository with:
--hf-upload-repo "username/repository-name"
Directory Structure¶
When you specify --hf-upload-dir "my_arbitrary_name", the framework creates:
https://huggingface.co/datasets/username/evaluation-results/
└── my_arbitrary_name/
└── Llama31_8B_Instruct_HF/
└── v0.1.0_ARC/
└── fewshot_3__samples_100_91ca2/
├── aggregated_results.json
├── metadata.json
├── output.jsonl
└── results.jsonl
This structure allows different users to maintain their own storage space within the shared repository.
Authentication¶
HuggingFace Token¶
Ensure you have a valid HuggingFace token with write access:
Get token: Visit HuggingFace tokens page
Set environment variable:
export HUGGINGFACE_HUB_TOKEN="your_token_here"
Or add to .env file:
HUGGINGFACE_HUB_TOKEN=your_token_here
Or use HuggingFace CLI login:
huggingface-cli loginThis will prompt you to enter your token interactively and store it securely.