eval_framework.benchmarks.utils package¶
Submodules¶
eval_framework.benchmarks.utils.ellamind_languages module¶
- class eval_framework.benchmarks.utils.ellamind_languages.EllamindLanguage(language, config)[source]¶
Bases:
objectA language of the EllaMind datasets: the framework language and the dataset config holding it.
- Parameters:
language (Language)
config (str)
-
config:
str¶
eval_framework.benchmarks.utils.global_mmlu_languages module¶
Localized prompt wording for Global-MMLU: one GlobalMMLULocale per language config.
- class eval_framework.benchmarks.utils.global_mmlu_languages.GlobalMMLULocale(*, subject_names, preamble_template, question_label, answer_label)[source]¶
Bases:
objectOne language of Global-MMLU: the subject names, the preamble and the question/answer labels.
- Parameters:
subject_names (dict[str, str])
preamble_template (str)
question_label (str)
answer_label (str)
-
answer_label:
str¶
-
preamble_template:
str¶
-
question_label:
str¶
-
subject_names:
dict[str,str]¶