Startseite Erkunden Hilfe
Anmelden
radu
/
LLamaRecipes
Mirror von https://github.com/facebookresearch/llama-recipes.git
Beobachten 1
Favorit hinzufügen 0
Fork 0
Dateien Issues 0 Wiki
Branch: fix_eval
Branches Tags
LLamaRecipes
/
end-to-end-use-cases
/
benchmarks
Kai Wu 00691affbf add mmlu_instruct for 3.2 vor 2 Monaten
..
inference bd210b105d Fix A LOT of links vor 3 Monaten
llm_eval_harness 00691affbf add mmlu_instruct for 3.2 vor 2 Monaten
README.md ae010af7d8 move and add Difflog vor 3 Monaten

README.md

Benchmarks

  • inference - a folder contains benchmark scripts that apply a throughput analysis for Llama models inference on various backends including on-prem, cloud and on-device.
  • llm_eval_harness - a folder that introduces lm-evaluation-harness, a tool to evaluate Llama models including quantized models focusing on quality. We also included a recipe that calculates Llama 3.1 evaluation metrics Using lm-evaluation-harness and instructions that calculate HuggingFace Open LLM Leaderboard v2 metrics.