Startseite Erkunden Hilfe
Anmelden
radu
/
LLamaRecipes
Mirror von https://github.com/facebookresearch/llama-recipes.git
Beobachten 1
Favorit hinzufügen 0
Fork 0
Dateien Issues 0 Wiki
Struktur: 63610f049e
Branches Tags
LLamaRecipes
/
end-to-end-use-cases
/
benchmarks
Igor Kasianenko 4ecdbf9639 typo fixes vor 5 Monaten
..
evals_synthetic_data 4ecdbf9639 typo fixes vor 5 Monaten
inference b6ca01882b revert working link vor 8 Monaten
llm_eval_harness 00691affbf add mmlu_instruct for 3.2 vor 8 Monaten
README.md ae010af7d8 move and add Difflog vor 9 Monaten

README.md

Benchmarks

  • inference - a folder contains benchmark scripts that apply a throughput analysis for Llama models inference on various backends including on-prem, cloud and on-device.
  • llm_eval_harness - a folder that introduces lm-evaluation-harness, a tool to evaluate Llama models including quantized models focusing on quality. We also included a recipe that calculates Llama 3.1 evaluation metrics Using lm-evaluation-harness and instructions that calculate HuggingFace Open LLM Leaderboard v2 metrics.