Aidan Do
ab1b1450d7
Add gpqa and math evals for instruct models
|
vor 11 Monaten | |
|---|---|---|
| .. | ||
| inference | vor 1 Jahr | |
| llm_eval_harness | vor 11 Monaten | |
| README.md | vor 1 Jahr | |
README.md
Benchmarks
- inference - a folder contains benchmark scripts that apply a throughput analysis for Llama models inference on various backends including on-prem, cloud and on-device.
- llm_eval_harness - a folder that introduces
lm-evaluation-harness, a tool to evaluate Llama models including quantized models focusing on quality. We also included a recipe that calculates Llama 3.1 evaluation metrics Usinglm-evaluation-harnessand instructions that calculate HuggingFace Open LLM Leaderboard v2 metrics.