LLamaRecipes/tools/benchmarks

Benchmarks

• inference - a folder contains benchmark scripts that apply a throughput analysis for Llama models inference on various backends including on-prem, cloud and on-device.
• llm_eval_harness - a folder contains a tool to evaluate fine-tuned Llama models including quantized models focusing on quality.