Suraj Subramanian
bbc55b4069
Merge remote-tracking branch 'upstream/main' into chester-benchmarks
|
%!s(int64=2) %!d(string=hai) anos | |
|---|---|---|
| .. | ||
| inference | %!s(int64=2) %!d(string=hai) anos | |
| llm_eval_harness | %!s(int64=2) %!d(string=hai) anos | |
| README.md | %!s(int64=2) %!d(string=hai) anos | |
README.md
Benchmarks
- inference - a folder contains benchmark scripts that apply a throughput analysis for Llama models inference on various backends including on-prem, cloud and on-device.
- llm_eval_harness - a folder contains a tool to evaluate fine-tuned Llama models including quantized models focusing on quality.