Ethan 059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.		1 vuosi sitten
..
README.md	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
backend_request_func.py	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
benchmark_latency.py	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
benchmark_prefix_caching.py	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
benchmark_serving.py	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
benchmark_throughput.py	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
launch_tgi_server.sh	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten
sonnet.txt	059f0a4ca1 Initial commit for Crusoe recipes, beginning with vLLM tutorial on benchmarking fp8.	1 vuosi sitten

Benchmarking vLLM

Downloading the ShareGPT dataset

You can download the dataset by running:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json