 Justin Lee
|
dc406b4769
setup meta-eval for benchmark, ray error
|
il y a 8 mois |
 Justin Lee
|
21e04c29bf
update mmlu pro
|
il y a 8 mois |
|
Justin Lee
|
e19b9e9e34
added fix split, gitignore and download mmlu script
|
il y a 8 mois |
|
Justin Lee
|
8d3a0479e5
updated env file
|
il y a 8 mois |
|
Justin Lee
|
9ffb292272
added inspect and modified harness
|
il y a 8 mois |
|
Justin Lee
|
eea96618cf
batching and parallelization, ran on baseline and lite
|
il y a 8 mois |
|
Justin Lee
|
4fd5f29414
revert to previous changes
|
il y a 8 mois |
|
Justin Lee
|
a6f448f362
<Replace this line with a title. Use 1 line only, 67 chars or less>
|
il y a 8 mois |
|
Justin Lee
|
becbe77ff3
attempt to fix json output format in eval
|
il y a 8 mois |
|
Justin Lee
|
03f2b8eddd
change gpu parallel size docs
|
il y a 9 mois |
|
Justin Lee
|
0bec41f86a
updated readme
|
il y a 9 mois |
|
Justin Lee
|
2776a35314
harness runcode
|
il y a 9 mois |
|
Justin Lee
|
314b6a874a
added updated llama-mmlu-pro and added human-eva
|
il y a 9 mois |
|
Justin Lee
|
5730a84b8a
beef up readme
|
il y a 9 mois |
|
Justin Lee
|
62b53676fb
update harness notebook
|
il y a 9 mois |
|
Justin Lee
|
1e4c6d22dd
update harness notebook
|
il y a 9 mois |
|
Justin Lee
|
e52e1d1ab4
updated prompt migration to use benchmark and also mipro, added meta implementation
|
il y a 9 mois |
|
Justin Lee
|
4d75fe97b5
update dir
|
il y a 9 mois |
|
Justin Lee
|
90d16cd7de
minor changes in eval, deleted formatter
|
il y a 10 mois |
|
Justin Lee
|
b85811d0b9
change eval dataset, include more robust judging, improved main
|
il y a 10 mois |
|
Justin Lee
|
43a2cbc220
adding eval dataset
|
il y a 10 mois |
|
Justin Lee
|
263b8b569d
placeholder readme
|
il y a 10 mois |
|
Justin Lee
|
096249bf33
add .env settings and configure yml
|
il y a 10 mois |
|
Justin Lee
|
a3e96e4e46
add engine and eval dataset
|
il y a 10 mois |
|
Justin Lee
|
08e41d0d0a
add usage guide and init
|
il y a 10 mois |
|
Justin Lee
|
2570d1642a
added evaluator and formatter and main
|
il y a 10 mois |
 Igor Kasianenko
|
4ad1c0f30c
Update README.md (#843)
|
il y a 9 mois |
 Naveen Reddy Gundlagutta
|
e951b567ba
Update README.md
|
il y a 9 mois |
 Sanyam Bhutani
|
5311cde8ed
Add FAQ (#842)
|
il y a 9 mois |
|
Sanyam Bhutani
|
cdf4a1ab46
Update README.md
|
il y a 9 mois |
