 Justin Lee
|
e1d64ca2f4
update gitignore, added mmlu 0shot and ran a bunch of test
|
8 months ago |
|
Justin Lee
|
479b1fbbd7
updated mmlu meta-eval for prompt migration
|
8 months ago |
|
Justin Lee
|
caeddccb8d
update utils
|
8 months ago |
|
Justin Lee
|
f8a6c7d79f
running mmlu pro with meta eval - fixed error
|
8 months ago |
 Chester Hu
|
07b191b17e
Merge pull request #2 from pia-papanna/tools-refactory-chester
|
1 year ago |
|
Justin Lee
|
dc406b4769
setup meta-eval for benchmark, ray error
|
8 months ago |
 Justin Lee
|
21e04c29bf
update mmlu pro
|
9 months ago |
|
Justin Lee
|
e19b9e9e34
added fix split, gitignore and download mmlu script
|
9 months ago |
|
Justin Lee
|
8d3a0479e5
updated env file
|
9 months ago |
|
Justin Lee
|
9ffb292272
added inspect and modified harness
|
9 months ago |
|
Justin Lee
|
eea96618cf
batching and parallelization, ran on baseline and lite
|
9 months ago |
|
Justin Lee
|
4fd5f29414
revert to previous changes
|
9 months ago |
|
Justin Lee
|
a6f448f362
<Replace this line with a title. Use 1 line only, 67 chars or less>
|
9 months ago |
|
Justin Lee
|
becbe77ff3
attempt to fix json output format in eval
|
9 months ago |
|
Justin Lee
|
03f2b8eddd
change gpu parallel size docs
|
9 months ago |
|
Justin Lee
|
0bec41f86a
updated readme
|
9 months ago |
|
Justin Lee
|
2776a35314
harness runcode
|
9 months ago |
|
Justin Lee
|
314b6a874a
added updated llama-mmlu-pro and added human-eva
|
9 months ago |
|
Justin Lee
|
5730a84b8a
beef up readme
|
9 months ago |
|
Justin Lee
|
62b53676fb
update harness notebook
|
9 months ago |
|
Justin Lee
|
1e4c6d22dd
update harness notebook
|
9 months ago |
|
Justin Lee
|
e52e1d1ab4
updated prompt migration to use benchmark and also mipro, added meta implementation
|
9 months ago |
|
Justin Lee
|
4d75fe97b5
update dir
|
9 months ago |
|
Justin Lee
|
90d16cd7de
minor changes in eval, deleted formatter
|
10 months ago |
|
Justin Lee
|
b85811d0b9
change eval dataset, include more robust judging, improved main
|
10 months ago |
|
Justin Lee
|
43a2cbc220
adding eval dataset
|
10 months ago |
|
Justin Lee
|
263b8b569d
placeholder readme
|
10 months ago |
|
Justin Lee
|
096249bf33
add .env settings and configure yml
|
10 months ago |
|
Justin Lee
|
a3e96e4e46
add engine and eval dataset
|
10 months ago |
|
Justin Lee
|
08e41d0d0a
add usage guide and init
|
10 months ago |
