|
@@ -1,6 +1,6 @@
|
|
|
# **ML Papers of The Week**
|
|
|
|
|
|
-[Subscribe to our newletter](https://nlpnews.substack.com/) to get a weekly list of top ML papers in your inbox.
|
|
|
+[Subscribe to our newsletter](https://nlpnews.substack.com/) to get a weekly list of top ML papers in your inbox.
|
|
|
|
|
|
At DAIR.AI we ❤️ reading ML papers so we've created this repo to highlight the top ML papers of every week.
|
|
|
|
|
@@ -39,7 +39,7 @@ At DAIR.AI we ❤️ reading ML papers so we've created this repo to highlight t
|
|
|
| 2) **LIMA** - a new 65B parameter LLaMa model fine-tuned on 1000 carefully curated prompts and responses; it doesn't use RLHF, generalizes well to unseen tasks not available in the training data, and generates responses equivalent or preferred to GPT-4 in 43% of cases, and even higher compared to Bard. | [Paper](https://arxiv.org/abs/2305.11206), [Tweet](https://twitter.com/violet_zct/status/1660789120069926912?s=20) |
|
|
|
| 3) **Voyager** - an LLM-powered embodied lifelong learning agent in Minecraft that can continuously explore worlds, acquire skills, and make novel discoveries without human intervention. | [Paper](https://arxiv.org/abs/2305.16291), [Tweet](https://twitter.com/DrJimFan/status/1662115266933972993?s=20) |
|
|
|
| 4) **Gorilla** - a finetuned LLaMA-based model that surpasses GPT-4 on writing API calls. This capability can help identify the right API, boosting the ability of LLMs to interact with external tools to complete specific tasks. | [Paper](https://arxiv.org/abs/2305.15334), [Tweet](https://twitter.com/omarsar0/status/1661540207206846464?s=20) |
|
|
|
-| 5. **The False Promise of Imitatiting Proprietary LLMs** - provides a critical analysis of models that are finetuned on the outputs of a stronger model; argues that model imitation is a false premise and that the higher leverage action to improve open source models is to develop better base models. | [Paper](https://arxiv.org/abs/2305.15717) , [Tweet](https://twitter.com/arankomatsuzaki/status/1661908342829187072?s=20)
|
|
|
+| 5. **The False Promise of Imitating Proprietary LLMs** - provides a critical analysis of models that are finetuned on the outputs of a stronger model; argues that model imitation is a false premise and that the higher leverage action to improve open source models is to develop better base models. | [Paper](https://arxiv.org/abs/2305.15717) , [Tweet](https://twitter.com/arankomatsuzaki/status/1661908342829187072?s=20)
|
|
|
| 6) **Sophia** - presents a simple scalable second-order optimizer that has negligible average per-step time and memory overhead; on language modeling, Sophia achieves 2x speed-up compared to Adam in the number of steps, total compute, and wall-clock time. | [Paper](https://arxiv.org/abs/2305.14342) , [Tweet](https://twitter.com/tengyuma/status/1661412995430219786?s=20) |
|
|
|
| 7) **The Larger They Are, the Harder They Fail** - shows that LLMs fail to generate correct Python code when default function names are swapped; they also strongly prefer incorrect continuation as they become bigger. | [Paper](https://arxiv.org/abs/2305.15507), [Tweet](https://twitter.com/AVMiceliBarone/status/1662150656327663617?s=20) |
|
|
|
| 8) **Model Evaluation for Extreme Risks** - discusses the importance of model evaluation for addressing extreme risks and making responsible decisions about model training, deployment, and security. | [Paper](https://arxiv.org/abs/2305.15324), [Tweet](https://twitter.com/soundboy/status/1661728733156503555?s=20) |
|