| 
					
				 | 
			
			
				@@ -1,6 +1,6 @@ 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 # **ML Papers of The Week** 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				  
			 | 
		
	
		
			
				 | 
				 | 
			
			
				-[Subscribe to our newletter](https://nlpnews.substack.com/) to get a weekly list of top ML papers in your inbox. 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				+[Subscribe to our newsletter](https://nlpnews.substack.com/) to get a weekly list of top ML papers in your inbox. 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				  
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 At DAIR.AI we ❤️ reading ML papers so we've created this repo to highlight the top ML papers of every week. 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				  
			 | 
		
	
	
		
			
				| 
					
				 | 
			
			
				@@ -39,7 +39,7 @@ At DAIR.AI we ❤️ reading ML papers so we've created this repo to highlight t 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 | 2) **LIMA** - a new 65B parameter LLaMa model fine-tuned on 1000 carefully curated prompts and responses; it doesn't use RLHF, generalizes well to unseen tasks not available in the training data, and generates responses equivalent or preferred to GPT-4 in 43% of cases, and even higher compared to Bard. | [Paper](https://arxiv.org/abs/2305.11206), [Tweet](https://twitter.com/violet_zct/status/1660789120069926912?s=20) | 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 | 3) **Voyager** - an LLM-powered embodied lifelong learning agent in Minecraft that can continuously explore worlds, acquire skills, and make novel discoveries without human intervention. | [Paper](https://arxiv.org/abs/2305.16291), [Tweet](https://twitter.com/DrJimFan/status/1662115266933972993?s=20) | 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 | 4) **Gorilla** - a finetuned LLaMA-based model that surpasses GPT-4 on writing API calls. This capability can help identify the right API, boosting the ability of LLMs to interact with external tools to complete specific tasks. | [Paper](https://arxiv.org/abs/2305.15334), [Tweet](https://twitter.com/omarsar0/status/1661540207206846464?s=20) | 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				-| 5. **The False Promise of Imitatiting Proprietary LLMs** -  provides a critical analysis of models that are finetuned on the outputs of a stronger model; argues that model imitation is a false premise and that the higher leverage action to improve open source models is to develop better base models. | [Paper](https://arxiv.org/abs/2305.15717) , [Tweet](https://twitter.com/arankomatsuzaki/status/1661908342829187072?s=20) 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				+| 5. **The False Promise of Imitating Proprietary LLMs** -  provides a critical analysis of models that are finetuned on the outputs of a stronger model; argues that model imitation is a false premise and that the higher leverage action to improve open source models is to develop better base models. | [Paper](https://arxiv.org/abs/2305.15717) , [Tweet](https://twitter.com/arankomatsuzaki/status/1661908342829187072?s=20) 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 | 6) **Sophia** - presents a simple scalable second-order optimizer that has negligible average per-step time and memory overhead; on language modeling, Sophia achieves 2x speed-up compared to Adam in the number of steps, total compute, and wall-clock time. | [Paper](https://arxiv.org/abs/2305.14342) , [Tweet](https://twitter.com/tengyuma/status/1661412995430219786?s=20)  | 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 | 7) **The Larger They Are, the Harder They Fail** - shows that LLMs fail to generate correct Python code when default function names are swapped; they also strongly prefer incorrect continuation as they become bigger. | [Paper](https://arxiv.org/abs/2305.15507),  [Tweet](https://twitter.com/AVMiceliBarone/status/1662150656327663617?s=20)  | 
			 | 
		
	
		
			
				 | 
				 | 
			
			
				 | 8) **Model Evaluation for Extreme Risks** - discusses the importance of model evaluation for addressing extreme risks and making responsible decisions about model training, deployment, and security. | [Paper](https://arxiv.org/abs/2305.15324),  [Tweet](https://twitter.com/soundboy/status/1661728733156503555?s=20)  | 
			 |