Merge branch 'main' of https://github.com/himanshushukla12/llama-recipes

README.md

@@ -76,7 +76,7 @@ Some recipes require the presence of langchain. To install the packages follow t
 ```
 pip install llama-recipes[langchain]
 ```
-Optional dependencies can also be combines with [option1,option2].
+Optional dependencies can also be combined with [option1,option2].
 
 #### Install from source
 To install from source e.g. for development use these commands. We're using hatchling as our build backend which requires an up-to-date pip as well as setuptools package.
@@ -119,7 +119,7 @@ Most of the code dealing with Llama usage is organized across 2 main folders: `r
 
 ### `recipes/`
 
-Contains examples are organized in folders by topic:
+Contains examples organized in folders by topic:
 | Subfolder | Description |
 |---|---|
 [quickstart](./recipes/quickstart) | The "Hello World" of using Llama, start here if you are new to using Llama.

recipes/quickstart/NotebookLlama/Step-1 PDF-Pre-Processing-Logic.ipynb

@@ -59,7 +59,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "pdf_path = './resources/2402.13116v3.pdf'\n",
+    "pdf_path = './resources/2402.13116v4.pdf'\n",
     "DEFAULT_MODEL = \"meta-llama/Llama-3.2-1B-Instruct\""
    ]
   },

recipes/quickstart/RAG/hello_llama_cloud.ipynb

@@ -37,7 +37,8 @@
     "!pip install sentence-transformers\n",
     "!pip install faiss-cpu\n",
     "!pip install bs4\n",
-    "!pip install replicate"
+    "!pip install replicate",
+    "!pip install langchain-community"
    ]
   },
   {

recipes/quickstart/finetuning/datasets/README.md

@@ -58,7 +58,7 @@ To add a custom dataset the following steps need to be performed.
 
 1. Create a dataset configuration after the schema described above. Examples can be found in [configs/datasets.py](../../../../src/llama_recipes/configs/datasets.py).
 2. Create a preprocessing routine which loads the data and returns a PyTorch style dataset. The signature for the preprocessing function needs to be (dataset_config, tokenizer, split_name) where split_name will be the string for train/validation split as defined in the dataclass.
-3. Register the dataset name and preprocessing function by inserting it as key and value into the DATASET_PREPROC dictionary in [utils/dataset_utils.py](../../../../src/llama_recipes/utils/dataset_utils.py)
+3. Register the dataset name and preprocessing function by inserting it as key and value into the DATASET_PREPROC dictionary in [datasets/__init__.py](../../../../src/llama_recipes/datasets/__init__.py)
 4. Set dataset field in training config to dataset name or use --dataset option of the `llama_recipes.finetuning` module or examples/finetuning.py training script.
 
 ## Application