{ "cells": [ { "cell_type": "markdown", "id": "1d5d6034", "metadata": {}, "source": [ "## Part 3: Setting up RAG Example and Validating our Retrieval Pipeline\n", "\n", "We are ready for the finale, but let's recap:\n", "\n", "- We started with an example dataset of 5000 images\n", "- In the first notebook, we cleaned this up for labelling and used `Llama-3.2-11B` model for labelling\n", "- In the second notebook, we cleaned up some hallucinations of the model and pre-processed the descriptions that were synthetically generated\n", "\n", "\n", "Step 3 is to setup a RAG pipeline and profit.\n", "\n", "We will use [lance-db](https://lancedb.com) since it's open source and Llama and open source go well together 🤝\n", "\n", "We also love free API credits and [Together](https://www.together.ai) is hosting 11B model for free. For our final demo, we will use their API and validate the same in this example." ] }, { "cell_type": "code", "execution_count": 12, "id": "ee4b18be-3bf0-4f7c-8ac5-ef68b7566750", "metadata": { "tags": [] }, "outputs": [], "source": [ "#!pip install lancedb rerankers together -q" ] }, { "cell_type": "markdown", "id": "add7aeb6", "metadata": {}, "source": [ "### Loading the Dataset and Creating Embeddings" ] }, { "cell_type": "code", "execution_count": 21, "id": "a13b002d-26bc-4fb0-bd08-193163f2a3d7", "metadata": { "tags": [] }, "outputs": [], "source": [ "import pandas as pd\n", "import os\n", "from together import Together\n", "\n", "os.environ[\"TOGETHER_API_KEY\"] = \"\"\n", "client = Together(api_key=os.environ.get('TOGETHER_API_KEY'))\n", "\n", "df = pd.read_csv(\"./MM-Demo/final_balanced_sample_dataset.csv\")" ] }, { "cell_type": "code", "execution_count": 22, "id": "e2078d99-959e-4bb6-a702-ff3e31761db2", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| \n", " | Filename\n", " | Title\n", " | Size\n", " | Gender\n", " | Description\n", " | Category\n", " | Type\n", " | 
|---|---|---|---|---|---|---|---|
| 0\n", " | d7ed1d64-2c65-427f-9ae4-eb4aaa3e2389.jpg\n", " | Stylish and Trendy Tank Top with Celestial Design\n", " | M\n", " | F\n", " | This white tank top is a stylish and trendy pi...\n", " | Tops\n", " | Casual\n", " | 
| 1\n", " | 5c1b7a77-1fa3-4af8-9722-cd38e45d89da.jpg\n", " | Classic White Sweatshirt\n", " | M\n", " | F\n", " | This classic white sweatshirt is a timeless pi...\n", " | Tops\n", " | Casual\n", " | 
| 2\n", " | b2e084c7-e3a0-4182-8671-b908544a7cf2.jpg\n", " | Grey T-shirt\n", " | M\n", " | Unisex\n", " | This is a short-sleeved, crew neck t-shirt tha...\n", " | T-Shirt\n", " | Casual\n", " | 
| 3\n", " | 87846aa9-86cc-404a-af2c-7e8fe941081d.jpg\n", " | Long-Sleeved V-Neck Shirt\n", " | L\n", " | U\n", " | A long-sleeved, V-neck shirt with a solid purp...\n", " | Tops\n", " | Casual\n", " | 
| 4\n", " | 04fa06fb-d71a-4293-9804-fe799375a682.jpg\n", " | Silver Metallic Buckle Sandals\n", " | L\n", " | F\n", " | These silver metallic buckle sandals feature a...\n", " | Shoes\n", " | Casual\n", " | 
| \n", " | Filename\n", " | Title\n", " | Size\n", " | Gender\n", " | Description\n", " | Category\n", " | Type\n", " | vector\n", " | 
|---|---|---|---|---|---|---|---|---|
| 0\n", " | d7ed1d64-2c65-427f-9ae4-eb4aaa3e2389.jpg\n", " | Stylish and Trendy Tank Top with Celestial Design\n", " | M\n", " | F\n", " | This white tank top is a stylish and trendy pi...\n", " | Tops\n", " | Casual\n", " | [-0.06355423, 0.02385288, 0.03382446, -0.00212...\n", " | 
| 1\n", " | 5c1b7a77-1fa3-4af8-9722-cd38e45d89da.jpg\n", " | Classic White Sweatshirt\n", " | M\n", " | F\n", " | This classic white sweatshirt is a timeless pi...\n", " | Tops\n", " | Casual\n", " | [-0.011691423, 0.049270794, 0.030319242, -0.02...\n", " | 
| 2\n", " | b2e084c7-e3a0-4182-8671-b908544a7cf2.jpg\n", " | Grey T-shirt\n", " | M\n", " | Unisex\n", " | This is a short-sleeved, crew neck t-shirt tha...\n", " | T-Shirt\n", " | Casual\n", " | [-0.010150914, 0.09637642, -0.0012558334, 0.04...\n", " | 
| 3\n", " | 87846aa9-86cc-404a-af2c-7e8fe941081d.jpg\n", " | Long-Sleeved V-Neck Shirt\n", " | L\n", " | U\n", " | A long-sleeved, V-neck shirt with a solid purp...\n", " | Tops\n", " | Casual\n", " | [-0.058710814, 0.053951632, -0.047531255, 0.03...\n", " | 
| 4\n", " | 04fa06fb-d71a-4293-9804-fe799375a682.jpg\n", " | Silver Metallic Buckle Sandals\n", " | L\n", " | F\n", " | These silver metallic buckle sandals feature a...\n", " | Shoes\n", " | Casual\n", " | [0.0123484805, 0.02398385, 0.059779372, -0.006...\n", " | 
| 5\n", " | 8f576f1a-839d-4fb2-a224-a4700b2d05da.jpg\n", " | Orange Long Sleeve T-Shirt\n", " | S\n", " | U\n", " | This long sleeve t-shirt is made of a lightwei...\n", " | Tops\n", " | Casual\n", " | [-0.016778396, 0.06664105, -0.010597771, 0.052...\n", " | 
| 6\n", " | e976a8f6-6731-485f-8a9a-2872a5208818.jpg\n", " | Green T-Shirt\n", " | L\n", " | M\n", " | The green t-shirt is a relaxed fit with a V-ne...\n", " | Tops\n", " | Casual\n", " | [0.015946185, 0.07543863, 0.032447945, 0.00442...\n", " | 
| 7\n", " | bbf0d9c7-663d-46d1-a9f8-66e8e5678541.jpg\n", " | White Quarter Zip Pullover Top\n", " | L\n", " | U\n", " | This pullover top has a quarter zip closure at...\n", " | Tops\n", " | Casual\n", " | [-0.05592024, 0.03401515, -0.0057456787, 0.030...\n", " | 
| 8\n", " | e25a7faa-7a49-4e72-a7ef-e74427f77784.jpg\n", " | Aviator Sunglasses T-Shirt\n", " | M\n", " | F\n", " | This sleeveless red T-shirt features a large b...\n", " | Tops\n", " | Casual\n", " | [-0.040058654, 0.090077035, -0.020889325, 0.03...\n", " | 
| 9\n", " | d995ac1f-fbd0-482c-a308-dafb6a93cfd0.jpg\n", " | Beige Top\n", " | M\n", " | F\n", " | This beige top has a boat neck, a pocket in fr...\n", " | Tops\n", " | Casual\n", " | [-0.020475557, 0.032990508, 0.0145089375, -0.0...\n", " | 
| \n", " | Filename\n", " | Title\n", " | Size\n", " | Gender\n", " | Description\n", " | Category\n", " | Type\n", " | vector\n", " | _distance\n", " | 
|---|---|---|---|---|---|---|---|---|---|
| 0\n", " | 7a6e7ddb-ac03-4d78-9d71-75320e5913da.jpg\n", " | Light-Washed Jeans\n", " | S\n", " | F\n", " | These light-washed jeans are a pair of stylish...\n", " | Jeans\n", " | Casual\n", " | [-0.063473865, 0.060125574, 0.02189098, -0.002...\n", " | 0.407864\n", " | 
| 1\n", " | 7a6e7ddb-ac03-4d78-9d71-75320e5913da.jpg\n", " | Light-Washed Jeans\n", " | S\n", " | F\n", " | These light-washed jeans are a pair of stylish...\n", " | Jeans\n", " | Casual\n", " | [-0.063473865, 0.060125574, 0.02189098, -0.002...\n", " | 0.407864\n", " | 
| 2\n", " | 8db4d594-4688-4447-a5b9-cd5c68b1870e.jpg\n", " | Black Pant\n", " | S\n", " | U\n", " | This pair of black pants appears to be made of...\n", " | Pants\n", " | Casual\n", " | [-0.014388849, 0.082536675, 0.011225891, -0.00...\n", " | 0.412849\n", " | 
| \n", " | filename\n", " | Description\n", " | vector\n", " | Title\n", " | Size\n", " | Category\n", " | Type\n", " | Gender\n", " | _score\n", " | 
|---|---|---|---|---|---|---|---|---|---|
| 0\n", " | 0e20abb0-d56a-4f83-b254-9db70fb92794.jpg\n", " | This shirt is a long-sleeved, black and grey s...\n", " | [-0.045025624, 0.041662034, -0.0006505165, 0.0...\n", " | Black and Grey Striped Shirt with Pink Accents\n", " | S\n", " | Tops\n", " | Casual\n", " | F\n", " | 52.089787\n", " | 
| 1\n", " | 10e51a41-3729-472e-b288-f3b8f895577a.jpg\n", " | This plaid shirt features a classic plaid patt...\n", " | [-0.044541422, 0.05006024, -0.04257077, 0.0401...\n", " | Plaid shirt with embroidered hem\n", " | L\n", " | Tops\n", " | Casual\n", " | F\n", " | 35.616638\n", " | 
| 2\n", " | bd289469-607c-45df-bad3-ed1057e277bf.jpg\n", " | This is a red and black plaid skirt with a cla...\n", " | [-0.021971317, 0.12156065, -0.002515854, 0.034...\n", " | Red and Black Plaid Skirt\n", " | S\n", " | Skirts\n", " | Casual\n", " | F\n", " | 34.096905\n", " | 
| 3\n", " | 64c5d20e-7cec-4965-8c50-315389f44b22.jpg\n", " | These light washed blue jeans are distressed w...\n", " | [-0.07545003, 0.05375045, 0.00353413, -0.01551...\n", " | Light Washed Blue Ripped Jeans\n", " | L\n", " | Jeans\n", " | Casual\n", " | M\n", " | 29.116760\n", " | 
| 4\n", " | b0c03127-9dfb-4573-8934-1958396937bf.jpg\n", " | This red flannel plaid shirt has a classic and...\n", " | [-0.03013145, 0.060508166, -0.03642542, 0.0301...\n", " | Red Flannel Plaid Shirt\n", " | S\n", " | Shirts\n", " | Casual\n", " | M\n", " | 28.444168\n", " | 
| \n", " | filename\n", " | Description\n", " | vector\n", " | Title\n", " | Size\n", " | Category\n", " | Type\n", " | Gender\n", " | _relevance_score\n", " | 
|---|---|---|---|---|---|---|---|---|---|
| 0\n", " | 0e20abb0-d56a-4f83-b254-9db70fb92794.jpg\n", " | This shirt is a long-sleeved, black and grey s...\n", " | [-0.045025624, 0.041662034, -0.0006505165, 0.0...\n", " | Black and Grey Striped Shirt with Pink Accents\n", " | S\n", " | Tops\n", " | Casual\n", " | F\n", " | 0.744141\n", " | 
| 1\n", " | 10e51a41-3729-472e-b288-f3b8f895577a.jpg\n", " | This plaid shirt features a classic plaid patt...\n", " | [-0.044541422, 0.05006024, -0.04257077, 0.0401...\n", " | Plaid shirt with embroidered hem\n", " | L\n", " | Tops\n", " | Casual\n", " | F\n", " | 0.735352\n", " | 
| 2\n", " | 300a4125-acec-46aa-b0b9-68b4b2142c70.jpg\n", " | These plaid shorts are a stylish and comfortab...\n", " | [-0.002765015, 0.030705247, -0.010424565, -0.0...\n", " | Plaid Shorts\n", " | S\n", " | Pants\n", " | Casual\n", " | M\n", " | 0.728516\n", " | 
| 3\n", " | bd289469-607c-45df-bad3-ed1057e277bf.jpg\n", " | This is a red and black plaid skirt with a cla...\n", " | [-0.021971317, 0.12156065, -0.002515854, 0.034...\n", " | Red and Black Plaid Skirt\n", " | S\n", " | Skirts\n", " | Casual\n", " | F\n", " | 0.699219\n", " | 
| 4\n", " | b0c03127-9dfb-4573-8934-1958396937bf.jpg\n", " | This red flannel plaid shirt has a classic and...\n", " | [-0.03013145, 0.060508166, -0.03642542, 0.0301...\n", " | Red Flannel Plaid Shirt\n", " | S\n", " | Shirts\n", " | Casual\n", " | M\n", " | 0.679199\n", " |