{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Retrieval-Augmented Generation (RAG) with Llama3 8B\n",
"[](https://colab.research.google.com/github/togethercomputer/together-cookbook/blob/main/Text_RAG.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction\n",
"\n",
"For AI models to be effective in specialized tasks, they often require domain-specific knowledge. For instance, a financial advisory chatbot needs to understand market trends and products offered by a specific bank, while an AI legal assistant must be equipped with knowledge of statutes, regulations, and past case law.\n",
"\n",
"A common solution is Retrieval-Augmented Generation (RAG), which retrieves relevant data from a knowledge base and combines it with the user’s prompt, thereby improving and customizing the model's output to the provided data.\n",
"\n",
"![](\"images/simple_RAG.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## RAG Explanation\n",
"\n",
"RAG operates by preprocessing a large knowledge base and dynamically retrieving relevant information at runtime.\n",
"\n",
"Here's a breakdown of the process:\n",
"\n",
"1. Indexing the Knowledge Base:\n",
"The corpus (collection of documents) is divided into smaller, manageable chunks of text. Each chunk is converted into a vector embedding using an embedding model. These embeddings are stored in a vector database optimized for similarity searches.\n",
"\n",
"2. Query Processing and Retrieval:\n",
"When a user submits a prompt that would initially go directly to a LLM we process that and extract a query, the system searches the vector database for chunks semantically similar to the query. The most relevant chunks are retrieved and injected into the prompt sent to the generative AI model.\n",
"\n",
"3. Response Generation:\n",
"The AI model then uses the retrieved information along with its pre-trained knowledge to generate a response. Not only does this reduce the likelihood of hallucination since relevant context is provided directly in the prompt but it also allows us to cite to source material as well.\n",
"\n",
"![](\"images/text_RAG.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Install libraries"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ubSQgZNalImb",
"outputId": "536b046c-306e-490c-97d8-40a7dfdaba24"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting together\n",
" Downloading together-1.3.3-py3-none-any.whl.metadata (11 kB)\n",
"Requirement already satisfied: aiohttp<4.0.0,>=3.9.3 in /usr/local/lib/python3.10/dist-packages (from together) (3.10.10)\n",
"Requirement already satisfied: click<9.0.0,>=8.1.7 in /usr/local/lib/python3.10/dist-packages (from together) (8.1.7)\n",
"Requirement already satisfied: eval-type-backport<0.3.0,>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from together) (0.2.0)\n",
"Requirement already satisfied: filelock<4.0.0,>=3.13.1 in /usr/local/lib/python3.10/dist-packages (from together) (3.16.1)\n",
"Requirement already satisfied: numpy>=1.23.5 in /usr/local/lib/python3.10/dist-packages (from together) (1.26.4)\n",
"Requirement already satisfied: pillow<11.0.0,>=10.3.0 in /usr/local/lib/python3.10/dist-packages (from together) (10.4.0)\n",
"Requirement already satisfied: pyarrow>=10.0.1 in /usr/local/lib/python3.10/dist-packages (from together) (16.1.0)\n",
"Requirement already satisfied: pydantic<3.0.0,>=2.6.3 in /usr/local/lib/python3.10/dist-packages (from together) (2.9.2)\n",
"Requirement already satisfied: requests<3.0.0,>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from together) (2.32.3)\n",
"Requirement already satisfied: rich<14.0.0,>=13.8.1 in /usr/local/lib/python3.10/dist-packages (from together) (13.9.2)\n",
"Requirement already satisfied: tabulate<0.10.0,>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from together) (0.9.0)\n",
"Requirement already satisfied: tqdm<5.0.0,>=4.66.2 in /usr/local/lib/python3.10/dist-packages (from together) (4.66.5)\n",
"Requirement already satisfied: typer<0.13,>=0.9 in /usr/local/lib/python3.10/dist-packages (from together) (0.12.5)\n",
"Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (2.4.3)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (1.3.1)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (24.2.0)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (1.4.1)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (6.1.0)\n",
"Requirement already satisfied: yarl<2.0,>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (1.15.2)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.9.3->together) (4.0.3)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3.0.0,>=2.6.3->together) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic<3.0.0,>=2.6.3->together) (2.23.4)\n",
"Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3.0.0,>=2.6.3->together) (4.12.2)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->together) (3.4.0)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->together) (3.10)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->together) (2.2.3)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->together) (2024.8.30)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich<14.0.0,>=13.8.1->together) (3.0.0)\n",
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich<14.0.0,>=13.8.1->together) (2.18.0)\n",
"Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from typer<0.13,>=0.9->together) (1.5.4)\n",
"Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich<14.0.0,>=13.8.1->together) (0.1.2)\n",
"Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from yarl<2.0,>=1.12.0->aiohttp<4.0.0,>=3.9.3->together) (0.2.0)\n",
"Downloading together-1.3.3-py3-none-any.whl (68 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m68.1/68.1 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hInstalling collected packages: together\n",
"Successfully installed together-1.3.3\n"
]
}
],
"source": [
"!pip install together"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "YO3t0PAHlpaE"
},
"outputs": [],
"source": [
"import together, os\n",
"from together import Together\n",
"\n",
"# Paste in your Together AI API Key or load it\n",
"TOGETHER_API_KEY = os.environ.get(\"TOGETHER_API_KEY\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Download and View the Dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Let's get the movies dataset\n",
"!wget https://raw.githubusercontent.com/togethercomputer/together-cookbook/refs/heads/main/datasets/movies.json\n",
"!mkdir datasets\n",
"!mv movies.json datasets/movies.json"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "2croRfETmD0s",
"outputId": "a6e88271-b4e0-4ead-ced7-4905ae4ea442"
},
"outputs": [
{
"data": {
"text/plain": [
"[{'title': 'Minions',\n",
" 'overview': 'Minions Stuart, Kevin and Bob are recruited by Scarlet Overkill, a super-villain who, alongside her inventor husband Herb, hatches a plot to take over the world.',\n",
" 'director': 'Kyle Balda',\n",
" 'genres': 'Family Animation Adventure Comedy',\n",
" 'tagline': 'Before Gru, they had a history of bad bosses'},\n",
" {'title': 'Interstellar',\n",
" 'overview': 'Interstellar chronicles the adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage.',\n",
" 'director': 'Christopher Nolan',\n",
" 'genres': 'Adventure Drama Science Fiction',\n",
" 'tagline': 'Mankind was born on Earth. It was never meant to die here.'},\n",
" {'title': 'Deadpool',\n",
" 'overview': 'Deadpool tells the origin story of former Special Forces operative turned mercenary Wade Wilson, who after being subjected to a rogue experiment that leaves him with accelerated healing powers, adopts the alter ego Deadpool. Armed with his new abilities and a dark, twisted sense of humor, Deadpool hunts down the man who nearly destroyed his life.',\n",
" 'director': 'Tim Miller',\n",
" 'genres': 'Action Adventure Comedy',\n",
" 'tagline': 'Witness the beginning of a happy ending'}]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import json\n",
"\n",
"with open('./datasets/movies.json', 'r') as file:\n",
" movies_data = json.load(file)\n",
"\n",
"movies_data[:3]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Implement Retreival Pipeline - \"R\" part of RAG\n",
"\n",
"Below we implement a simple retreival pipeline:\n",
"1. Embed movie documents + query\n",
"2. Obtain top k movies ranked based on cosine similarities between the query and movie vectors."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "I0o-ZpaDlsgZ"
},
"outputs": [],
"source": [
"# This function will be used to access the Together API to generate embeddings for the movie plots\n",
"\n",
"from typing import List\n",
"import numpy as np\n",
"\n",
"def generate_embeddings(input_texts: List[str], model_api_string: str) -> List[List[float]]:\n",
" \"\"\"Generate embeddings from Together python library.\n",
"\n",
" Args:\n",
" input_texts: a list of string input texts.\n",
" model_api_string: str. An API string for a specific embedding model of your choice.\n",
"\n",
" Returns:\n",
" embeddings_list: a list of embeddings. Each element corresponds to the each input text.\n",
" \"\"\"\n",
" together_client = together.Together(api_key = TOGETHER_API_KEY)\n",
" outputs = together_client.embeddings.create(\n",
" input=input_texts,\n",
" model=model_api_string,\n",
" )\n",
" return np.array([x.embedding for x in outputs.data])"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "uwk2bwNGl84p",
"outputId": "46797774-5d73-412a-db76-ee5e0d94ac85"
},
"outputs": [
{
"data": {
"text/plain": [
"['Minions Minions Stuart, Kevin and Bob are recruited by Scarlet Overkill, a super-villain who, alongside her inventor husband Herb, hatches a plot to take over the world. Before Gru, they had a history of bad bosses',\n",
" 'Interstellar Interstellar chronicles the adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage. Mankind was born on Earth. It was never meant to die here.',\n",
" 'Deadpool Deadpool tells the origin story of former Special Forces operative turned mercenary Wade Wilson, who after being subjected to a rogue experiment that leaves him with accelerated healing powers, adopts the alter ego Deadpool. Armed with his new abilities and a dark, twisted sense of humor, Deadpool hunts down the man who nearly destroyed his life. Witness the beginning of a happy ending',\n",
" 'Guardians of the Galaxy Light years from Earth, 26 years after being abducted, Peter Quill finds himself the prime target of a manhunt after discovering an orb wanted by Ronan the Accuser. All heroes start somewhere.',\n",
" \"Mad Max: Fury Road An apocalyptic story set in the furthest reaches of our planet, in a stark desert landscape where humanity is broken, and most everyone is crazed fighting for the necessities of life. Within this world exist two rebels on the run who just might be able to restore order. There's Max, a man of action and a man of few words, who seeks peace of mind following the loss of his wife and child in the aftermath of the chaos. And Furiosa, a woman of action and a woman who believes her path to survival may be achieved if she can make it across the desert back to her childhood homeland. What a Lovely Day.\",\n",
" 'Jurassic World Twenty-two years after the events of Jurassic Park, Isla Nublar now features a fully functioning dinosaur theme park, Jurassic World, as originally envisioned by John Hammond. The park is open.',\n",
" \"Pirates of the Caribbean: The Curse of the Black Pearl Jack Sparrow, a freewheeling 17th-century pirate who roams the Caribbean Sea, butts heads with a rival pirate bent on pillaging the village of Port Royal. When the governor's daughter is kidnapped, Sparrow decides to help the girl's love save her. But their seafaring mission is hardly simple. Prepare to be blown out of the water.\",\n",
" 'Dawn of the Planet of the Apes A group of scientists in San Francisco struggle to stay alive in the aftermath of a plague that is wiping out humanity, while Caesar tries to maintain dominance over his community of intelligent apes. One last chance for peace.',\n",
" 'The Hunger Games: Mockingjay - Part 1 Katniss Everdeen reluctantly becomes the symbol of a mass rebellion against the autocratic Capitol. Fire burns brighter in the darkness',\n",
" 'Big Hero 6 The special bond that develops between plus-sized inflatable robot Baymax, and prodigy Hiro Hamada, who team up with a group of friends to form a band of high-tech heroes. From the creators of Wreck-it Ralph and Frozen']"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Concatenate the title, overview, and tagline of each movie\n",
"# this makes the text that will be embedded for each movie more informative\n",
"# as a result the embeddings will be richer and capture this information.\n",
"\n",
"to_embed = []\n",
"for movie in movies_data[:1000]:\n",
" text = ''\n",
" for field in ['title', 'overview', 'tagline']:\n",
" value = movie.get(field, '')\n",
" text += str(value) + ' '\n",
" to_embed.append(text.strip())\n",
"\n",
"to_embed[:10]"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "QMqmezr8mCUG"
},
"outputs": [],
"source": [
"# Use bge-base-en-v1.5 model to generate embeddings\n",
"embeddings = generate_embeddings(to_embed, 'BAAI/bge-base-en-v1.5')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"id": "g1HTX3bGmYCk"
},
"outputs": [],
"source": [
"# Generate the vector embeddings for the query\n",
"query = \"super hero action movie with a timeline twist\"\n",
"\n",
"query_embedding = generate_embeddings([query], 'BAAI/bge-base-en-v1.5')[0]"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"id": "eqQe4VsBmp7w"
},
"outputs": [],
"source": [
"from sklearn.metrics.pairwise import cosine_similarity\n",
"\n",
"# Calculate cosine similarity between the query embedding and each movie embedding\n",
"similarity_scores = cosine_similarity([query_embedding], embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "coKcFNlzmsrh",
"outputId": "2ee108a4-e98d-4c6f-b09e-4eb009ed6e2f"
},
"outputs": [
{
"data": {
"text/plain": [
"(1, 1000)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# We get a similarity score for each of our 1000 movies - the higher the score, the more similar the movie is to the query\n",
"similarity_scores.shape"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"id": "0rvxmdvKmuhY"
},
"outputs": [],
"source": [
"# Get the indices of the highest to lowest values\n",
"indices = np.argsort(-similarity_scores)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3m0zGxlLm1Zp",
"outputId": "0c2c5d55-ed80-4b8e-ce56-f2750d308dea"
},
"outputs": [
{
"data": {
"text/plain": [
"['The Incredibles',\n",
" 'Watchmen',\n",
" 'Mr. Peabody & Sherman',\n",
" 'Due Date',\n",
" 'The Next Three Days',\n",
" 'Super 8',\n",
" 'Iron Man',\n",
" 'After Earth',\n",
" 'Men in Black 3',\n",
" 'Despicable Me 2']"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"top_10_sorted_titles = [movies_data[index]['title'] for index in indices[0]][:10]\n",
"\n",
"top_10_sorted_titles"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Retreiver Function\n",
"\n",
"Once we understand the steps in the retriever pipeline above we can simplify it into the function below."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "R6GLqD9_m2WN"
},
"outputs": [],
"source": [
"def retreive(query: str, top_k: int = 5, index: np.ndarray = None) -> List[int]:\n",
" \"\"\"\n",
" Retrieve the top-k most similar items from an index based on a query.\n",
" Args:\n",
" query (str): The query string to search for.\n",
" top_k (int, optional): The number of top similar items to retrieve. Defaults to 5.\n",
" index (np.ndarray, optional): The index array containing embeddings to search against. Defaults to None.\n",
" Returns:\n",
" List[int]: A list of indices corresponding to the top-k most similar items in the index.\n",
" \"\"\"\n",
" \n",
" query_embedding = generate_embeddings([query], 'BAAI/bge-base-en-v1.5')[0]\n",
" similarity_scores = cosine_similarity([query_embedding], index)\n",
"\n",
" return np.argsort(-similarity_scores)[0][:top_k]"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "gJ989oVqnrW-",
"outputId": "9b63fd61-fd3a-4c97-b1c7-faddc54b7d47"
},
"outputs": [
{
"data": {
"text/plain": [
"array([172, 265, 768, 621, 929])"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"retreive(\"super hero action movie with a timeline twist\", top_k=5, index = embeddings)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generation Step - \"G\" part of RAG\n",
"\n",
"Below we will inject/augment the information the retreival pipeline extracts into the prompt to the Llama3 8b Model. \n",
"\n",
"This will help guide the generation by grounding it from facts in our knowledge base!"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"id": "YLoZYcx9nvAZ"
},
"outputs": [],
"source": [
"# Extract out the titles and overviews of the top 10 most similar movies\n",
"titles = [movies_data[index]['title'] for index in indices[0]][:10]\n",
"overviews = [movies_data[index]['overview'] for index in indices[0]][:10]"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "gTuUW3HOn_AA",
"outputId": "65909533-a0f4-45f2-bfe2-021c94d75aac"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"What a delightful mix of plots! Here's a story that weaves them together:\n",
"\n",
"In a world where superheroes are a thing of the past, Bob Parr, aka Mr. Incredible, has given up his life of saving the world to become an insurance adjuster in the suburbs. His wife, Helen, aka Elastigirl, has also hung up her superhero suit to raise their three children. However, when Bob receives a mysterious assignment from a secret organization, he's forced to don his old costume once again.\n",
"\n",
"As Bob delves deeper into the assignment, he discovers that it's connected to a sinister plot to destroy the world. The plot is masterminded by a group of rogue superheroes, who were once part of the Watchmen, a group of vigilantes that were disbanded by the government in the 1980s.\n",
"\n",
"The Watchmen, led by the enigmatic Rorschach, have been secretly rebuilding their team and are now determined to take revenge on the world that wronged them. Bob must team up with his old friends, including the brilliant scientist, Dr. Manhattan, to stop the Watchmen and prevent their destruction.\n",
"\n",
"Meanwhile, in a different part of the world, a young boy named Sherman, who has a genius-level IQ, has built a time-travel machine with his dog, Penny. When the machine is stolen, Sherman and Penny must travel through time to prevent a series of catastrophic events from occurring.\n",
"\n",
"As they travel through time, they encounter a group of friends who are making a zombie movie with a Super-8 camera. The friends, including a young boy named Charles, witness a train derailment and soon discover that it was no accident. They team up with Sherman and Penny to uncover the truth behind the crash and prevent a series of unexplained events and disappearances.\n",
"\n",
"As the story unfolds, Bob and his friends must navigate a complex web of time travel and alternate realities to stop the Watchmen and prevent the destruction of the world. Along the way, they encounter a group of agents from the Men in Black, who are trying to prevent a catastrophic event from occurring.\n",
"\n",
"The agents, led by Agents J and K, are on a mission to stop a powerful new super criminal, who is threatening to destroy the world. They team up with Bob and his friends to prevent the destruction and save the world.\n",
"\n",
"In the end, Bob and his friends succeed in stopping the Watchmen and preventing the destruction of the world. However, the journey is not without its challenges, and Bob must confront his own demons and learn to balance his life as a superhero with his life as a husband and father.\n",
"\n",
"The story concludes with Bob and his family returning to their normal lives, but with a newfound appreciation for the importance of family and the power of teamwork. The movie ends with a shot of the Parr family, including their three children, who are all wearing superhero costumes, ready to take on the next adventure that comes their way.\n"
]
}
],
"source": [
"client = Together(api_key = TOGETHER_API_KEY)\n",
"\n",
"# Generate a story based on the top 10 most similar movies\n",
"\n",
"response = client.chat.completions.create(\n",
" model=\"meta-llama/Llama-3-8b-chat-hf\",\n",
" messages=[\n",
" {\"role\": \"system\", \"content\": \"You are a pulitzer award winning craftful story teller. Given only the overview of different plots you can weave together an interesting storyline.\"},\n",
" {\"role\": \"user\", \"content\": f\"Tell me a story about {titles}. Here is some information about them {overviews}\"},\n",
" ],\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we can see a simple RAG pipeline where we use semantic search to perform retreival and pass relevant information into the prompt of a LLM to condition its generation.\n",
"\n",
"To learn more about the Together AI API please refer to the [docs here](https://docs.together.ai/docs/introduction)!"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}