|
|
@@ -1,937 +0,0 @@
|
|
|
-{
|
|
|
- "cells": [
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "09211e76-286f-4b12-acd7-cfb082dc2d66",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "# Llama 3 Cookbook with LlamaIndex and Groq\n",
|
|
|
- "\n",
|
|
|
- "<a href=\"https://colab.research.google.com/github/meta-llama/llama-recipes/blob/main/recipes/llama_api_providers/llama3_cookbook_groq.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
|
|
|
- "\n",
|
|
|
- "Meta developed and released the Meta [Llama 3](https://ai.meta.com/blog/meta-llama-3/) family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks.\n",
|
|
|
- "\n",
|
|
|
- "In this notebook, we demonstrate how to use Llama 3 with LlamaIndex for a comprehensive set of use cases. \n",
|
|
|
- "1. Basic completion / chat \n",
|
|
|
- "2. Basic RAG (Vector Search, Summarization)\n",
|
|
|
- "3. Advanced RAG (Routing)\n",
|
|
|
- "4. Text-to-SQL \n",
|
|
|
- "5. Structured Data Extraction\n",
|
|
|
- "6. Chat Engine + Memory\n",
|
|
|
- "7. Agents\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "We use Llama3-8B and Llama3-70B through [Groq](https://groq.com) - you can sign up there to get a free trial API key."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "de2901c0-e20d-48e5-9385-dbca2258c564",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## Installation and Setup"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "bcf643ac-b025-4812-aaed-f8f85d1ba505",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "!pip install llama-index\n",
|
|
|
- "!pip install llama-index-llms-groq\n",
|
|
|
- "!pip install llama-index-embeddings-huggingface\n",
|
|
|
- "!pip install llama-parse"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "641fa5c8-d63e-47f8-b5bc-ebf994f6e314",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "import nest_asyncio\n",
|
|
|
- "\n",
|
|
|
- "nest_asyncio.apply()"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "1714ea83-6cd4-44bb-b53f-4499126c3809",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Setup LLM using Groq\n",
|
|
|
- "\n",
|
|
|
- "To use [Groq](https://groq.com), you need to make sure that `GROQ_API_KEY` is specified as an environment variable."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "5d46440c",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "import os\n",
|
|
|
- "\n",
|
|
|
- "os.environ[\"GROQ_API_KEY\"] = \"YOUR_GROQ_API_KEY\""
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "d5256970-eba4-499a-b438-8766a290a61a",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.llms.groq import Groq\n",
|
|
|
- "\n",
|
|
|
- "llm = Groq(model=\"llama3-8b-8192\")\n",
|
|
|
- "llm_70b = Groq(model=\"llama3-70b-8192\")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "41c3f154-d345-465d-8eed-63b99adbd3ca",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Setup Embedding Model"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "0cda736d-e414-44e3-8c15-6be49f5f0282",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.embeddings.huggingface import HuggingFaceEmbedding\n",
|
|
|
- "\n",
|
|
|
- "embed_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-small-en-v1.5\")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "3625cf29-7c56-475a-8efd-fbe8ffce194d",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Define Global Settings Configuration\n",
|
|
|
- "\n",
|
|
|
- "In LlamaIndex, you can define global settings so you don't have to pass the LLM / embedding model objects everywhere."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "be3565d1-cc5b-4149-ad5a-7be8f7818e0c",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core import Settings\n",
|
|
|
- "\n",
|
|
|
- "Settings.llm = llm\n",
|
|
|
- "Settings.embed_model = embed_model"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "42449b68-47f5-40cf-9207-191307b25e8e",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Download Data\n",
|
|
|
- "\n",
|
|
|
- "Here you'll download data that's used in section 2 and onwards.\n",
|
|
|
- "\n",
|
|
|
- "We'll download some articles on Kendrick, Drake, and their beef (as of May 2024)."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "59b18640-cdfa-42c1-ab53-115983c1fdc4",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "!mkdir data\n",
|
|
|
- "!wget \"https://www.dropbox.com/scl/fi/t1soxfjdp0v44an6sdymd/drake_kendrick_beef.pdf?rlkey=u9546ymb7fj8lk2v64r6p5r5k&st=wjzzrgil&dl=1\" -O data/drake_kendrick_beef.pdf\n",
|
|
|
- "!wget \"https://www.dropbox.com/scl/fi/nts3n64s6kymner2jppd6/drake.pdf?rlkey=hksirpqwzlzqoejn55zemk6ld&st=mohyfyh4&dl=1\" -O data/drake.pdf\n",
|
|
|
- "!wget \"https://www.dropbox.com/scl/fi/8ax2vnoebhmy44bes2n1d/kendrick.pdf?rlkey=fhxvn94t5amdqcv9vshifd3hj&st=dxdtytn6&dl=1\" -O data/kendrick.pdf"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "9edee491-05f8-4fbb-9394-baa82f1e5087",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Load Data\n",
|
|
|
- "\n",
|
|
|
- "We load data using LlamaParse by default, but you can also choose to opt for our free pypdf reader (in SimpleDirectoryReader by default) if you don't have an account! \n",
|
|
|
- "\n",
|
|
|
- "1. LlamaParse: Signup for an account here: cloud.llamaindex.ai. You get 1k free pages a day, and paid plan is 7k free pages + 0.3c per additional page. LlamaParse is a good option if you want to parse complex documents, like PDFs with charts, tables, and more. \n",
|
|
|
- "\n",
|
|
|
- "2. Default PDF Parser (In `SimpleDirectoryReader`). If you don't want to signup for an account / use a PDF service, just use the default PyPDF reader bundled in our file loader. It's a good choice for getting started!"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "b648635a-2672-407f-bae6-01660e5426d7",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "# Uncomment this code if you want to use LlamaParse\n",
|
|
|
- "# from llama_parse import LlamaParse\n",
|
|
|
- "\n",
|
|
|
- "# docs_kendrick = LlamaParse(result_type=\"text\").load_data(\"./data/kendrick.pdf\")\n",
|
|
|
- "# docs_drake = LlamaParse(result_type=\"text\").load_data(\"./data/drake.pdf\")\n",
|
|
|
- "# docs_both = LlamaParse(result_type=\"text\").load_data(\n",
|
|
|
- "# \"./data/drake_kendrick_beef.pdf\"\n",
|
|
|
- "# )\n",
|
|
|
- "\n",
|
|
|
- "# Uncomment this code if you want to use SimpleDirectoryReader / default PDF Parser\n",
|
|
|
- "# from llama_index.core import SimpleDirectoryReader\n",
|
|
|
- "\n",
|
|
|
- "# docs_kendrick = SimpleDirectoryReader(input_files=[\"data/kendrick.pdf\"]).load_data()\n",
|
|
|
- "# docs_drake = SimpleDirectoryReader(input_files=[\"data/drake.pdf\"]).load_data()\n",
|
|
|
- "# docs_both = SimpleDirectoryReader(input_files=[\"data/drake_kendrick_beef.pdf\"]).load_data()"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "071a8f44-2765-4d57-b8da-15d3c718874d",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 1. Basic Completion and Chat"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "c0b1ace8-32fb-46b2-a065-8817ddc0310b",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Call complete with a prompt"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "a2db43f9-74af-453c-9f83-8db0379c3302",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = llm.complete(\"do you like drake or kendrick better?\")\n",
|
|
|
- "\n",
|
|
|
- "print(response)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "89326153-e2d2-4136-8193-fb27d20670c3",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "stream_response = llm.stream_complete(\n",
|
|
|
- " \"you're a drake fan. tell me why you like drake more than kendrick\"\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "for t in stream_response:\n",
|
|
|
- " print(t.delta, end=\"\")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "a4558339-c8a1-4d26-a430-eb71768b5351",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Call chat with a list of messages"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "5f393031-f743-4a28-a122-71817e3fbd1b",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core.llms import ChatMessage\n",
|
|
|
- "\n",
|
|
|
- "messages = [\n",
|
|
|
- " ChatMessage(role=\"system\", content=\"You are Kendrick.\"),\n",
|
|
|
- " ChatMessage(role=\"user\", content=\"Write a verse.\"),\n",
|
|
|
- "]\n",
|
|
|
- "response = llm.chat(messages)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "8e9551fc-0efc-4671-bc57-339121004c39",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "print(response)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "6a67a33d-fe7d-4381-983f-ca3a6945995d",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 2. Basic RAG (Vector Search, Summarization)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "c104a0c5-e43b-475b-9fa6-186906c1f327",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Basic RAG (Vector Search)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "216787b7-e40a-43fc-a4ca-c43cb798ce9e",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core import VectorStoreIndex\n",
|
|
|
- "\n",
|
|
|
- "index = VectorStoreIndex.from_documents(docs_both)\n",
|
|
|
- "query_engine = index.as_query_engine(similarity_top_k=3)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "a854e9d3-70f1-4927-a2f6-59e90c31f2f0",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = query_engine.query(\"Tell me about family matters\")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "da796970-bc38-4cb4-9d32-ebd1b71d4bdc",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "print(str(response))"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "eff935b7-4f37-4758-8997-82fb0852e732",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Basic RAG (Summarization)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "dfe72300-7a38-453e-b1f2-bc1c00a01ff7",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core import SummaryIndex\n",
|
|
|
- "\n",
|
|
|
- "summary_index = SummaryIndex.from_documents(docs_both)\n",
|
|
|
- "summary_engine = summary_index.as_query_engine()"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "178f1f12-51f7-4b45-9346-c16ed12b3b8d",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = summary_engine.query(\n",
|
|
|
- " \"Given your assessment of this article, who won the beef?\"\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "b8125382-d576-4b99-a0da-2fbb71a5b19b",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "print(str(response))"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "68918eb6-f1e6-460c-b1d5-fb49c3fed4b8",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 3. Advanced RAG (Routing)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "94fd7097-0287-4522-8e43-3e088291fa8a",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Build a Router that can choose whether to do vector search or summarization"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "3949dd41-e9a1-47f6-900f-4f987cad3f84",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core.tools import QueryEngineTool, ToolMetadata\n",
|
|
|
- "\n",
|
|
|
- "vector_tool = QueryEngineTool(\n",
|
|
|
- " index.as_query_engine(),\n",
|
|
|
- " metadata=ToolMetadata(\n",
|
|
|
- " name=\"vector_search\",\n",
|
|
|
- " description=\"Useful for searching for specific facts.\",\n",
|
|
|
- " ),\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "summary_tool = QueryEngineTool(\n",
|
|
|
- " index.as_query_engine(response_mode=\"tree_summarize\"),\n",
|
|
|
- " metadata=ToolMetadata(\n",
|
|
|
- " name=\"summary\",\n",
|
|
|
- " description=\"Useful for summarizing an entire document.\",\n",
|
|
|
- " ),\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "d063d07b-c03e-4b26-8556-e3c058d2fd52",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core.query_engine import RouterQueryEngine\n",
|
|
|
- "\n",
|
|
|
- "query_engine = RouterQueryEngine.from_defaults(\n",
|
|
|
- " [vector_tool, summary_tool], select_multi=False, verbose=True, llm=llm_70b\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "response = query_engine.query(\n",
|
|
|
- " \"Tell me about the song meet the grahams - why is it significant\"\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "396aad75-5a71-4bd9-a760-7f13fe223079",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "print(response)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "a795f0bc-e871-4580-8983-6fb27d421fc5",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 4. Text-to-SQL \n",
|
|
|
- "\n",
|
|
|
- "Here, we download and use a sample SQLite database with 11 tables, with various info about music, playlists, and customers. We will limit to a select few tables for this test."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "a5096501-92c3-41af-a871-ade869d710fb",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "!wget \"https://www.sqlitetutorial.net/wp-content/uploads/2018/03/chinook.zip\" -O \"./data/chinook.zip\"\n",
|
|
|
- "!unzip \"./data/chinook.zip\""
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "d4db989e-c18d-4416-928e-7be4ead4d869",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from sqlalchemy import (\n",
|
|
|
- " create_engine,\n",
|
|
|
- " MetaData,\n",
|
|
|
- " Table,\n",
|
|
|
- " Column,\n",
|
|
|
- " String,\n",
|
|
|
- " Integer,\n",
|
|
|
- " select,\n",
|
|
|
- " column,\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "engine = create_engine(\"sqlite:///chinook.db\")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "bf6ed233-0ea3-4d4f-8c33-5b6d558b89b9",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core import SQLDatabase\n",
|
|
|
- "\n",
|
|
|
- "sql_database = SQLDatabase(engine)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "debae423-1004-40f6-9356-e1c3add4d965",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core.indices.struct_store import NLSQLTableQueryEngine\n",
|
|
|
- "\n",
|
|
|
- "query_engine = NLSQLTableQueryEngine(\n",
|
|
|
- " sql_database=sql_database,\n",
|
|
|
- " tables=[\"albums\", \"tracks\", \"artists\"],\n",
|
|
|
- " llm=llm_70b,\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "a65ecd70-09c4-4872-b712-3a8235d03db2",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = query_engine.query(\"What are some albums?\")\n",
|
|
|
- "\n",
|
|
|
- "print(response)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "c12b93ef-d6d1-4d15-9cb2-343070f72851",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = query_engine.query(\"What are some artists? Limit it to 5.\")\n",
|
|
|
- "\n",
|
|
|
- "print(response)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "2c243d38-c6ac-445c-b9d4-53a9ae013b7b",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "This last query should be a more complex join"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "553741c2-1050-445d-979a-ae2150ee3248",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = query_engine.query(\n",
|
|
|
- " \"What are some tracks from the artist AC/DC? Limit it to 3\"\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "print(response)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "300689d7-9e67-4404-9898-27404ee6d4b5",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "print(response.metadata[\"sql_query\"])"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "1419fe67-aa6a-47db-88cd-9bb251c15615",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 5. Structured Data Extraction\n",
|
|
|
- "\n",
|
|
|
- "An important use case for function calling is extracting structured objects. LlamaIndex provides an intuitive interface for this through `structured_predict` - simply define the target Pydantic class (can be nested), and given a prompt, we extract out the desired object.\n",
|
|
|
- "\n",
|
|
|
- "**NOTE**: Since there's no native function calling support with Llama3, the structured extraction is performed by prompting the LLM + output parsing."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "4432f35a-5f29-45e9-a928-32e6d77b158e",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.llms.groq import Groq\n",
|
|
|
- "from llama_index.core.prompts import PromptTemplate\n",
|
|
|
- "from pydantic import BaseModel\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "class Restaurant(BaseModel):\n",
|
|
|
- " \"\"\"A restaurant with name, city, and cuisine.\"\"\"\n",
|
|
|
- "\n",
|
|
|
- " name: str\n",
|
|
|
- " city: str\n",
|
|
|
- " cuisine: str\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "llm = Groq(model=\"llama3-8b-8192\", pydantic_program_mode=\"llm\")\n",
|
|
|
- "prompt_tmpl = PromptTemplate(\n",
|
|
|
- " \"Generate a restaurant in a given city {city_name}\"\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "2c451f52-a051-4ba2-a683-0c1fd258d986",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "restaurant_obj = llm.structured_predict(\n",
|
|
|
- " Restaurant, prompt_tmpl, city_name=\"Miami\"\n",
|
|
|
- ")\n",
|
|
|
- "print(restaurant_obj)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "839018a9-b65f-4824-83f7-2e4e52b55c5d",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 6. Adding Chat History to RAG (Chat Engine)\n",
|
|
|
- "\n",
|
|
|
- "In this section we create a stateful chatbot from a RAG pipeline, with our chat engine abstraction.\n",
|
|
|
- "\n",
|
|
|
- "Unlike a stateless query engine, the chat engine maintains conversation history (through a memory module like buffer memory). It performs retrieval given a condensed question, and feeds the condensed question + context + chat history into the final LLM prompt.\n",
|
|
|
- "\n",
|
|
|
- "Related resource: https://docs.llamaindex.ai/en/stable/examples/chat_engine/chat_engine_condense_plus_context/"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "27e56315-9513-4b32-bf9a-ce97c3ab52df",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core.memory import ChatMemoryBuffer\n",
|
|
|
- "from llama_index.core.chat_engine import CondensePlusContextChatEngine\n",
|
|
|
- "\n",
|
|
|
- "memory = ChatMemoryBuffer.from_defaults(token_limit=3900)\n",
|
|
|
- "\n",
|
|
|
- "chat_engine = CondensePlusContextChatEngine.from_defaults(\n",
|
|
|
- " index.as_retriever(),\n",
|
|
|
- " memory=memory,\n",
|
|
|
- " llm=llm,\n",
|
|
|
- " context_prompt=(\n",
|
|
|
- " \"You are a chatbot, able to have normal interactions, as well as talk\"\n",
|
|
|
- " \" about the Kendrick and Drake beef.\"\n",
|
|
|
- " \"Here are the relevant documents for the context:\\n\"\n",
|
|
|
- " \"{context_str}\"\n",
|
|
|
- " \"\\nInstruction: Use the previous chat history, or the context above, to interact and help the user.\"\n",
|
|
|
- " ),\n",
|
|
|
- " verbose=True,\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "b24524d2-fdce-4237-8ecc-67f139302303",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = chat_engine.chat(\n",
|
|
|
- " \"Tell me about the songs Drake released in the beef.\"\n",
|
|
|
- ")\n",
|
|
|
- "print(str(response))"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "f9a87a16-2864-4c48-95e7-a2103e119242",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = chat_engine.chat(\"What about Kendrick?\")\n",
|
|
|
- "print(str(response))"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "a7fa07ed-58f0-445e-bbd3-4ad8bac6598e",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## 7. Agents\n",
|
|
|
- "\n",
|
|
|
- "Here we build agents with Llama 3. We perform RAG over simple functions as well as the documents above."
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "aa98d735-5d43-413f-aab3-fc3adeed81b1",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Agents And Tools"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "fb73a01f-8a2e-4dd6-91f8-710c92b81c56",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "import json\n",
|
|
|
- "from typing import Sequence, List\n",
|
|
|
- "\n",
|
|
|
- "from llama_index.core.llms import ChatMessage\n",
|
|
|
- "from llama_index.core.tools import BaseTool, FunctionTool\n",
|
|
|
- "from llama_index.core.agent import ReActAgent\n",
|
|
|
- "\n",
|
|
|
- "import nest_asyncio\n",
|
|
|
- "\n",
|
|
|
- "nest_asyncio.apply()"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "efbee832-9786-4551-93f2-01ee90fa0f4d",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Define Tools"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "b2058b36-8053-4dc8-9218-c286702ecf66",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "def multiply(a: int, b: int) -> int:\n",
|
|
|
- " \"\"\"Multiple two integers and returns the result integer\"\"\"\n",
|
|
|
- " return a * b\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "def add(a: int, b: int) -> int:\n",
|
|
|
- " \"\"\"Add two integers and returns the result integer\"\"\"\n",
|
|
|
- " return a + b\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "def subtract(a: int, b: int) -> int:\n",
|
|
|
- " \"\"\"Subtract two integers and returns the result integer\"\"\"\n",
|
|
|
- " return a - b\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "def divide(a: int, b: int) -> int:\n",
|
|
|
- " \"\"\"Divides two integers and returns the result integer\"\"\"\n",
|
|
|
- " return a / b\n",
|
|
|
- "\n",
|
|
|
- "\n",
|
|
|
- "multiply_tool = FunctionTool.from_defaults(fn=multiply)\n",
|
|
|
- "add_tool = FunctionTool.from_defaults(fn=add)\n",
|
|
|
- "subtract_tool = FunctionTool.from_defaults(fn=subtract)\n",
|
|
|
- "divide_tool = FunctionTool.from_defaults(fn=divide)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "22d7d4dc-e2ce-402c-9350-0e7010d0080c",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### ReAct Agent"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "72a48053-e30d-4884-bcac-80752047d940",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "agent = ReActAgent.from_tools(\n",
|
|
|
- " [multiply_tool, add_tool, subtract_tool, divide_tool],\n",
|
|
|
- " llm=llm_70b,\n",
|
|
|
- " verbose=True,\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "7ada828a-3b05-4fc1-90e8-986c5607ae61",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Querying"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "9c0b1e56-d9f7-4615-a15a-c91fea1adb00",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = agent.chat(\"What is (121 + 2) * 5?\")\n",
|
|
|
- "print(str(response))"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "67ce45f6-bdd4-42aa-8f74-43a50f14094e",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### ReAct Agent With RAG QueryEngine Tools"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "97fce5f1-eacf-4ecc-9e83-072e74d3a2a9",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "from llama_index.core import (\n",
|
|
|
- " SimpleDirectoryReader,\n",
|
|
|
- " VectorStoreIndex,\n",
|
|
|
- " StorageContext,\n",
|
|
|
- " load_index_from_storage,\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "from llama_index.core.tools import QueryEngineTool, ToolMetadata"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "23963d00-e3d2-4ce1-9ac3-aa486bf4b1a5",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Create ReAct Agent using RAG QueryEngine Tools"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "1844dbbd-477c-4c4d-bb18-2c2e16a75a50",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "This may take 4 minutes to run:"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "66ab1e60-3374-4eb9-b7dc-c28db3b47c51",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "drake_index = VectorStoreIndex.from_documents(docs_drake)\n",
|
|
|
- "drake_query_engine = drake_index.as_query_engine(similarity_top_k=3)\n",
|
|
|
- "\n",
|
|
|
- "kendrick_index = VectorStoreIndex.from_documents(docs_kendrick)\n",
|
|
|
- "kendrick_query_engine = kendrick_index.as_query_engine(similarity_top_k=3)"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "0e241fe9-f390-4be5-b3c4-da4f56db01ef",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "drake_tool = QueryEngineTool(\n",
|
|
|
- " drake_index.as_query_engine(),\n",
|
|
|
- " metadata=ToolMetadata(\n",
|
|
|
- " name=\"drake_search\",\n",
|
|
|
- " description=\"Useful for searching over Drake's life.\",\n",
|
|
|
- " ),\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "kendrick_tool = QueryEngineTool(\n",
|
|
|
- " kendrick_index.as_query_engine(),\n",
|
|
|
- " metadata=ToolMetadata(\n",
|
|
|
- " name=\"kendrick_search\",\n",
|
|
|
- " description=\"Useful for searching over Kendrick's life.\",\n",
|
|
|
- " ),\n",
|
|
|
- ")\n",
|
|
|
- "\n",
|
|
|
- "query_engine_tools = [drake_tool, kendrick_tool]"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "b922feac-b221-4737-92c6-e63eeab4eab7",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "agent = ReActAgent.from_tools(\n",
|
|
|
- " query_engine_tools,\n",
|
|
|
- " llm=llm_70b,\n",
|
|
|
- " verbose=True,\n",
|
|
|
- ")"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "id": "7e38edc8-47f8-4f1a-ad87-bc3a9e31a65e",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "### Querying"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "id": "035c2c8b-5a5e-4df0-a423-4c2d6054f457",
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": [
|
|
|
- "response = agent.chat(\"Tell me about how Kendrick and Drake grew up\")\n",
|
|
|
- "print(str(response))"
|
|
|
- ]
|
|
|
- }
|
|
|
- ],
|
|
|
- "metadata": {
|
|
|
- "kernelspec": {
|
|
|
- "display_name": "Python 3 (ipykernel)",
|
|
|
- "language": "python",
|
|
|
- "name": "python3"
|
|
|
- },
|
|
|
- "language_info": {
|
|
|
- "codemirror_mode": {
|
|
|
- "name": "ipython",
|
|
|
- "version": 3
|
|
|
- },
|
|
|
- "file_extension": ".py",
|
|
|
- "mimetype": "text/x-python",
|
|
|
- "name": "python",
|
|
|
- "nbconvert_exporter": "python",
|
|
|
- "pygments_lexer": "ipython3",
|
|
|
- "version": "3.10.14"
|
|
|
- }
|
|
|
- },
|
|
|
- "nbformat": 4,
|
|
|
- "nbformat_minor": 5
|
|
|
-}
|
Hamid Shojanazeri