{ "cells": [ { "cell_type": "markdown", "id": "4e11912c", "metadata": {}, "source": [ "# Function Calling 101: An eCommerce Use Case" ] }, { "cell_type": "markdown", "id": "75a04a76", "metadata": {}, "source": [ "## 1. Introduction to Function Calling" ] }, { "cell_type": "markdown", "id": "64a7d176", "metadata": {}, "source": [ "### 1a. What is function calling and why is it important?" ] }, { "cell_type": "markdown", "id": "84ab38d9", "metadata": {}, "source": [ "Function calling (or tool use) in the context of large language models (LLMs) is the process of an LLM invoking a pre-defined function instead of generating a text response. LLMs are **non-deterministic**, offering flexibility and creativity, but this can lead to inconsistencies and occasional hallucinations, with the training data often being outdated. In contrast, traditional software is **deterministic**, executing tasks precisely as programmed but lacking adaptability. Function calling with LLMs aims to combine the best of both worlds: leveraging the flexibility and creativity of LLMs while ensuring consistent, repeatable actions and reducing hallucinations by utilizing pre-defined functions." ] }, { "cell_type": "markdown", "id": "5384f37a", "metadata": {}, "source": [ "### 1b. What is it doing?" ] }, { "cell_type": "markdown", "id": "c51bdac3", "metadata": {}, "source": [ "Function calling essentially arms your LLM with custom tools to perform specific tasks that a generic LLM might struggle with. During an interaction, the LLM determines which tool to call and what parameters to use, allowing it to execute actions it otherwise couldn’t. This enables the LLM to either perform an action directly or relay the function’s output back to itself, providing more context for a follow-up chat completion. By integrating these custom tools, function calling enhances the LLM’s capabilities and precision, enabling more complex and accurate responses." ] }, { "cell_type": "markdown", "id": "434d4ed5", "metadata": {}, "source": [ "### 1c. What are some use cases?" ] }, { "cell_type": "markdown", "id": "57b41881", "metadata": {}, "source": [ "Function calling with LLMs can be applied to a variety of practical scenarios, significantly enhancing the capabilities of LLMs. Here are some organized and expanded use cases:\n", "\n", "\n", "**1. Real-Time Information Retrieval:** LLMs can use function calling to access up-to-date information by querying APIs, databases or search tools, like the [Yahoo Finance API](https://finance.yahoo.com/) or [Tavily Search API](https://tavily.com/). This is particularly useful in domains where information changes frequently, or when you want to surface internal data to the user.\n", "\n", "**2. Mathematical Calculations:** LLMs often face challenges with precise mathematical computations. By leveraging function calling, these calculations can be offloaded to specialized functions, ensuring accuracy and reliability.\n", "\n", "**3. API Integration for Enhanced Functionality:** Function calling can significantly expand the capabilities of an LLM by integrating it with various APIs. This allows the LLM to perform tasks such as booking appointments, managing calendars, handling customer service requests, and more. By leveraging specific APIs, the LLM can process detailed parameters like appointment times, customer names, contact information, and service details, ensuring efficient and accurate task execution." ] }, { "cell_type": "markdown", "id": "23825e9a", "metadata": {}, "source": [ "## 2. Function Calling Implementation with Groq: eCommerce Use Case" ] }, { "cell_type": "markdown", "id": "cdac7b10", "metadata": {}, "source": [ "In this notebook, we'll use show how function calling can be used for an eCommerce use case, where our LLM will take on the role of a helpful customer service representative, able to use tools to create orders and get prices on products. We will be interacting as a customer named Tom Testuser." ] }, { "cell_type": "markdown", "id": "03f13180", "metadata": {}, "source": [ "We will be using [Airtable](https://airtable.com/) as our backend database for this demo, and will use the Airtable API to read and write from `customers`, `products` and `orders` tables. You can view the Airtable base [here](https://airtable.com/appQZ9KdhmjcDVSGx/shrlg9MAetUslmX2Z), but will need to copy it into your own Airtable base (click “copy base” in the upper banner) in order to fully follow along with this guide and build on top of it.\n" ] }, { "cell_type": "markdown", "id": "63aadc9e", "metadata": {}, "source": [ "### 2a. Setup" ] }, { "cell_type": "markdown", "id": "d5af0b86", "metadata": {}, "source": [ "We will be using Meta's Llama 3-70B model for this demo. Note that you will need a Groq API Key to proceed and can create an account [here](https://console.groq.com/) to generate one for free.\n", "\n", "You will also need to create an Airtable account and provision an [Airtable Personal Access Token](https://airtable.com/create/tokens) with `data.record:read` and `data.record:write` scopes. The Airtable Base ID will be in the URL of the base you copy from above.\n", "\n", "Finally, our System Message will provide relevant context to the LLM: that it is a customer service assistant for an ecommerce company, and that it is interacting with a customer named Tom Testuser (ID: 10)." ] }, { "cell_type": "code", "execution_count": 4, "id": "32d7cdcd", "metadata": {}, "outputs": [], "source": [ "# Setup\n", "import json\n", "import os\n", "import random\n", "import urllib.parse\n", "from datetime import datetime\n", "\n", "import requests\n", "from groq import Groq\n", "\n", "# Initialize Groq client and model\n", "client = Groq(api_key=os.getenv(\"GROQ_API_KEY\"))\n", "MODEL = \"llama3-70b-8192\"\n", "\n", "# Airtable variables\n", "airtable_api_token = os.environ[\"AIRTABLE_API_TOKEN\"]\n", "airtable_base_id = os.environ[\"AIRTABLE_BASE_ID\"]" ] }, { "cell_type": "code", "execution_count": 5, "id": "0db27033", "metadata": {}, "outputs": [], "source": [ "SYSTEM_MESSAGE = \"\"\"\n", "You are a helpful customer service LLM for an ecommerce company that processes orders and retrieves information about products.\n", "You are currently chatting with Tom Testuser, Customer ID: 10\n", "\"\"\"" ] }, { "cell_type": "markdown", "id": "c44f7c50-8cd7-43fd-9868-c7b2306a30d7", "metadata": {}, "source": [ "### 2b. Tool Creation" ] }, { "cell_type": "markdown", "id": "53ca84b3-f5d6-4a4e-9a95-7b26dd61a524", "metadata": {}, "source": [ "First we must define the functions (tools) that the LLM will have access to. For our use case, we will use the Airtable API to create an order (POST request to the orders table), get product prices (GET request to the products table) and get product ID (GET request to the products table).\n", "\n", "We will then compile these tools in a list that can be passed to the LLM. Note that we must provide proper descriptions of the functions and parameters so that they can be called appropriately given the user input:" ] }, { "cell_type": "code", "execution_count": 6, "id": "64e18dfc", "metadata": {}, "outputs": [], "source": [ "# Creates an order given a product_id and customer_id\n", "def create_order(product_id, customer_id):\n", " headers = {\n", " \"Authorization\": f\"Bearer {airtable_api_token}\",\n", " \"Content-Type\": \"application/json\",\n", " }\n", " url = f\"https://api.airtable.com/v0/{airtable_base_id}/orders\"\n", " order_id = random.randint(1, 100000) # Randomly assign an order_id\n", " order_datetime = datetime.utcnow().strftime(\n", " \"%Y-%m-%dT%H:%M:%SZ\"\n", " ) # Assign order date as now\n", " data = {\n", " \"fields\": {\n", " \"order_id\": order_id,\n", " \"product_id\": product_id,\n", " \"customer_id\": customer_id,\n", " \"order_date\": order_datetime,\n", " }\n", " }\n", " response = requests.post(url, headers=headers, json=data)\n", " return str(response.json())\n", "\n", "\n", "# Gets the price for a product, given the name of the product\n", "def get_product_price(product_name):\n", " api_token = os.environ[\"AIRTABLE_API_TOKEN\"]\n", " base_id = os.environ[\"AIRTABLE_BASE_ID\"]\n", " headers = {\"Authorization\": f\"Bearer {airtable_api_token}\"}\n", " formula = f\"{{name}}='{product_name}'\"\n", " encoded_formula = urllib.parse.quote(formula)\n", " url = f\"https://api.airtable.com/v0/{airtable_base_id}/products?filterByFormula={encoded_formula}\"\n", " response = requests.get(url, headers=headers)\n", " product_price = response.json()[\"records\"][0][\"fields\"][\"price\"]\n", " return \"$\" + str(product_price)\n", "\n", "\n", "# Gets product ID given a product name\n", "def get_product_id(product_name):\n", " api_token = os.environ[\"AIRTABLE_API_TOKEN\"]\n", " base_id = os.environ[\"AIRTABLE_BASE_ID\"]\n", " headers = {\"Authorization\": f\"Bearer {airtable_api_token}\"}\n", " formula = f\"{{name}}='{product_name}'\"\n", " encoded_formula = urllib.parse.quote(formula)\n", " url = f\"https://api.airtable.com/v0/{airtable_base_id}/products?filterByFormula={encoded_formula}\"\n", " response = requests.get(url, headers=headers)\n", " product_id = response.json()[\"records\"][0][\"fields\"][\"product_id\"]\n", " return str(product_id)" ] }, { "cell_type": "markdown", "id": "51a7a120", "metadata": {}, "source": [ "The necessary structure to compile our list of tools so that the LLM can use them; note that we must provide proper descriptions of the functions and parameters so that they can be called appropriately given the user input:" ] }, { "cell_type": "code", "execution_count": 7, "id": "b5a12541", "metadata": {}, "outputs": [], "source": [ "tools = [\n", " # First function: create_order\n", " {\n", " \"type\": \"function\",\n", " \"function\": {\n", " \"name\": \"create_order\",\n", " \"description\": \"Creates an order given a product_id and customer_id. If a product name is provided, you must get the product ID first. After placing the order indicate that it was placed successfully and output the details.\",\n", " \"parameters\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"product_id\": {\n", " \"type\": \"integer\",\n", " \"description\": \"The ID of the product\",\n", " },\n", " \"customer_id\": {\n", " \"type\": \"integer\",\n", " \"description\": \"The ID of the customer\",\n", " },\n", " },\n", " \"required\": [\"product_id\", \"customer_id\"],\n", " },\n", " },\n", " },\n", " # Second function: get_product_price\n", " {\n", " \"type\": \"function\",\n", " \"function\": {\n", " \"name\": \"get_product_price\",\n", " \"description\": \"Gets the price for a product, given the name of the product. Just return the price, do not do any calculations.\",\n", " \"parameters\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"product_name\": {\n", " \"type\": \"string\",\n", " \"description\": \"The name of the product (must be title case, i.e. 'Microphone', 'Laptop')\",\n", " }\n", " },\n", " \"required\": [\"product_name\"],\n", " },\n", " },\n", " },\n", " # Third function: get_product_id\n", " {\n", " \"type\": \"function\",\n", " \"function\": {\n", " \"name\": \"get_product_id\",\n", " \"description\": \"Gets product ID given a product name\",\n", " \"parameters\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"product_name\": {\n", " \"type\": \"string\",\n", " \"description\": \"The name of the product (must be title case, i.e. 'Microphone', 'Laptop')\",\n", " }\n", " },\n", " \"required\": [\"product_name\"],\n", " },\n", " },\n", " },\n", "]" ] }, { "cell_type": "markdown", "id": "cf6325f3", "metadata": {}, "source": [ "### 2c. Simple Function Calling" ] }, { "cell_type": "markdown", "id": "3b1dd8ba", "metadata": {}, "source": [ "First, let's start out by just making a simple function call with only one tool. We will ask the customer service LLM to place an order for a product with Product ID 5." ] }, { "cell_type": "markdown", "id": "92c77018", "metadata": {}, "source": [ "The two key parameters we need to include in our chat completion are `tools=tools` and `tool_choice=\"auto\"`, which provides the model with the available tools we've just defined and tells it to use one if appropriate (`tool_choice=\"auto\"` gives the LLM the option of using any, all or none of the available functions. To mandate a specific function call, we could use `tool_choice={\"type\": \"function\", \"function\": {\"name\":\"create_order\"}}`). \n", "\n", "When the LLM decides to use a tool, the response is *not* a conversational chat, but a JSON object containing the tool choice and tool parameters. From there, we can execute the LLM-identified tool with the LLM-identified parameters, and feed the response *back* to the LLM for a second request so that it can respond with appropriate context from the tool it just used:" ] }, { "cell_type": "code", "execution_count": 8, "id": "482b2251", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "First LLM Call (Tool Use) Response: ChoiceMessage(content=None, role='assistant', tool_calls=[ChoiceMessageToolCall(id='call_cnyc', function=ChoiceMessageToolCallFunction(arguments='{\"customer_id\":10,\"product_id\":5}', name='create_order'), type='function')])\n", "\n", "\n", "Second LLM Call Response: Your order has been successfully placed!\n", "\n", "Order details:\n", "\n", "* Order ID: 24255\n", "* Product ID: 5\n", "* Customer ID: 10 (that's you, Tom Testuser!)\n", "* Order Date: 2024-05-31 13:59:03\n", "\n", "We'll process your order shortly. You'll receive an email with further updates on your order status. If you have any questions or concerns, feel free to ask!\n" ] } ], "source": [ "user_prompt = \"Please place an order for Product ID 5\"\n", "messages = [\n", " {\"role\": \"system\", \"content\": SYSTEM_MESSAGE},\n", " {\n", " \"role\": \"user\",\n", " \"content\": user_prompt,\n", " },\n", "]\n", "\n", "# Step 1: send the conversation and available functions to the model\n", "response = client.chat.completions.create(\n", " model=MODEL,\n", " messages=messages,\n", " tools=tools,\n", " tool_choice=\"auto\", # Let the LLM decide if it should use one of the available tools\n", " max_tokens=4096,\n", ")\n", "\n", "response_message = response.choices[0].message\n", "tool_calls = response_message.tool_calls\n", "print(\"First LLM Call (Tool Use) Response:\", response_message)\n", "# Step 2: check if the model wanted to call a function\n", "if tool_calls:\n", " # Step 3: call the function and append the tool call to our list of messages\n", " available_functions = {\n", " \"create_order\": create_order,\n", " }\n", " messages.append(\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": tool_call.id,\n", " \"function\": {\n", " \"name\": tool_call.function.name,\n", " \"arguments\": tool_call.function.arguments,\n", " },\n", " \"type\": tool_call.type,\n", " }\n", " for tool_call in tool_calls\n", " ],\n", " }\n", " )\n", " # Step 4: send the info for each function call and function response to the model\n", " tool_call = tool_calls[0]\n", " function_name = tool_call.function.name\n", " function_to_call = available_functions[function_name]\n", " function_args = json.loads(tool_call.function.arguments)\n", " function_response = function_to_call(\n", " product_id=function_args.get(\"product_id\"),\n", " customer_id=function_args.get(\"customer_id\"),\n", " )\n", " messages.append(\n", " {\n", " \"tool_call_id\": tool_call.id,\n", " \"role\": \"tool\",\n", " \"name\": function_name,\n", " \"content\": function_response,\n", " }\n", " ) # extend conversation with function response\n", " # Send the result back to the LLM to complete the chat\n", " second_response = client.chat.completions.create(\n", " model=MODEL, messages=messages\n", " ) # get a new response from the model where it can see the function response\n", " print(\"\\n\\nSecond LLM Call Response:\", second_response.choices[0].message.content)" ] }, { "cell_type": "markdown", "id": "cb60a037", "metadata": {}, "source": [ "Here is the entire message sequence for a simple tool call:" ] }, { "cell_type": "code", "execution_count": 9, "id": "fce83d48", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\n", " {\n", " \"role\": \"system\",\n", " \"content\": \"\\nYou are a helpful customer service LLM for an ecommerce company that processes orders and retrieves information about products.\\nYou are currently chatting with Tom Testuser, Customer ID: 10\\n\"\n", " },\n", " {\n", " \"role\": \"user\",\n", " \"content\": \"Please place an order for Product ID 5\"\n", " },\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": \"call_cnyc\",\n", " \"function\": {\n", " \"name\": \"create_order\",\n", " \"arguments\": \"{\\\"customer_id\\\":10,\\\"product_id\\\":5}\"\n", " },\n", " \"type\": \"function\"\n", " }\n", " ]\n", " },\n", " {\n", " \"tool_call_id\": \"call_cnyc\",\n", " \"role\": \"tool\",\n", " \"name\": \"create_order\",\n", " \"content\": \"{'id': 'recWasb2AECLJiRj1', 'createdTime': '2024-05-31T13:59:04.000Z', 'fields': {'order_id': 24255, 'product_id': 5, 'customer_id': 10, 'order_date': '2024-05-31T13:59:03.000Z'}}\"\n", " }\n", "]\n" ] } ], "source": [ "print(json.dumps(messages, indent=2))" ] }, { "cell_type": "markdown", "id": "513fff34", "metadata": {}, "source": [ "### 2d. Parallel Tool Use" ] }, { "cell_type": "markdown", "id": "b50964e8", "metadata": {}, "source": [ "If we need multiple function calls that **do not** depend on each other, we can run them in parallel - meaning, multiple function calls will be identified within a single chat request. Here, we are asking for the price of both a Laptop and a Microphone, which requires multiple calls of the `get_product_price` function. Note that in using parallel tool use, *the LLM itself* will decide if it needs to make multiple function calls. So we don't need to make any changes to our chat completion code, but *do* need to be able to iterate over multiple tool calls after the tools are identified." ] }, { "cell_type": "markdown", "id": "9e0f5a0e", "metadata": {}, "source": [ "*parallel tool use is only available for Llama-based models at this time (5/27/2024)*" ] }, { "cell_type": "code", "execution_count": 10, "id": "5ec93e21", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "First LLM Call (Tool Use) Response: ChoiceMessage(content=None, role='assistant', tool_calls=[ChoiceMessageToolCall(id='call_88r0', function=ChoiceMessageToolCallFunction(arguments='{\"product_name\":\"Laptop\"}', name='get_product_price'), type='function'), ChoiceMessageToolCall(id='call_vva6', function=ChoiceMessageToolCallFunction(arguments='{\"product_name\":\"Microphone\"}', name='get_product_price'), type='function')])\n", "\n", "\n", "Second LLM Call Response: So, the price of the Laptop is $753.03 and the price of the Microphone is $276.23. The total comes out to be $1,029.26.\n" ] } ], "source": [ "user_prompt = \"Please get the price for the Laptop and Microphone\"\n", "messages = [\n", " {\"role\": \"system\", \"content\": SYSTEM_MESSAGE},\n", " {\n", " \"role\": \"user\",\n", " \"content\": user_prompt,\n", " },\n", "]\n", "\n", "# Step 1: send the conversation and available functions to the model\n", "response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=tools, tool_choice=\"auto\", max_tokens=4096\n", ")\n", "\n", "response_message = response.choices[0].message\n", "tool_calls = response_message.tool_calls\n", "print(\"First LLM Call (Tool Use) Response:\", response_message)\n", "# Step 2: check if the model wanted to call a function\n", "if tool_calls:\n", " # Step 3: call the function and append the tool call to our list of messages\n", " available_functions = {\n", " \"get_product_price\": get_product_price,\n", " } # only one function in this example, but you can have multiple\n", " messages.append(\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": tool_call.id,\n", " \"function\": {\n", " \"name\": tool_call.function.name,\n", " \"arguments\": tool_call.function.arguments,\n", " },\n", " \"type\": tool_call.type,\n", " }\n", " for tool_call in tool_calls\n", " ],\n", " }\n", " )\n", " # Step 4: send the info for each function call and function response to the model\n", " # Iterate over all tool calls\n", " for tool_call in tool_calls:\n", " function_name = tool_call.function.name\n", " function_to_call = available_functions[function_name]\n", " function_args = json.loads(tool_call.function.arguments)\n", " function_response = function_to_call(\n", " product_name=function_args.get(\"product_name\")\n", " )\n", " messages.append(\n", " {\n", " \"tool_call_id\": tool_call.id,\n", " \"role\": \"tool\",\n", " \"name\": function_name,\n", " \"content\": function_response,\n", " }\n", " ) # extend conversation with function response\n", " second_response = client.chat.completions.create(\n", " model=MODEL, messages=messages\n", " ) # get a new response from the model where it can see the function response\n", " print(\"\\n\\nSecond LLM Call Response:\", second_response.choices[0].message.content)" ] }, { "cell_type": "markdown", "id": "90082fd7", "metadata": {}, "source": [ "Here is the entire message sequence for a parallel tool call:" ] }, { "cell_type": "code", "execution_count": 11, "id": "50d953b7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\n", " {\n", " \"role\": \"system\",\n", " \"content\": \"\\nYou are a helpful customer service LLM for an ecommerce company that processes orders and retrieves information about products.\\nYou are currently chatting with Tom Testuser, Customer ID: 10\\n\"\n", " },\n", " {\n", " \"role\": \"user\",\n", " \"content\": \"Please get the price for the Laptop and Microphone\"\n", " },\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": \"call_88r0\",\n", " \"function\": {\n", " \"name\": \"get_product_price\",\n", " \"arguments\": \"{\\\"product_name\\\":\\\"Laptop\\\"}\"\n", " },\n", " \"type\": \"function\"\n", " },\n", " {\n", " \"id\": \"call_vva6\",\n", " \"function\": {\n", " \"name\": \"get_product_price\",\n", " \"arguments\": \"{\\\"product_name\\\":\\\"Microphone\\\"}\"\n", " },\n", " \"type\": \"function\"\n", " }\n", " ]\n", " },\n", " {\n", " \"tool_call_id\": \"call_88r0\",\n", " \"role\": \"tool\",\n", " \"name\": \"get_product_price\",\n", " \"content\": \"$753.03\"\n", " },\n", " {\n", " \"tool_call_id\": \"call_vva6\",\n", " \"role\": \"tool\",\n", " \"name\": \"get_product_price\",\n", " \"content\": \"$276.23\"\n", " }\n", "]\n" ] } ], "source": [ "print(json.dumps(messages, indent=2))" ] }, { "cell_type": "markdown", "id": "53959911", "metadata": {}, "source": [ "### 2e. Multiple Tool Use" ] }, { "cell_type": "markdown", "id": "1d6f5a39", "metadata": {}, "source": [ "Multiple Tool Use is for when we need to use multiple functions where the input to one of the functions **depends on the output** of another function. Unlike parallel tool use, with multiple tool use we will only output a single tool call per LLM request, and then make a separate LLM request to call the next tool. To do this, we'll add a WHILE loop to continuously send LLM requests with our updated message sequence until it has enough information to no longer need to call any more tools. (Note that this solution is generalizable to both simple and parallel tool calling as well)." ] }, { "cell_type": "markdown", "id": "946576e9", "metadata": {}, "source": [ "In our first example we invoked the `create_order` function by providing the product ID directly; since that is a bit clunky, we will first use the `get_product_id` function to get the product ID associated with the product name, then use that ID to call `create_order`:" ] }, { "cell_type": "code", "execution_count": 13, "id": "6ea17b01", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LLM Call (Tool Use) Response: ChoiceMessage(content=None, role='assistant', tool_calls=[ChoiceMessageToolCall(id='call_6yd2', function=ChoiceMessageToolCallFunction(arguments='{\"product_name\":\"Microphone\"}', name='get_product_id'), type='function')])\n", "LLM Call (Tool Use) Response: ChoiceMessage(content=None, role='assistant', tool_calls=[ChoiceMessageToolCall(id='call_mnv6', function=ChoiceMessageToolCallFunction(arguments='{\"customer_id\":10,\"product_id\":15}', name='create_order'), type='function')])\n", "\n", "\n", "Final LLM Call Response: Your order with ID 42351 has been successfully placed! The details are: product ID 15, customer ID 10, and order date 2024-05-31T13:59:40.000Z.\n" ] } ], "source": [ "user_prompt = \"Please place an order for a Microphone\"\n", "messages = [\n", " {\"role\": \"system\", \"content\": SYSTEM_MESSAGE},\n", " {\n", " \"role\": \"user\",\n", " \"content\": user_prompt,\n", " },\n", "]\n", "# Continue to make LLM calls until it no longer decides to use a tool\n", "tool_call_identified = True\n", "while tool_call_identified:\n", " response = client.chat.completions.create(\n", " model=MODEL, messages=messages, tools=tools, tool_choice=\"auto\", max_tokens=4096\n", " )\n", " response_message = response.choices[0].message\n", " tool_calls = response_message.tool_calls\n", " # Step 2: check if the model wanted to call a function\n", " if tool_calls:\n", " print(\"LLM Call (Tool Use) Response:\", response_message)\n", " # Step 3: call the function and append the tool call to our list of messages\n", " available_functions = {\n", " \"create_order\": create_order,\n", " \"get_product_id\": get_product_id,\n", " }\n", " messages.append(\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": tool_call.id,\n", " \"function\": {\n", " \"name\": tool_call.function.name,\n", " \"arguments\": tool_call.function.arguments,\n", " },\n", " \"type\": tool_call.type,\n", " }\n", " for tool_call in tool_calls\n", " ],\n", " }\n", " )\n", "\n", " # Step 4: send the info for each function call and function response to the model\n", " for tool_call in tool_calls:\n", " function_name = tool_call.function.name\n", " function_to_call = available_functions[function_name]\n", " function_args = json.loads(tool_call.function.arguments)\n", " if function_name == \"get_product_id\":\n", " function_response = function_to_call(\n", " product_name=function_args.get(\"product_name\")\n", " )\n", " elif function_name == \"create_order\":\n", " function_response = function_to_call(\n", " customer_id=function_args.get(\"customer_id\"),\n", " product_id=function_args.get(\"product_id\"),\n", " )\n", " messages.append(\n", " {\n", " \"tool_call_id\": tool_call.id,\n", " \"role\": \"tool\",\n", " \"name\": function_name,\n", " \"content\": function_response,\n", " }\n", " ) # extend conversation with function response\n", " else:\n", " print(\"\\n\\nFinal LLM Call Response:\", response.choices[0].message.content)\n", " tool_call_identified = False" ] }, { "cell_type": "markdown", "id": "865b15f0", "metadata": {}, "source": [ "Here is the entire message sequence for a multiple tool call:" ] }, { "cell_type": "code", "execution_count": 14, "id": "bda72263", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\n", " {\n", " \"role\": \"system\",\n", " \"content\": \"\\nYou are a helpful customer service LLM for an ecommerce company that processes orders and retrieves information about products.\\nYou are currently chatting with Tom Testuser, Customer ID: 10\\n\"\n", " },\n", " {\n", " \"role\": \"user\",\n", " \"content\": \"Please place an order for a Microphone\"\n", " },\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": \"call_6yd2\",\n", " \"function\": {\n", " \"name\": \"get_product_id\",\n", " \"arguments\": \"{\\\"product_name\\\":\\\"Microphone\\\"}\"\n", " },\n", " \"type\": \"function\"\n", " }\n", " ]\n", " },\n", " {\n", " \"tool_call_id\": \"call_6yd2\",\n", " \"role\": \"tool\",\n", " \"name\": \"get_product_id\",\n", " \"content\": \"15\"\n", " },\n", " {\n", " \"role\": \"assistant\",\n", " \"tool_calls\": [\n", " {\n", " \"id\": \"call_mnv6\",\n", " \"function\": {\n", " \"name\": \"create_order\",\n", " \"arguments\": \"{\\\"customer_id\\\":10,\\\"product_id\\\":15}\"\n", " },\n", " \"type\": \"function\"\n", " }\n", " ]\n", " },\n", " {\n", " \"tool_call_id\": \"call_mnv6\",\n", " \"role\": \"tool\",\n", " \"name\": \"create_order\",\n", " \"content\": \"{'id': 'rectr27e5TP1UMREM', 'createdTime': '2024-05-31T13:59:41.000Z', 'fields': {'order_id': 42351, 'product_id': 15, 'customer_id': 10, 'order_date': '2024-05-31T13:59:40.000Z'}}\"\n", " }\n", "]\n" ] } ], "source": [ "print(json.dumps(messages, indent=2))" ] }, { "cell_type": "markdown", "id": "159b38ec", "metadata": {}, "source": [ "### 2f. Langchain Integration" ] }, { "cell_type": "markdown", "id": "899ceec7", "metadata": {}, "source": [ "Finally, Groq function calling is compatible with [Langchain](https://python.langchain.com/v0.1/docs/modules/tools/), by converting your functions into Langchain tools. Here is an example using our `get_product_price` function:" ] }, { "cell_type": "code", "execution_count": 15, "id": "4f38cece", "metadata": {}, "outputs": [], "source": [ "from langchain_groq import ChatGroq\n", "\n", "llm = ChatGroq(groq_api_key=os.getenv(\"GROQ_API_KEY\"), model=MODEL)" ] }, { "cell_type": "markdown", "id": "84f9d041-a00c-4f03-a8d4-2d1e63f132c2", "metadata": {}, "source": [ "When defining Langchain tools, put the function description as a string at the beginning of the function" ] }, { "cell_type": "code", "execution_count": 16, "id": "9c52872c", "metadata": {}, "outputs": [], "source": [ "from langchain_core.tools import tool\n", "\n", "@tool\n", "def create_order(product_id, customer_id):\n", " \"\"\"\n", " Creates an order given a product_id and customer_id.\n", " If a product name is provided, you must get the product ID first.\n", " After placing the order indicate that it was placed successfully and output the details.\n", "\n", " product_id: ID of the product\n", " customer_id: ID of the customer\n", " \"\"\"\n", " api_token = os.environ[\"AIRTABLE_API_TOKEN\"]\n", " base_id = os.environ[\"AIRTABLE_BASE_ID\"]\n", " headers = {\n", " \"Authorization\": f\"Bearer {api_token}\",\n", " \"Content-Type\": \"application/json\",\n", " }\n", " url = f\"https://api.airtable.com/v0/{base_id}/orders\"\n", " order_id = random.randint(1, 100000) # Randomly assign an order_id\n", " order_datetime = datetime.utcnow().strftime(\n", " \"%Y-%m-%dT%H:%M:%SZ\"\n", " ) # Assign order date as now\n", " data = {\n", " \"fields\": {\n", " \"order_id\": order_id,\n", " \"product_id\": product_id,\n", " \"customer_id\": customer_id,\n", " \"order_date\": order_datetime,\n", " }\n", " }\n", " response = requests.post(url, headers=headers, json=data)\n", " return str(response.json())\n", "\n", "\n", "@tool\n", "def get_product_price(product_name):\n", " \"\"\"\n", " Gets the price for a product, given the name of the product.\n", " Just return the price, do not do any calculations.\n", "\n", " product_name: The name of the product (must be title case, i.e. 'Microphone', 'Laptop')\n", " \"\"\"\n", " api_token = os.environ[\"AIRTABLE_API_TOKEN\"]\n", " base_id = os.environ[\"AIRTABLE_BASE_ID\"]\n", " headers = {\"Authorization\": f\"Bearer {api_token}\"}\n", " formula = f\"{{name}}='{product_name}'\"\n", " encoded_formula = urllib.parse.quote(formula)\n", " url = f\"https://api.airtable.com/v0/{base_id}/products?filterByFormula={encoded_formula}\"\n", " response = requests.get(url, headers=headers)\n", " product_price = response.json()[\"records\"][0][\"fields\"][\"price\"]\n", " return \"$\" + str(product_price)\n", "\n", "\n", "@tool\n", "def get_product_id(product_name):\n", " \"\"\"\n", " Gets product ID given a product name\n", "\n", " product_name: The name of the product (must be title case, i.e. 'Microphone', 'Laptop')\n", " \"\"\"\n", " api_token = os.environ[\"AIRTABLE_API_TOKEN\"]\n", " base_id = os.environ[\"AIRTABLE_BASE_ID\"]\n", " headers = {\"Authorization\": f\"Bearer {api_token}\"}\n", " formula = f\"{{name}}='{product_name}'\"\n", " encoded_formula = urllib.parse.quote(formula)\n", " url = f\"https://api.airtable.com/v0/{base_id}/products?filterByFormula={encoded_formula}\"\n", " response = requests.get(url, headers=headers)\n", " product_id = response.json()[\"records\"][0][\"fields\"][\"product_id\"]\n", " return str(product_id)\n", "\n", "\n", "# Add tools to our LLM\n", "tools = [create_order, get_product_price, get_product_id]\n", "llm_with_tools = llm.bind_tools(tools)\n" ] }, { "cell_type": "code", "execution_count": 17, "id": "968145b2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[{'name': 'get_product_id', 'args': {'product_name': 'Microphone'}, 'id': 'call_7f8y'}, {'name': 'create_order', 'args': {'product_id': '{result of get_product_id}', 'customer_id': ''}, 'id': 'call_zt5c'}]\n" ] } ], "source": [ "from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage\n", "\n", "user_prompt = \"Please place an order for a Microphone\"\n", "print(llm_with_tools.invoke(user_prompt).tool_calls)" ] }, { "cell_type": "code", "execution_count": 18, "id": "d245e8ac", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Your order has been placed successfully! Your order ID is 87812.\n" ] } ], "source": [ "from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage\n", "\n", "available_tools = {\n", " \"create_order\": create_order,\n", " \"get_product_price\": get_product_price,\n", " \"get_product_id\": get_product_id,\n", "}\n", "messages = [SystemMessage(SYSTEM_MESSAGE), HumanMessage(user_prompt)]\n", "tool_call_identified = True\n", "while tool_call_identified:\n", " ai_msg = llm_with_tools.invoke(messages)\n", " messages.append(ai_msg)\n", " for tool_call in ai_msg.tool_calls:\n", " selected_tool = available_tools[tool_call[\"name\"]]\n", " tool_output = selected_tool.invoke(tool_call[\"args\"])\n", " messages.append(ToolMessage(tool_output, tool_call_id=tool_call[\"id\"]))\n", " if len(ai_msg.tool_calls) == 0:\n", " tool_call_identified = False\n", "\n", "print(ai_msg.content)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.3" } }, "nbformat": 4, "nbformat_minor": 5 }