{ "cells": [ { "cell_type": "markdown", "id": "dbfff3b2-fbfd-4ba5-929c-c45d028801f3", "metadata": {}, "source": [ "# Llama LLM API Client Notebook" ] }, { "cell_type": "markdown", "id": "6fcc5513-4c08-42e5-9b96-20b37482d1ea", "metadata": {}, "source": [ "## This Notebook will send a request to the Llama LLM model's API and retrieve the response." ] }, { "cell_type": "markdown", "id": "5f98a3c4-81a6-44be-987a-cd4f5adcef77", "metadata": {}, "source": [ "### First we will import os, gradio and the LlamaAPIClient" ] }, { "cell_type": "code", "execution_count": 1, "id": "4c2a01b9-23d3-4f0e-9c1f-92b2b2b37231", "metadata": {}, "outputs": [], "source": [ "import os\n", "import gradio as gr\n", "from llama_api_client import LlamaAPIClient" ] }, { "cell_type": "markdown", "id": "e3e17aea-0422-4ad4-8ada-94e56e5b0fcf", "metadata": {}, "source": [ "# Use the following [link](https://l.facebook.com/l.php?u=https%3A%2F%2Fllama.developer.meta.com%2F&h=AT1np-Q96F0Qym-qTXCKJuPUqZePWH7EWzib2mRB0XSqjajogDg91FWpn-_MRnaVS7CGCR1o4Z3wff5uQNe55poibnARsHoU_Yf-uKANWztHmiSgl56GTo5ERQR1o8GmHatZ2Eb_hdCGIO4iD3-c0QZN5Sdq) to retrieve your Llama API key. Next, configure your environment by setting the \"LLAMA_API_KEY\" variable to the obtained key using one of the following steps." ] }, { "cell_type": "code", "execution_count": 25, "id": "8ff2c746-44ba-4594-9293-6fdae986fa1b", "metadata": {}, "outputs": [], "source": [ "# either run the following command in your cli terminal\n", "# export LLAMA_API_KEY='Llama API key'\n", "# Or\n", "# Directly set your LLAMA_API_KEY environment variable here\n", "# os.environ['LLAMA_API_KEY'] = 'Llama API key' " ] }, { "cell_type": "markdown", "id": "e7855614-9a8b-421d-b4ec-32b278c24f90", "metadata": {}, "source": [ "### Run the \"infer\" function to create an interactive UI where you can enter your input prompt for the LLM in the designated \"user_input\" field and view the generated response in the adjacent \"output\" field." ] }, { "cell_type": "code", "execution_count": 53, "id": "fd340fa0-a116-4f13-999b-60f5f99f2dc3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7890\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# You can switch between the following models:\n", "# Llama-3.3-70B-Instruct\n", "# Llama-4-Maverick-17B-128E-Instruct-FP8\n", "def infer(user_input:str, model_id:str):\n", " response = client.chat.completions.create(\n", " model=model_id,\n", " messages=[\n", " {\"role\": \"user\", \"content\": user_input},\n", " ])\n", " return response.completion_message.content.text\n", "\n", "\n", "demo = gr.Interface(fn=infer,\n", " inputs=[gr.Textbox(),gr.Text(\"Llama-4-Maverick-17B-128E-Instruct-FP8\")], \n", " outputs=gr.Textbox(), #<-- Gradio TextBox component as an output component\n", " )\n", "\n", "demo.launch()" ] }, { "cell_type": "markdown", "id": "1ba1007d-5730-4b2e-a52e-ce4a8bb2313b", "metadata": {}, "source": [ "### The model can be used on a range of use cases, including factoid-based question answering, to demonstrate its effectiveness and versatility in real-world scenarios." ] }, { "cell_type": "code", "execution_count": 52, "id": "bc3dbb92-134f-477d-a4f4-67cd565a0618", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7889\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# You can switch between the following models:\n", "# Llama-3.3-70B-Instruct\n", "# Llama-4-Maverick-17B-128E-Instruct-FP8\n", "def infer(user_input:str, model_id:str):\n", " response = client.chat.completions.create(\n", " model=model_id,\n", " messages=[\n", " {\"role\": \"user\", \"content\": user_input},\n", " ])\n", " return response.completion_message.content.text\n", "\n", "\n", "demo = gr.Interface(fn=infer,\n", " inputs=[gr.Textbox(),gr.Text(\"Llama-4-Maverick-17B-128E-Instruct-FP8\")], \n", " outputs=gr.Textbox(), #<-- Gradio TextBox component as an output component\n", " )\n", "\n", "demo.launch()" ] }, { "cell_type": "code", "execution_count": 54, "id": "5a47117e-700a-4025-bdf4-1f7972b0cd39", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7891\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# You can switch between the following models:\n", "# Llama-3.3-70B-Instruct\n", "# Llama-4-Maverick-17B-128E-Instruct-FP8\n", "def infer(user_input:str, model_id:str):\n", " response = client.chat.completions.create(\n", " model=model_id,\n", " messages=[\n", " {\"role\": \"user\", \"content\": user_input},\n", " ])\n", " return response.completion_message.content.text\n", "\n", "\n", "demo = gr.Interface(fn=infer,\n", " inputs=[gr.Textbox(),gr.Text(\"Llama-4-Maverick-17B-128E-Instruct-FP8\")], \n", " outputs=gr.Textbox(), #<-- Gradio TextBox component as an output component\n", " )\n", "\n", "demo.launch()" ] }, { "cell_type": "code", "execution_count": null, "id": "77d87202-da25-41e3-b3d2-7b0aa979e739", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }