Browse Source

Add langgraph tool calling agent

Lance Martin 11 months ago
parent
commit
b9074e1d09

+ 25 - 18
recipes/use_cases/agents/langchain/README.md

@@ -2,42 +2,51 @@
 
 
 LLM agents use [planning, memory, and tools](https://lilianweng.github.io/posts/2023-06-23-agent/) to accomplish tasks.
 LLM agents use [planning, memory, and tools](https://lilianweng.github.io/posts/2023-06-23-agent/) to accomplish tasks.
 
 
-LangChain offers several different ways to implement agents.
+Agents can empower llama3 we important new capabilities.
 
 
-(1) Use [AgentExecutor](https://python.langchain.com/docs/modules/agents/quick_start/) with [tool-calling](https://python.langchain.com/docs/integrations/chat/) versions of Llama 3.
+Here, we will show how to give llama3 the ability to perform web search, as well as multi-modality: image generation (text-to-image), image analysis (image-to-text), and voice (text-to-speech) tools!
 
 
-(2) Use [LangGraph](https://python.langchain.com/docs/langgraph), a library from LangChain that can be used to build reliable agents with Llama 3.
+LangChain offers several different ways to implement agents with Llama 3. 
+
+We will show 3 different approaches:
+
+(1) `Tool calling agent` - Uses [AgentExecutor](https://python.langchain.com/docs/modules/agents/quick_start/) with [tool-calling](https://python.langchain.com/docs/integrations/chat/) versions of Llama 3.
+(2) `LangGraph tool calling agent` - Uses [LangGraph](https://python.langchain.com/docs/langgraph) with [tool-calling](https://python.langchain.com/docs/integrations/chat/) versions of Llama 3.
+(3) `LangGraph custom agent` - Uses [LangGraph](https://python.langchain.com/docs/langgraph) with **any** version of Llama 3 (so long as it supports supports structured output).
+
+As we move from option (1) to (3) the degree of customization and flexibility increaces:
+
+* Option (1) is great for getting started quickly with minimal code, but requires a version of Llama 3 with reliable tool-calling, is the least customiable, and uses high-level agent executor abstraction. 
+* Option (2) is more customizable than (1), but still requires a version of Llama 3 with reliable tool-calling.
+* Option (3) does not a version of Llama 3 with reliable tool-calling and is the most customizable, but requires the most work to implement. 
 
 
 ---
 ---
 
 
-### AgentExecutor Agent
+### `Tool calling agent` with AgentExecutor
 
 
 AgentExecutor is the runtime for an agent. AgentExecutor calls the agent, executes the actions it chooses, passes the action outputs back to the agent, and repeats.
 AgentExecutor is the runtime for an agent. AgentExecutor calls the agent, executes the actions it chooses, passes the action outputs back to the agent, and repeats.
 
 
 Our first notebook, `tool-calling-agent`, shows how to build a [tool calling agent](https://python.langchain.com/docs/modules/agents/agent_types/tool_calling/) with AgentExecutor and Llama 3.
 Our first notebook, `tool-calling-agent`, shows how to build a [tool calling agent](https://python.langchain.com/docs/modules/agents/agent_types/tool_calling/) with AgentExecutor and Llama 3.
 
 
-This shows how to build an agent that uses web search, text2image, image2text, and text2speech tools.
-
 --- 
 --- 
 
 
-### LangGraph Agent
+### `LangGraph tool calling agent`
 
 
 [LangGraph](https://python.langchain.com/docs/langgraph) is a library from LangChain that can be used to build reliable agents.
 [LangGraph](https://python.langchain.com/docs/langgraph) is a library from LangChain that can be used to build reliable agents.
 
 
-LangGraph can be used to build agents with a few pieces:
-- **Planning:** Define a control flow of steps that you want the agent to take (a graph)
-- **Memory:** Persist information (graph state) across these steps
-- **Tool use:** Modify state at any step
+Our second notebook, `langgraph-tool-calling-agent`, shows an alternative to AgentExecutor for building a Llama 3 powered agent. 
+
+--- 
 
 
-Our second notebook, `langgraph-agent`, shows an alternative way to AgentExecutor to build a Llama 3 powered agent in LangGraph. 
+### `LangGraph custom agent`
 
 
-It discusses some of the trade-offs between AgentExecutor and LangGraph. 
+Our third notebook, `langgraph-custom-agent`, shows how to build a Llama 3 powered agent without reliance on tool-calling. 
 
 
 --- 
 --- 
 
 
 ### LangGraph RAG Agent
 ### LangGraph RAG Agent
 
 
-Our third notebook, `langgraph-rag-agent`, shows how to apply LangGraph to build advanced Llama 3 powered RAG agents that use ideas from 3 papers:
+Our fourth notebook, `langgraph-rag-agent`, shows how to apply LangGraph to build a custom Llama 3 powered RAG agent that use ideas from 3 papers:
 
 
 * Corrective-RAG (CRAG) [paper](https://arxiv.org/pdf/2401.15884.pdf) uses self-grading on retrieved documents and web-search fallback if documents are not relevant.
 * Corrective-RAG (CRAG) [paper](https://arxiv.org/pdf/2401.15884.pdf) uses self-grading on retrieved documents and web-search fallback if documents are not relevant.
 * Self-RAG [paper](https://arxiv.org/abs/2310.11511) adds self-grading on generations for hallucinations and for ability to answer the question.
 * Self-RAG [paper](https://arxiv.org/abs/2310.11511) adds self-grading on generations for hallucinations and for ability to answer the question.
@@ -50,12 +59,10 @@ We implement each approach as a control flow in LangGraph:
 
 
 We will build from CRAG (blue, below) to Self-RAG (green) and finally to Adaptive RAG (red):
 We will build from CRAG (blue, below) to Self-RAG (green) and finally to Adaptive RAG (red):
 
 
-![Screenshot 2024-05-03 at 10 50 02 AM](https://github.com/rlancemartin/llama-recipes/assets/122662504/ec4aa1cd-3c7e-4cd1-a1e7-7deddc4033a8)
-
 --- 
 --- 
 
 
 ### Local LangGraph RAG Agent
 ### Local LangGraph RAG Agent
 
 
-Our fourth notebook, `langgraph-rag-agent-local`, shows how to apply LangGraph to build advanced RAG agents using Llama 3 that run locally and reliably.
+Our fifth notebook, `langgraph-rag-agent-local`, shows how to apply LangGraph to build advanced RAG agents using Llama 3 that run locally and reliably.
 
 
-See this [video overview](https://www.youtube.com/watch?v=sgnrL7yo1TE) for more detail.
+See this [video overview](https://www.youtube.com/watch?v=sgnrL7yo1TE) for more detail on the design of this agent.

+ 1 - 1
recipes/use_cases/agents/langchain/langgraph-custom-agent.ipynb

@@ -42,7 +42,7 @@
     "\n",
     "\n",
     "We can still implement an agent in LangGraph.\n",
     "We can still implement an agent in LangGraph.\n",
     "\n",
     "\n",
-    "As we showed before, we'll give llama3 with various multi-model capabilities using an agent. \n",
+    "As we showed before, we'll give Llama 3 with various multi-model capabilities using an agent. \n",
     "\n",
     "\n",
     "![Screenshot 2024-05-14 at 3.28.29 PM.png](attachment:bfb9c05d-e857-423e-bacd-26e52fff67a9.png)"
     "![Screenshot 2024-05-14 at 3.28.29 PM.png](attachment:bfb9c05d-e857-423e-bacd-26e52fff67a9.png)"
    ]
    ]

+ 1 - 1
recipes/use_cases/agents/langchain/langgraph-tool-calling-agent.ipynb

@@ -36,7 +36,7 @@
     "\n",
     "\n",
     "![Screenshot 2024-05-14 at 2.17.55 PM.png](attachment:76e788ea-72ee-4a39-9f6c-6911d2937b84.png)\n",
     "![Screenshot 2024-05-14 at 2.17.55 PM.png](attachment:76e788ea-72ee-4a39-9f6c-6911d2937b84.png)\n",
     "\n",
     "\n",
-    "As we showed before, we'll augment a tool-calling version of llama3 with various multi-model capabilities using an agent. \n",
+    "As we showed before, we'll augment a tool-calling version of Llama 3 with various multi-model capabilities using an agent. \n",
     "\n",
     "\n",
     "### Enviorment\n",
     "### Enviorment\n",
     "\n",
     "\n",

+ 2 - 2
recipes/use_cases/agents/langchain/tool-calling-agent.ipynb

@@ -34,7 +34,7 @@
     "\n",
     "\n",
     "LangChain's [agent executor](https://python.langchain.com/docs/modules/agents/agent_types/tool_calling/) offers a simple way to quickly get started with agents.\n",
     "LangChain's [agent executor](https://python.langchain.com/docs/modules/agents/agent_types/tool_calling/) offers a simple way to quickly get started with agents.\n",
     "\n",
     "\n",
-    "Here, we will show how to augment a tool-calling version of llama3 with various multi-model capabilities using an agent. \n",
+    "Here, we will show how to augment a tool-calling version of Llama 3 with various multi-model capabilities using an agent. \n",
     "\n",
     "\n",
     "![image.png](attachment:6af2e76d-6f3d-44c9-a128-c19e70a400e9.png)"
     "![image.png](attachment:6af2e76d-6f3d-44c9-a128-c19e70a400e9.png)"
    ]
    ]
@@ -471,7 +471,7 @@
    "source": [
    "source": [
     "## LLM\n",
     "## LLM\n",
     "\n",
     "\n",
-    "Here, we need a Llama3 model that supports tool use.\n",
+    "Here, we need a Llama 3 model that supports tool use.\n",
     "\n",
     "\n",
     "This can be accomplished via prompt engineering (e.g., see [here](https://replicate.com/hamelsmu/llama-3-70b-instruct-awq-with-tools)) or fine-tuning (e.g., see [here](https://huggingface.co/mzbac/llama-3-8B-Instruct-function-calling) and [here](https://huggingface.co/mzbac/llama-3-8B-Instruct-function-calling)).\n",
     "This can be accomplished via prompt engineering (e.g., see [here](https://replicate.com/hamelsmu/llama-3-70b-instruct-awq-with-tools)) or fine-tuning (e.g., see [here](https://huggingface.co/mzbac/llama-3-8B-Instruct-function-calling) and [here](https://huggingface.co/mzbac/llama-3-8B-Instruct-function-calling)).\n",
     "\n",
     "\n",