{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### Imports & Env Setup" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2\n", "import sys\n", "import os\n", "from dotenv import load_dotenv\n", "load_dotenv()\n", "\n", "import dspy\n", "sys.path.append(os.path.abspath('../'))\n", "from prompt_migrator.benchmarks import llama_mmlu_pro, leaderboard_mmlu_pro" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Configuration" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "NUM_THREADS = 16\n", "\n", "FEW_SHOTS = 5\n", "\n", "# See https://docs.litellm.ai/docs/providers/vllm for details\n", "TASK_MODEL = dspy.LM(\n", " \"hosted_vllm/meta-llama/Llama-3.3-70B-Instruct\",\n", " api_base = 'http://localhost:8000/v1' , # or api_base ?\n", " # api_version: Optional[str] = None,\n", " # api_key: Optional[str] = None,\n", " # seed: Optional[int] = None,\n", " # max_tokens: Optional[int] = None,\n", " # timeout: Optional[Union[float, int]] = None,\n", ")\n", "PROMPT_MODEL = dspy.LM(\n", " \"hosted_vllm/meta-llama/Llama-3.3-70B-Instruct\",\n", " api_base = 'http://localhost:8000/v1', # or api_base ?\n", " # api_version: Optional[str] = None,\n", " # api_key: Optional[str] = None,\n", " # seed: Optional[int] = None,\n", " # max_tokens: Optional[int] = None,\n", " # timeout: Optional[Union[float, int]] = None,\n", ")\n", "\n", "dspy.configure(lm=TASK_MODEL)\n", "\n", "# replace this with llama_mmlu_pro or whatever\n", "benchmark = leaderboard_mmlu_pro\n", "\n", "# Without chain of thought:\n", "# program = dspy.Predict(\n", "# benchmark.signature(\"\")\n", "# )\n", "\n", "# With chain of thought:\n", "program = dspy.ChainOfThought(\n", " benchmark.signature(\"\") # put your initial system prompt here, or leave blank\n", ")\n", "\n", "evaluate = dspy.Evaluate(\n", " devset=[],\n", " metric=benchmark.metric,\n", " num_threads=NUM_THREADS,\n", " display_progress=True,\n", " display_table=True,\n", " return_all_scores=True,\n", " return_outputs=True,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load dataset" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(1203, 2165, 8664)" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trainset, valset, testset = benchmark.datasets(\n", " train_size=0.1,\n", " validation_size=0.2,\n", ")\n", "\n", "len(trainset), len(valset), len(testset)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Optimize Subset + Evaluation" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:44:49 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "RUNNING WITH THE FOLLOWING LIGHT AUTO RUN SETTINGS:\n", "num_trials: 7\n", "minibatch: False\n", "num_candidates: 5\n", "valset size: 20\n", "\n", "2025/01/15 17:44:49 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "==> STEP 1: BOOTSTRAP FEWSHOT EXAMPLES <==\n", "2025/01/15 17:44:49 INFO dspy.teleprompt.mipro_optimizer_v2: These will be used as few-shot example candidates for our program and for creating instructions.\n", "\n", "2025/01/15 17:44:49 INFO dspy.teleprompt.mipro_optimizer_v2: Bootstrapping N=5 sets of demonstrations...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Bootstrapping set 1/5\n", "Bootstrapping set 2/5\n", "Bootstrapping set 3/5\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 20%|███████████████████████████████████████████████ | 4/20 [00:20<01:23, 5.19s/it]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Bootstrapped 4 full traces after 4 examples for up to 1 rounds, amounting to 4 attempts.\n", "Bootstrapping set 4/5\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 40%|██████████████████████████████████████████████████████████████████████████████████████████████ | 8/20 [00:52<01:18, 6.56s/it]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Bootstrapped 4 full traces after 8 examples for up to 1 rounds, amounting to 8 attempts.\n", "Bootstrapping set 5/5\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 20%|███████████████████████████████████████████████ | 4/20 [00:21<01:24, 5.29s/it]\n", "2025/01/15 17:46:23 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "==> STEP 2: PROPOSE INSTRUCTION CANDIDATES <==\n", "2025/01/15 17:46:23 INFO dspy.teleprompt.mipro_optimizer_v2: We will use the few-shot examples from the previous step, a generated dataset summary, a summary of the program code, and a randomly selected prompting tip to propose instructions.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Bootstrapped 2 full traces after 4 examples for up to 1 rounds, amounting to 4 attempts.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:46:47 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "Proposing instructions...\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: Proposed Instructions for Predictor 0:\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: 0: Given the fields `question`, `options`, produce the fields `reasoning`, `answer`.\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: 1: To address the task effectively, provide a detailed, step-by-step explanation for your reasoning when answering multiple-choice questions across various subjects, including biology, chemistry, physics, and social sciences. Ensure your response includes the following elements: \n", "1. A clear understanding of the question being asked.\n", "2. An evaluation of each option based on relevant knowledge and critical thinking.\n", "3. A logical deduction of the most appropriate answer.\n", "4. A concise summary of your reasoning process.\n", "5. The final answer choice selected from the provided options.\n", "\n", "When constructing your response, consider the complexity and diversity of the questions, and tailor your reasoning to demonstrate a broad range of knowledge and analytical skills. This approach will facilitate the development of a robust and reliable question-answering system capable of handling a wide spectrum of educational and general knowledge queries.\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: 2: To answer multiple-choice questions that require reasoning and analysis of the subject matter, follow these steps: \n", "\n", "1. Read the question carefully and identify the key concepts and information provided.\n", "2. Analyze the options and determine which ones are plausible based on the information provided in the question.\n", "3. Use a combination of natural language processing and knowledge retrieval to generate a step-by-step reasoning process to arrive at an answer.\n", "4. Evaluate the reasoning process and select the most appropriate answer based on the analysis.\n", "5. Provide a clear and concise explanation of the reasoning process used to arrive at the answer.\n", "\n", "Given the fields `question`, `options`, produce the fields `reasoning`, `answer` by following the above steps and using a language model to generate a step-by-step explanation of how to arrive at the answer.\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: 3: To address the given task effectively, I propose the following instruction: \n", "\n", "\"Given a multiple-choice question across various subjects, including but not limited to biology, chemistry, physics, and social sciences, and a list of possible options, generate a detailed, step-by-step reasoning process to arrive at the correct answer. Consider the context, key concepts, and any relevant theories or principles that apply to the question. Ensure the reasoning is clear, logical, and easy to follow, and conclude by selecting the correct answer from the provided options.\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: 4: You are a highly skilled expert in a high-stakes testing environment, and your task is to answer a series of challenging multiple-choice questions to determine your suitability for a prestigious position. The questions will cover a wide range of subjects, including biology, chemistry, physics, and social sciences. You must carefully read each question, analyze the options, and provide a step-by-step reasoning process to arrive at the correct answer. The correct answer and your reasoning will be evaluated by a panel of judges, and your performance will determine your eligibility for the position. Given the fields `question`, `options`, produce the fields `reasoning`, `answer` to demonstrate your expertise and critical thinking skills.\n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "\n", "2025/01/15 17:47:40 INFO dspy.teleprompt.mipro_optimizer_v2: Evaluating the default program...\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Average Metric: 5.00 / 20 (25.0%): 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:24<00:00, 1.24s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:48:05 INFO dspy.evaluate.evaluate: Average Metric: 5 / 20 (25.0%)\n", "2025/01/15 17:48:05 INFO dspy.teleprompt.mipro_optimizer_v2: Default program score: 25.0\n", "\n", "2025/01/15 17:48:05 INFO dspy.teleprompt.mipro_optimizer_v2: ==> STEP 3: FINDING OPTIMAL PROMPT PARAMETERS <==\n", "2025/01/15 17:48:05 INFO dspy.teleprompt.mipro_optimizer_v2: We will evaluate the program over a series of trials with different combinations of instructions and few-shot examples to find the optimal combination using Bayesian Optimization.\n", "\n", "2025/01/15 17:48:05 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 1 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 15.00 / 20 (75.0%): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:23<00:00, 1.19s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:48:29 INFO dspy.evaluate.evaluate: Average Metric: 15 / 20 (75.0%)\n", "2025/01/15 17:48:29 INFO dspy.teleprompt.mipro_optimizer_v2: \u001b[92mBest full score so far!\u001b[0m Score: 75.0\n", "2025/01/15 17:48:29 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 75.0 with parameters ['Predictor 0: Instruction 1', 'Predictor 0: Few-Shot Set 1'].\n", "2025/01/15 17:48:29 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0]\n", "2025/01/15 17:48:29 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 75.0\n", "2025/01/15 17:48:29 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:48:29 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 2 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 16.00 / 20 (80.0%): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:22<00:00, 1.13s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:48:52 INFO dspy.evaluate.evaluate: Average Metric: 16 / 20 (80.0%)\n", "2025/01/15 17:48:52 INFO dspy.teleprompt.mipro_optimizer_v2: \u001b[92mBest full score so far!\u001b[0m Score: 80.0\n", "2025/01/15 17:48:52 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 80.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 1'].\n", "2025/01/15 17:48:52 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0, 80.0]\n", "2025/01/15 17:48:52 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 80.0\n", "2025/01/15 17:48:52 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:48:52 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 3 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 15.00 / 20 (75.0%): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:22<00:00, 1.14s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:49:15 INFO dspy.evaluate.evaluate: Average Metric: 15 / 20 (75.0%)\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 75.0 with parameters ['Predictor 0: Instruction 4', 'Predictor 0: Few-Shot Set 1'].\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0, 80.0, 75.0]\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 80.0\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 4 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 16.00 / 20 (80.0%): 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 1428.33it/s]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:49:15 INFO dspy.evaluate.evaluate: Average Metric: 16 / 20 (80.0%)\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 80.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 1'].\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0, 80.0, 75.0, 80.0]\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 80.0\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:49:15 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 5 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 14.00 / 20 (70.0%): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:25<00:00, 1.30s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:49:41 INFO dspy.evaluate.evaluate: Average Metric: 14 / 20 (70.0%)\n", "2025/01/15 17:49:41 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 70.0 with parameters ['Predictor 0: Instruction 4', 'Predictor 0: Few-Shot Set 3'].\n", "2025/01/15 17:49:41 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0, 80.0, 75.0, 80.0, 70.0]\n", "2025/01/15 17:49:41 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 80.0\n", "2025/01/15 17:49:41 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:49:41 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 6 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 15.00 / 20 (75.0%): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:24<00:00, 1.25s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:50:06 INFO dspy.evaluate.evaluate: Average Metric: 15 / 20 (75.0%)\n", "2025/01/15 17:50:06 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 75.0 with parameters ['Predictor 0: Instruction 0', 'Predictor 0: Few-Shot Set 1'].\n", "2025/01/15 17:50:06 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0, 80.0, 75.0, 80.0, 70.0, 75.0]\n", "2025/01/15 17:50:06 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 80.0\n", "2025/01/15 17:50:06 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:50:06 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 7 / 7 =====\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Average Metric: 15.00 / 20 (75.0%): 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00, 1.34s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:50:33 INFO dspy.evaluate.evaluate: Average Metric: 15 / 20 (75.0%)\n", "2025/01/15 17:50:33 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 75.0 with parameters ['Predictor 0: Instruction 4', 'Predictor 0: Few-Shot Set 4'].\n", "2025/01/15 17:50:33 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [25.0, 75.0, 80.0, 75.0, 80.0, 70.0, 75.0, 75.0]\n", "2025/01/15 17:50:33 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 80.0\n", "2025/01/15 17:50:33 INFO dspy.teleprompt.mipro_optimizer_v2: =======================\n", "\n", "\n", "2025/01/15 17:50:33 INFO dspy.teleprompt.mipro_optimizer_v2: Returning best identified program with score 80.0!\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "subset_size = 20\n", "optimizer = dspy.MIPROv2(\n", " metric=benchmark.metric,\n", " auto=\"light\",\n", " num_threads=NUM_THREADS,\n", " task_model=TASK_MODEL,\n", " prompt_model=PROMPT_MODEL,\n", " max_labeled_demos=FEW_SHOTS,\n", ")\n", "\n", "optimized_program = optimizer.compile(\n", " program,\n", " trainset=trainset[:subset_size],\n", " valset=valset[:subset_size],\n", " requires_permission_to_run=False,\n", ")" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "BEST PROMPT:\n", " To answer multiple-choice questions that require reasoning and analysis of the subject matter, follow these steps: \n", "\n", "1. Read the question carefully and identify the key concepts and information provided.\n", "2. Analyze the options and determine which ones are plausible based on the information provided in the question.\n", "3. Use a combination of natural language processing and knowledge retrieval to generate a step-by-step reasoning process to arrive at an answer.\n", "4. Evaluate the reasoning process and select the most appropriate answer based on the analysis.\n", "5. Provide a clear and concise explanation of the reasoning process used to arrive at the answer.\n", "\n", "Given the fields `question`, `options`, produce the fields `reasoning`, `answer` by following the above steps and using a language model to generate a step-by-step explanation of how to arrive at the answer.\n" ] } ], "source": [ "print(\"BEST PROMPT:\\n\", optimized_program.predict.signature.instructions)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Average Metric: 13.00 / 20 (65.0%): 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00, 1.32s/it]" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:52:03 INFO dspy.evaluate.evaluate: Average Metric: 13 / 20 (65.0%)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
questionoptionsexample_answerreasoningpred_answermetric
0A certain map uses a scale of 1 inch equals 25 miles. How many mil...[A. 50, B. 150, C. 100, D. 25, E. 5, F. 200, G. 75, H. 125, I. 175...HTo find the number of miles represented by 5 inches on the map, we...B
1When a ball at rest hangs by a single vertical string tension in t...['A. may be equal to mg depending on the speed of the ball', 'B. i...DTo solve this problem, we need to consider the forces acting on th...D✔️ [True]
2A 125-kW, 250-V, 1800-rev/min, cumulative compound d-c generator h...['A. 90%, 200 hp', 'B. 87.5%, 195 hp', 'C. 84.7%, 175 hp', 'D. 88....DTo find the efficiency and input horsepower requirements of the cu...E
3The diameter of the objective lens of a telescope is 30 mm and the...[A. 80%, B. 70%, C. 100%, D. 30%, E. 40%, F. 90%, G. 25%, H. 60%, ...BTo determine the fraction of the area of the objective lens that i...B✔️ [True]
4The temperature at the point $(x, y, z)$ in a substance with condu...[A. $800\\pi$, B. $1504\\pi$, C. $1248\\pi$, D. $1560\\pi$, E. $960\\pi...CTo find the rate of heat flow inward across the cylindrical surfac...C✔️ [True]
5Which of the following must be done when universal screening data ...['A. The school must invest in more modern educational technology....GWhen universal screening data show that very few students are succ...G✔️ [True]
6Sulfurylchloride decomposes according to the equation, SO_2CI_2(g)...['A. 1.49 × 10^-2, .0386', 'B. 7.45 × 10^-3, .0542', 'C. 2.98 × 10...ATo find the equilibrium constant K_p, we can use the equation: K_p...A✔️ [True]
7A number’s prime factors are 2, 5, 7, 13, and 31. Which of the fol...[A. 10, B. 25, C. 6, D. 8, E. 15, F. 30, G. 20, H. 4]ATo find a factor that must be a factor of the number, we need to l...A✔️ [True]
8How many ribs are there in the human body?[A. 42, B. 24, C. 18, D. 36, E. 40, F. 28, G. 22, H. 30, I. 20, J....BThe human body typically has 12 pairs of ribs, which are attached ...B✔️ [True]
9To what extent have biological agents been weaponized?['A. Military professionals believe that the majority of biologica...CThe question asks about the extent to which biological agents have...A
10A swimming pool is circular with a $40-\\mathrm{ft}$ diameter. The ...['A. $1800\\\\pi$ $\\\\mathrm{ft}^3$', 'B. $1400\\\\pi$ $\\\\mathrm{ft}^3$...ATo find the volume of water in the pool, we first need to understa...A✔️ [True]
11Determine the heat of vaporization ofacetonitrilewith a normal boi...['A. 7.8 kcal mol^-1', 'B. 6.8 kcal mol^-1', 'C. 7.0 kcal mol^-1',...ETo determine the heat of vaporization of acetonitrile, we can use ...A
12Feinberg claims that the best way to pursue happiness is to:['A. strive for success.', 'B. pursue knowledge.', 'C. pursue happ...GFeinberg's claim is related to the concept of happiness and how to...G✔️ [True]
13Consider a profit-maximizing firm in a perfectly competitive marke...[\"A. lead the firm to reduce workers' wages as they now contribute...BIn a perfectly competitive market, firms are price takers, meaning...B is incorrect because while the firm will hire more workers, the ...
14Which Confucian philosopher is described as a mystic due to his f...[A. Zhuangzi, B. Dao, C. Xunzi, D. Laozi, E. Zisi, F. Wang Yangmin...GThe question asks for a Confucian philosopher who is described as ...A
15What children's TV character is known as 'Da Niao' in China?['A. Barney', 'B. Tinky Winky', 'C. Mickey Mouse', 'D. Winnie the ...EThe question asks for a children's TV character known as 'Da Niao'...E✔️ [True]
16A buyer filed a lawsuit against a seller based on a written contra...['A. admissible, because the original contract was lost.', 'B. ina...JThe best evidence rule generally requires that the original docume...A
17Mr. Williams owns a piece of property assessed at $7,800 in a city...[A. $218.40, B. $220.00, C. $234.00, D. $210.00, E. $225.60, F. $2...ATo find the amount of property taxes Mr. Williams pays, we need to...A✔️ [True]
18An externality['A. results in a deficit of goods in the market', 'B. results in ...GAn externality refers to a situation where the production or consu...G✔️ [True]
19If GNP = $2,000 billion and the velocity of money is 4, what isthe...['A. $300 billion', 'B. $250 billion', 'C. $700 billion', 'D. $100...JTo find the money supply, we can use the equation: M * V = GNP, wh...J✔️ [True]
\n", "
" ], "text/plain": [ " question \\\n", "0 A certain map uses a scale of 1 inch equals 25 miles. How many mil... \n", "1 When a ball at rest hangs by a single vertical string tension in t... \n", "2 A 125-kW, 250-V, 1800-rev/min, cumulative compound d-c generator h... \n", "3 The diameter of the objective lens of a telescope is 30 mm and the... \n", "4 The temperature at the point $(x, y, z)$ in a substance with condu... \n", "5 Which of the following must be done when universal screening data ... \n", "6 Sulfurylchloride decomposes according to the equation, SO_2CI_2(g)... \n", "7 A number’s prime factors are 2, 5, 7, 13, and 31. Which of the fol... \n", "8 How many ribs are there in the human body? \n", "9 To what extent have biological agents been weaponized? \n", "10 A swimming pool is circular with a $40-\\mathrm{ft}$ diameter. The ... \n", "11 Determine the heat of vaporization ofacetonitrilewith a normal boi... \n", "12 Feinberg claims that the best way to pursue happiness is to: \n", "13 Consider a profit-maximizing firm in a perfectly competitive marke... \n", "14 Which Confucian philosopher is described as a mystic due to his f... \n", "15 What children's TV character is known as 'Da Niao' in China? \n", "16 A buyer filed a lawsuit against a seller based on a written contra... \n", "17 Mr. Williams owns a piece of property assessed at $7,800 in a city... \n", "18 An externality \n", "19 If GNP = $2,000 billion and the velocity of money is 4, what isthe... \n", "\n", " options \\\n", "0 [A. 50, B. 150, C. 100, D. 25, E. 5, F. 200, G. 75, H. 125, I. 175... \n", "1 ['A. may be equal to mg depending on the speed of the ball', 'B. i... \n", "2 ['A. 90%, 200 hp', 'B. 87.5%, 195 hp', 'C. 84.7%, 175 hp', 'D. 88.... \n", "3 [A. 80%, B. 70%, C. 100%, D. 30%, E. 40%, F. 90%, G. 25%, H. 60%, ... \n", "4 [A. $800\\pi$, B. $1504\\pi$, C. $1248\\pi$, D. $1560\\pi$, E. $960\\pi... \n", "5 ['A. The school must invest in more modern educational technology.... \n", "6 ['A. 1.49 × 10^-2, .0386', 'B. 7.45 × 10^-3, .0542', 'C. 2.98 × 10... \n", "7 [A. 10, B. 25, C. 6, D. 8, E. 15, F. 30, G. 20, H. 4] \n", "8 [A. 42, B. 24, C. 18, D. 36, E. 40, F. 28, G. 22, H. 30, I. 20, J.... \n", "9 ['A. Military professionals believe that the majority of biologica... \n", "10 ['A. $1800\\\\pi$ $\\\\mathrm{ft}^3$', 'B. $1400\\\\pi$ $\\\\mathrm{ft}^3$... \n", "11 ['A. 7.8 kcal mol^-1', 'B. 6.8 kcal mol^-1', 'C. 7.0 kcal mol^-1',... \n", "12 ['A. strive for success.', 'B. pursue knowledge.', 'C. pursue happ... \n", "13 [\"A. lead the firm to reduce workers' wages as they now contribute... \n", "14 [A. Zhuangzi, B. Dao, C. Xunzi, D. Laozi, E. Zisi, F. Wang Yangmin... \n", "15 ['A. Barney', 'B. Tinky Winky', 'C. Mickey Mouse', 'D. Winnie the ... \n", "16 ['A. admissible, because the original contract was lost.', 'B. ina... \n", "17 [A. $218.40, B. $220.00, C. $234.00, D. $210.00, E. $225.60, F. $2... \n", "18 ['A. results in a deficit of goods in the market', 'B. results in ... \n", "19 ['A. $300 billion', 'B. $250 billion', 'C. $700 billion', 'D. $100... \n", "\n", " example_answer \\\n", "0 H \n", "1 D \n", "2 D \n", "3 B \n", "4 C \n", "5 G \n", "6 A \n", "7 A \n", "8 B \n", "9 C \n", "10 A \n", "11 E \n", "12 G \n", "13 B \n", "14 G \n", "15 E \n", "16 J \n", "17 A \n", "18 G \n", "19 J \n", "\n", " reasoning \\\n", "0 To find the number of miles represented by 5 inches on the map, we... \n", "1 To solve this problem, we need to consider the forces acting on th... \n", "2 To find the efficiency and input horsepower requirements of the cu... \n", "3 To determine the fraction of the area of the objective lens that i... \n", "4 To find the rate of heat flow inward across the cylindrical surfac... \n", "5 When universal screening data show that very few students are succ... \n", "6 To find the equilibrium constant K_p, we can use the equation: K_p... \n", "7 To find a factor that must be a factor of the number, we need to l... \n", "8 The human body typically has 12 pairs of ribs, which are attached ... \n", "9 The question asks about the extent to which biological agents have... \n", "10 To find the volume of water in the pool, we first need to understa... \n", "11 To determine the heat of vaporization of acetonitrile, we can use ... \n", "12 Feinberg's claim is related to the concept of happiness and how to... \n", "13 In a perfectly competitive market, firms are price takers, meaning... \n", "14 The question asks for a Confucian philosopher who is described as ... \n", "15 The question asks for a children's TV character known as 'Da Niao'... \n", "16 The best evidence rule generally requires that the original docume... \n", "17 To find the amount of property taxes Mr. Williams pays, we need to... \n", "18 An externality refers to a situation where the production or consu... \n", "19 To find the money supply, we can use the equation: M * V = GNP, wh... \n", "\n", " pred_answer \\\n", "0 B \n", "1 D \n", "2 E \n", "3 B \n", "4 C \n", "5 G \n", "6 A \n", "7 A \n", "8 B \n", "9 A \n", "10 A \n", "11 A \n", "12 G \n", "13 B is incorrect because while the firm will hire more workers, the ... \n", "14 A \n", "15 E \n", "16 A \n", "17 A \n", "18 G \n", "19 J \n", "\n", " metric \n", "0 \n", "1 ✔️ [True] \n", "2 \n", "3 ✔️ [True] \n", "4 ✔️ [True] \n", "5 ✔️ [True] \n", "6 ✔️ [True] \n", "7 ✔️ [True] \n", "8 ✔️ [True] \n", "9 \n", "10 ✔️ [True] \n", "11 \n", "12 ✔️ [True] \n", "13 \n", "14 \n", "15 ✔️ [True] \n", "16 \n", "17 ✔️ [True] \n", "18 ✔️ [True] \n", "19 ✔️ [True] " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "score, results, all_scores = evaluate(\n", " optimized_program,\n", " devset=testset[:subset_size],\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Medium Optimization" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2025/01/15 17:52:08 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "RUNNING WITH THE FOLLOWING MEDIUM AUTO RUN SETTINGS:\n", "num_trials: 25\n", "minibatch: True\n", "num_candidates: 19\n", "valset size: 300\n", "\n", "2025/01/15 17:52:08 INFO dspy.teleprompt.mipro_optimizer_v2: \n", "==> STEP 1: BOOTSTRAP FEWSHOT EXAMPLES <==\n", "2025/01/15 17:52:08 INFO dspy.teleprompt.mipro_optimizer_v2: These will be used as few-shot example candidates for our program and for creating instructions.\n", "\n", "2025/01/15 17:52:08 INFO dspy.teleprompt.mipro_optimizer_v2: Bootstrapping N=19 sets of demonstrations...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Bootstrapping set 1/19\n", "Bootstrapping set 2/19\n", "Bootstrapping set 3/19\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0%|▌ | 2/500 [00:09<40:54, 4.93s/it]\n" ] }, { "ename": "KeyboardInterrupt", "evalue": "", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[37], line 11\u001b[0m\n\u001b[1;32m 1\u001b[0m subset_size \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m500\u001b[39m\n\u001b[1;32m 2\u001b[0m optimizer \u001b[38;5;241m=\u001b[39m dspy\u001b[38;5;241m.\u001b[39mMIPROv2(\n\u001b[1;32m 3\u001b[0m metric\u001b[38;5;241m=\u001b[39mbenchmark\u001b[38;5;241m.\u001b[39mmetric,\n\u001b[1;32m 4\u001b[0m auto\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmedium\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 8\u001b[0m max_labeled_demos\u001b[38;5;241m=\u001b[39mFEW_SHOTS,\n\u001b[1;32m 9\u001b[0m )\n\u001b[0;32m---> 11\u001b[0m optimized_program \u001b[38;5;241m=\u001b[39m \u001b[43moptimizer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompile\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 12\u001b[0m \u001b[43m \u001b[49m\u001b[43mprogram\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 13\u001b[0m \u001b[43m \u001b[49m\u001b[43mtrainset\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtrainset\u001b[49m\u001b[43m[\u001b[49m\u001b[43m:\u001b[49m\u001b[43msubset_size\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 14\u001b[0m \u001b[43m \u001b[49m\u001b[43mvalset\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mvalset\u001b[49m\u001b[43m[\u001b[49m\u001b[43m:\u001b[49m\u001b[43msubset_size\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 15\u001b[0m \u001b[43m \u001b[49m\u001b[43mrequires_permission_to_run\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m 16\u001b[0m \u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:169\u001b[0m, in \u001b[0;36mMIPROv2.compile\u001b[0;34m(self, student, trainset, teacher, valset, num_trials, max_bootstrapped_demos, max_labeled_demos, seed, minibatch, minibatch_size, minibatch_full_eval_steps, program_aware_proposer, data_aware_proposer, view_data_batch_size, tip_aware_proposer, fewshot_aware_proposer, requires_permission_to_run)\u001b[0m\n\u001b[1;32m 159\u001b[0m evaluate \u001b[38;5;241m=\u001b[39m Evaluate(\n\u001b[1;32m 160\u001b[0m devset\u001b[38;5;241m=\u001b[39mvalset,\n\u001b[1;32m 161\u001b[0m metric\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmetric,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 165\u001b[0m display_progress\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m,\n\u001b[1;32m 166\u001b[0m )\n\u001b[1;32m 168\u001b[0m \u001b[38;5;66;03m# Step 1: Bootstrap few-shot examples\u001b[39;00m\n\u001b[0;32m--> 169\u001b[0m demo_candidates \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_bootstrap_fewshot_examples\u001b[49m\u001b[43m(\u001b[49m\u001b[43mprogram\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtrainset\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mteacher\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 171\u001b[0m \u001b[38;5;66;03m# Step 2: Propose instruction candidates\u001b[39;00m\n\u001b[1;32m 172\u001b[0m instruction_candidates \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_propose_instructions(\n\u001b[1;32m 173\u001b[0m program,\n\u001b[1;32m 174\u001b[0m trainset,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 180\u001b[0m fewshot_aware_proposer,\n\u001b[1;32m 181\u001b[0m )\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:387\u001b[0m, in \u001b[0;36mMIPROv2._bootstrap_fewshot_examples\u001b[0;34m(self, program, trainset, seed, teacher)\u001b[0m\n\u001b[1;32m 384\u001b[0m zeroshot \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmax_bootstrapped_demos \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m0\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmax_labeled_demos \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m0\u001b[39m\n\u001b[1;32m 386\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 387\u001b[0m demo_candidates \u001b[38;5;241m=\u001b[39m \u001b[43mcreate_n_fewshot_demo_sets\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 388\u001b[0m \u001b[43m \u001b[49m\u001b[43mstudent\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mprogram\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 389\u001b[0m \u001b[43m \u001b[49m\u001b[43mnum_candidate_sets\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnum_candidates\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 390\u001b[0m \u001b[43m \u001b[49m\u001b[43mtrainset\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtrainset\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 391\u001b[0m \u001b[43m \u001b[49m\u001b[43mmax_labeled_demos\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[1;32m 392\u001b[0m \u001b[43m \u001b[49m\u001b[43mLABELED_FEWSHOT_EXAMPLES_IN_CONTEXT\u001b[49m\n\u001b[1;32m 393\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mzeroshot\u001b[49m\n\u001b[1;32m 394\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmax_labeled_demos\u001b[49m\n\u001b[1;32m 395\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 396\u001b[0m \u001b[43m \u001b[49m\u001b[43mmax_bootstrapped_demos\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[1;32m 397\u001b[0m \u001b[43m \u001b[49m\u001b[43mBOOTSTRAPPED_FEWSHOT_EXAMPLES_IN_CONTEXT\u001b[49m\n\u001b[1;32m 398\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mzeroshot\u001b[49m\n\u001b[1;32m 399\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmax_bootstrapped_demos\u001b[49m\n\u001b[1;32m 400\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 401\u001b[0m \u001b[43m \u001b[49m\u001b[43mmetric\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmetric\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 402\u001b[0m \u001b[43m \u001b[49m\u001b[43mmax_errors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmax_errors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 403\u001b[0m \u001b[43m \u001b[49m\u001b[43mteacher\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mteacher\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 404\u001b[0m \u001b[43m \u001b[49m\u001b[43mteacher_settings\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mteacher_settings\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 405\u001b[0m \u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 406\u001b[0m \u001b[43m \u001b[49m\u001b[43mmetric_threshold\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmetric_threshold\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 407\u001b[0m \u001b[43m \u001b[49m\u001b[43mrng\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrng\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 408\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 409\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m 410\u001b[0m logger\u001b[38;5;241m.\u001b[39minfo(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError generating few-shot examples: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/teleprompt/utils.py:334\u001b[0m, in \u001b[0;36mcreate_n_fewshot_demo_sets\u001b[0;34m(student, num_candidate_sets, trainset, max_labeled_demos, max_bootstrapped_demos, metric, teacher_settings, max_errors, max_rounds, labeled_sample, min_num_samples, metric_threshold, teacher, include_non_bootstrapped, seed, rng)\u001b[0m\n\u001b[1;32m 324\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m seed \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m:\n\u001b[1;32m 325\u001b[0m \u001b[38;5;66;03m# unshuffled few-shot\u001b[39;00m\n\u001b[1;32m 326\u001b[0m program \u001b[38;5;241m=\u001b[39m BootstrapFewShot(\n\u001b[1;32m 327\u001b[0m metric\u001b[38;5;241m=\u001b[39mmetric,\n\u001b[1;32m 328\u001b[0m max_errors\u001b[38;5;241m=\u001b[39mmax_errors,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 332\u001b[0m max_rounds\u001b[38;5;241m=\u001b[39mmax_rounds,\n\u001b[1;32m 333\u001b[0m )\n\u001b[0;32m--> 334\u001b[0m program2 \u001b[38;5;241m=\u001b[39m \u001b[43mprogram\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompile\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstudent\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mteacher\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mteacher\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtrainset\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtrainset_copy\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 336\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 337\u001b[0m \u001b[38;5;66;03m# shuffled few-shot\u001b[39;00m\n\u001b[1;32m 338\u001b[0m rng\u001b[38;5;241m.\u001b[39mshuffle(trainset_copy)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/teleprompt/bootstrap.py:84\u001b[0m, in \u001b[0;36mBootstrapFewShot.compile\u001b[0;34m(self, student, teacher, trainset)\u001b[0m\n\u001b[1;32m 82\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_prepare_student_and_teacher(student, teacher)\n\u001b[1;32m 83\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_prepare_predictor_mappings()\n\u001b[0;32m---> 84\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_bootstrap\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 86\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstudent \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_train()\n\u001b[1;32m 87\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstudent\u001b[38;5;241m.\u001b[39m_compiled \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/teleprompt/bootstrap.py:157\u001b[0m, in \u001b[0;36mBootstrapFewShot._bootstrap\u001b[0;34m(self, max_bootstraps)\u001b[0m\n\u001b[1;32m 154\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m round_idx \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mrange\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmax_rounds):\n\u001b[1;32m 155\u001b[0m bootstrap_attempts \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m\n\u001b[0;32m--> 157\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_bootstrap_one_example\u001b[49m\u001b[43m(\u001b[49m\u001b[43mexample\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mround_idx\u001b[49m\u001b[43m)\u001b[49m:\n\u001b[1;32m 158\u001b[0m bootstrapped[example_idx] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m\n\u001b[1;32m 159\u001b[0m \u001b[38;5;28;01mbreak\u001b[39;00m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/teleprompt/bootstrap.py:193\u001b[0m, in \u001b[0;36mBootstrapFewShot._bootstrap_one_example\u001b[0;34m(self, example, round_idx)\u001b[0m\n\u001b[1;32m 190\u001b[0m predictor_cache[name] \u001b[38;5;241m=\u001b[39m predictor\u001b[38;5;241m.\u001b[39mdemos\n\u001b[1;32m 191\u001b[0m predictor\u001b[38;5;241m.\u001b[39mdemos \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m predictor\u001b[38;5;241m.\u001b[39mdemos \u001b[38;5;28;01mif\u001b[39;00m x \u001b[38;5;241m!=\u001b[39m example]\n\u001b[0;32m--> 193\u001b[0m prediction \u001b[38;5;241m=\u001b[39m \u001b[43mteacher\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mexample\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minputs\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 194\u001b[0m trace \u001b[38;5;241m=\u001b[39m dspy\u001b[38;5;241m.\u001b[39msettings\u001b[38;5;241m.\u001b[39mtrace\n\u001b[1;32m 196\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m name, predictor \u001b[38;5;129;01min\u001b[39;00m teacher\u001b[38;5;241m.\u001b[39mnamed_predictors():\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/utils/callback.py:234\u001b[0m, in \u001b[0;36mwith_callbacks..wrapper\u001b[0;34m(instance, *args, **kwargs)\u001b[0m\n\u001b[1;32m 232\u001b[0m \u001b[38;5;66;03m# If no callbacks are provided, just call the function\u001b[39;00m\n\u001b[1;32m 233\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m callbacks:\n\u001b[0;32m--> 234\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43minstance\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 236\u001b[0m \u001b[38;5;66;03m# Generate call ID as the unique identifier for the call, this is useful for instrumentation.\u001b[39;00m\n\u001b[1;32m 237\u001b[0m call_id \u001b[38;5;241m=\u001b[39m uuid\u001b[38;5;241m.\u001b[39muuid4()\u001b[38;5;241m.\u001b[39mhex\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/primitives/program.py:22\u001b[0m, in \u001b[0;36mModule.__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 20\u001b[0m \u001b[38;5;129m@with_callbacks\u001b[39m\n\u001b[1;32m 21\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__call__\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[0;32m---> 22\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mforward\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/predict/chain_of_thought.py:20\u001b[0m, in \u001b[0;36mChainOfThought.forward\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mforward\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[0;32m---> 20\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpredict\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/utils/callback.py:234\u001b[0m, in \u001b[0;36mwith_callbacks..wrapper\u001b[0;34m(instance, *args, **kwargs)\u001b[0m\n\u001b[1;32m 232\u001b[0m \u001b[38;5;66;03m# If no callbacks are provided, just call the function\u001b[39;00m\n\u001b[1;32m 233\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m callbacks:\n\u001b[0;32m--> 234\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43minstance\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 236\u001b[0m \u001b[38;5;66;03m# Generate call ID as the unique identifier for the call, this is useful for instrumentation.\u001b[39;00m\n\u001b[1;32m 237\u001b[0m call_id \u001b[38;5;241m=\u001b[39m uuid\u001b[38;5;241m.\u001b[39muuid4()\u001b[38;5;241m.\u001b[39mhex\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/predict/predict.py:81\u001b[0m, in \u001b[0;36mPredict.__call__\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 79\u001b[0m \u001b[38;5;129m@with_callbacks\u001b[39m\n\u001b[1;32m 80\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__call__\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[0;32m---> 81\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mforward\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/predict/predict.py:111\u001b[0m, in \u001b[0;36mPredict.forward\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m 109\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mdspy\u001b[39;00m\n\u001b[1;32m 110\u001b[0m adapter \u001b[38;5;241m=\u001b[39m dspy\u001b[38;5;241m.\u001b[39msettings\u001b[38;5;241m.\u001b[39madapter \u001b[38;5;129;01mor\u001b[39;00m dspy\u001b[38;5;241m.\u001b[39mChatAdapter()\n\u001b[0;32m--> 111\u001b[0m completions \u001b[38;5;241m=\u001b[39m \u001b[43madapter\u001b[49m\u001b[43m(\u001b[49m\u001b[43mlm\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlm_kwargs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconfig\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msignature\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msignature\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdemos\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdemos\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minputs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 113\u001b[0m pred \u001b[38;5;241m=\u001b[39m Prediction\u001b[38;5;241m.\u001b[39mfrom_completions(completions, signature\u001b[38;5;241m=\u001b[39msignature)\n\u001b[1;32m 115\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m kwargs\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m_trace\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mTrue\u001b[39;00m) \u001b[38;5;129;01mand\u001b[39;00m dspy\u001b[38;5;241m.\u001b[39msettings\u001b[38;5;241m.\u001b[39mtrace \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/adapters/base.py:20\u001b[0m, in \u001b[0;36mAdapter.__call__\u001b[0;34m(self, lm, lm_kwargs, signature, demos, inputs)\u001b[0m\n\u001b[1;32m 17\u001b[0m inputs_ \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mformat(signature, demos, inputs)\n\u001b[1;32m 18\u001b[0m inputs_ \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mdict\u001b[39m(prompt\u001b[38;5;241m=\u001b[39minputs_) \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(inputs_, \u001b[38;5;28mstr\u001b[39m) \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28mdict\u001b[39m(messages\u001b[38;5;241m=\u001b[39minputs_)\n\u001b[0;32m---> 20\u001b[0m outputs \u001b[38;5;241m=\u001b[39m \u001b[43mlm\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43minputs_\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mlm_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 21\u001b[0m values \u001b[38;5;241m=\u001b[39m []\n\u001b[1;32m 23\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/utils/callback.py:234\u001b[0m, in \u001b[0;36mwith_callbacks..wrapper\u001b[0;34m(instance, *args, **kwargs)\u001b[0m\n\u001b[1;32m 232\u001b[0m \u001b[38;5;66;03m# If no callbacks are provided, just call the function\u001b[39;00m\n\u001b[1;32m 233\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m callbacks:\n\u001b[0;32m--> 234\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43minstance\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 236\u001b[0m \u001b[38;5;66;03m# Generate call ID as the unique identifier for the call, this is useful for instrumentation.\u001b[39;00m\n\u001b[1;32m 237\u001b[0m call_id \u001b[38;5;241m=\u001b[39m uuid\u001b[38;5;241m.\u001b[39muuid4()\u001b[38;5;241m.\u001b[39mhex\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/clients/lm.py:101\u001b[0m, in \u001b[0;36mLM.__call__\u001b[0;34m(self, prompt, messages, **kwargs)\u001b[0m\n\u001b[1;32m 98\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 99\u001b[0m completion \u001b[38;5;241m=\u001b[39m cached_litellm_text_completion \u001b[38;5;28;01mif\u001b[39;00m cache \u001b[38;5;28;01melse\u001b[39;00m litellm_text_completion\n\u001b[0;32m--> 101\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[43mcompletion\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 102\u001b[0m \u001b[43m \u001b[49m\u001b[43mrequest\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mdict\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 103\u001b[0m \u001b[43m \u001b[49m\u001b[43mnum_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnum_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 104\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 105\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m kwargs\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mlogprobs\u001b[39m\u001b[38;5;124m\"\u001b[39m):\n\u001b[1;32m 106\u001b[0m outputs \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m 107\u001b[0m {\n\u001b[1;32m 108\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtext\u001b[39m\u001b[38;5;124m\"\u001b[39m: c\u001b[38;5;241m.\u001b[39mmessage\u001b[38;5;241m.\u001b[39mcontent \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mhasattr\u001b[39m(c, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessage\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;28;01melse\u001b[39;00m c[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtext\u001b[39m\u001b[38;5;124m\"\u001b[39m],\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 111\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m c \u001b[38;5;129;01min\u001b[39;00m response[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mchoices\u001b[39m\u001b[38;5;124m\"\u001b[39m]\n\u001b[1;32m 112\u001b[0m ]\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/clients/lm.py:299\u001b[0m, in \u001b[0;36mrequest_cache..decorator..wrapper\u001b[0;34m(request, *args, **kwargs)\u001b[0m\n\u001b[1;32m 295\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[1;32m 296\u001b[0m \u001b[38;5;66;03m# If the cache key cannot be computed (e.g. because it contains a value that cannot\u001b[39;00m\n\u001b[1;32m 297\u001b[0m \u001b[38;5;66;03m# be converted to JSON), bypass the cache and call the target function directly\u001b[39;00m\n\u001b[1;32m 298\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m func(request, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m--> 299\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc_cached\u001b[49m\u001b[43m(\u001b[49m\u001b[43mkey\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/cachetools/__init__.py:771\u001b[0m, in \u001b[0;36mcached..decorator..wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 769\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n\u001b[1;32m 770\u001b[0m \u001b[38;5;28;01mpass\u001b[39;00m \u001b[38;5;66;03m# key not found\u001b[39;00m\n\u001b[0;32m--> 771\u001b[0m v \u001b[38;5;241m=\u001b[39m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 772\u001b[0m \u001b[38;5;66;03m# in case of a race, prefer the item already in the cache\u001b[39;00m\n\u001b[1;32m 773\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/clients/lm.py:289\u001b[0m, in \u001b[0;36mrequest_cache..decorator..func_cached\u001b[0;34m(key, request, *args, **kwargs)\u001b[0m\n\u001b[1;32m 280\u001b[0m \u001b[38;5;129m@cached\u001b[39m(\n\u001b[1;32m 281\u001b[0m \u001b[38;5;66;03m# NB: cachetools doesn't support maxsize=None; it recommends using float(\"inf\") instead\u001b[39;00m\n\u001b[1;32m 282\u001b[0m cache\u001b[38;5;241m=\u001b[39mLRUCache(maxsize\u001b[38;5;241m=\u001b[39mmaxsize \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mfloat\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124minf\u001b[39m\u001b[38;5;124m\"\u001b[39m)),\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 287\u001b[0m )\n\u001b[1;32m 288\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mfunc_cached\u001b[39m(key: \u001b[38;5;28mstr\u001b[39m, request: Dict[\u001b[38;5;28mstr\u001b[39m, Any], \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[0;32m--> 289\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/clients/lm.py:308\u001b[0m, in \u001b[0;36mcached_litellm_completion\u001b[0;34m(request, num_retries)\u001b[0m\n\u001b[1;32m 306\u001b[0m \u001b[38;5;129m@request_cache\u001b[39m(maxsize\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m)\n\u001b[1;32m 307\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcached_litellm_completion\u001b[39m(request: Dict[\u001b[38;5;28mstr\u001b[39m, Any], num_retries: \u001b[38;5;28mint\u001b[39m):\n\u001b[0;32m--> 308\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mlitellm_completion\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 309\u001b[0m \u001b[43m \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 310\u001b[0m \u001b[43m \u001b[49m\u001b[43mcache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mno-cache\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mno-store\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 311\u001b[0m \u001b[43m \u001b[49m\u001b[43mnum_retries\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnum_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 312\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/dspy/clients/lm.py:326\u001b[0m, in \u001b[0;36mlitellm_completion\u001b[0;34m(request, num_retries, cache)\u001b[0m\n\u001b[1;32m 324\u001b[0m stream \u001b[38;5;241m=\u001b[39m dspy\u001b[38;5;241m.\u001b[39msettings\u001b[38;5;241m.\u001b[39msend_stream\n\u001b[1;32m 325\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m stream \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m--> 326\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mlitellm\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletion\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 327\u001b[0m \u001b[43m \u001b[49m\u001b[43mcache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcache\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 328\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mretry_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 329\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 330\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 332\u001b[0m \u001b[38;5;66;03m# The stream is already opened, and will be closed by the caller.\u001b[39;00m\n\u001b[1;32m 333\u001b[0m stream \u001b[38;5;241m=\u001b[39m cast(MemoryObjectSendStream, stream)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/litellm/utils.py:900\u001b[0m, in \u001b[0;36mclient..wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 898\u001b[0m print_verbose(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError while checking max token limit: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mstr\u001b[39m(e)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 899\u001b[0m \u001b[38;5;66;03m# MODEL CALL\u001b[39;00m\n\u001b[0;32m--> 900\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[43moriginal_function\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 901\u001b[0m end_time \u001b[38;5;241m=\u001b[39m datetime\u001b[38;5;241m.\u001b[39mdatetime\u001b[38;5;241m.\u001b[39mnow()\n\u001b[1;32m 902\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01min\u001b[39;00m kwargs \u001b[38;5;129;01mand\u001b[39;00m kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/litellm/main.py:1573\u001b[0m, in \u001b[0;36mcompletion\u001b[0;34m(model, messages, timeout, temperature, top_p, n, stream, stream_options, stop, max_completion_tokens, max_tokens, modalities, prediction, audio, presence_penalty, frequency_penalty, logit_bias, user, response_format, seed, tools, tool_choice, logprobs, top_logprobs, parallel_tool_calls, deployment_id, extra_headers, functions, function_call, base_url, api_version, api_key, model_list, **kwargs)\u001b[0m\n\u001b[1;32m 1571\u001b[0m \u001b[38;5;66;03m## COMPLETION CALL\u001b[39;00m\n\u001b[1;32m 1572\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m-> 1573\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[43mopenai_chat_completions\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletion\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 1574\u001b[0m \u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1575\u001b[0m \u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1576\u001b[0m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1577\u001b[0m \u001b[43m \u001b[49m\u001b[43mmodel_response\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmodel_response\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1578\u001b[0m \u001b[43m \u001b[49m\u001b[43mprint_verbose\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mprint_verbose\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1579\u001b[0m \u001b[43m \u001b[49m\u001b[43mapi_key\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mapi_key\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1580\u001b[0m \u001b[43m \u001b[49m\u001b[43mapi_base\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mapi_base\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1581\u001b[0m \u001b[43m \u001b[49m\u001b[43macompletion\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43macompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1582\u001b[0m \u001b[43m \u001b[49m\u001b[43mlogging_obj\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlogging\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1583\u001b[0m \u001b[43m \u001b[49m\u001b[43moptional_params\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptional_params\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1584\u001b[0m \u001b[43m \u001b[49m\u001b[43mlitellm_params\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlitellm_params\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1585\u001b[0m \u001b[43m \u001b[49m\u001b[43mlogger_fn\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlogger_fn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1586\u001b[0m \u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;66;43;03m# type: ignore\u001b[39;49;00m\n\u001b[1;32m 1587\u001b[0m \u001b[43m \u001b[49m\u001b[43mcustom_prompt_dict\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustom_prompt_dict\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1588\u001b[0m \u001b[43m \u001b[49m\u001b[43mclient\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mclient\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;66;43;03m# pass AsyncOpenAI, OpenAI client\u001b[39;49;00m\n\u001b[1;32m 1589\u001b[0m \u001b[43m \u001b[49m\u001b[43morganization\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43morganization\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1590\u001b[0m \u001b[43m \u001b[49m\u001b[43mcustom_llm_provider\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcustom_llm_provider\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1591\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1592\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m 1593\u001b[0m \u001b[38;5;66;03m## LOGGING - log the original exception returned\u001b[39;00m\n\u001b[1;32m 1594\u001b[0m logging\u001b[38;5;241m.\u001b[39mpost_call(\n\u001b[1;32m 1595\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessages,\n\u001b[1;32m 1596\u001b[0m api_key\u001b[38;5;241m=\u001b[39mapi_key,\n\u001b[1;32m 1597\u001b[0m original_response\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mstr\u001b[39m(e),\n\u001b[1;32m 1598\u001b[0m additional_args\u001b[38;5;241m=\u001b[39m{\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mheaders\u001b[39m\u001b[38;5;124m\"\u001b[39m: headers},\n\u001b[1;32m 1599\u001b[0m )\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/litellm/llms/openai/openai.py:619\u001b[0m, in \u001b[0;36mOpenAIChatCompletion.completion\u001b[0;34m(self, model_response, timeout, optional_params, litellm_params, logging_obj, model, messages, print_verbose, api_key, api_base, api_version, dynamic_params, azure_ad_token, acompletion, logger_fn, headers, custom_prompt_dict, client, organization, custom_llm_provider, drop_params)\u001b[0m\n\u001b[1;32m 606\u001b[0m \u001b[38;5;66;03m## LOGGING\u001b[39;00m\n\u001b[1;32m 607\u001b[0m logging_obj\u001b[38;5;241m.\u001b[39mpre_call(\n\u001b[1;32m 608\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessages,\n\u001b[1;32m 609\u001b[0m api_key\u001b[38;5;241m=\u001b[39mopenai_client\u001b[38;5;241m.\u001b[39mapi_key,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 615\u001b[0m },\n\u001b[1;32m 616\u001b[0m )\n\u001b[1;32m 618\u001b[0m headers, response \u001b[38;5;241m=\u001b[39m (\n\u001b[0;32m--> 619\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmake_sync_openai_chat_completion_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 620\u001b[0m \u001b[43m \u001b[49m\u001b[43mopenai_client\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mopenai_client\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 621\u001b[0m \u001b[43m \u001b[49m\u001b[43mdata\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdata\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 622\u001b[0m \u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 623\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 624\u001b[0m )\n\u001b[1;32m 626\u001b[0m logging_obj\u001b[38;5;241m.\u001b[39mmodel_call_details[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mresponse_headers\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m headers\n\u001b[1;32m 627\u001b[0m stringified_response \u001b[38;5;241m=\u001b[39m response\u001b[38;5;241m.\u001b[39mmodel_dump()\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/litellm/llms/openai/openai.py:421\u001b[0m, in \u001b[0;36mOpenAIChatCompletion.make_sync_openai_chat_completion_request\u001b[0;34m(self, openai_client, data, timeout)\u001b[0m\n\u001b[1;32m 419\u001b[0m raw_response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m 420\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 421\u001b[0m raw_response \u001b[38;5;241m=\u001b[39m \u001b[43mopenai_client\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletions\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mwith_raw_response\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 422\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mdata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m 423\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 425\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mhasattr\u001b[39m(raw_response, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mheaders\u001b[39m\u001b[38;5;124m\"\u001b[39m):\n\u001b[1;32m 426\u001b[0m headers \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mdict\u001b[39m(raw_response\u001b[38;5;241m.\u001b[39mheaders)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/openai/_legacy_response.py:356\u001b[0m, in \u001b[0;36mto_raw_response_wrapper..wrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 352\u001b[0m extra_headers[RAW_RESPONSE_HEADER] \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtrue\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 354\u001b[0m kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mextra_headers\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m extra_headers\n\u001b[0;32m--> 356\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m cast(LegacyAPIResponse[R], \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/openai/_utils/_utils.py:279\u001b[0m, in \u001b[0;36mrequired_args..inner..wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 277\u001b[0m msg \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMissing required argument: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mquote(missing[\u001b[38;5;241m0\u001b[39m])\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 278\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(msg)\n\u001b[0;32m--> 279\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/openai/resources/chat/completions.py:859\u001b[0m, in \u001b[0;36mCompletions.create\u001b[0;34m(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, prediction, presence_penalty, reasoning_effort, response_format, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)\u001b[0m\n\u001b[1;32m 817\u001b[0m \u001b[38;5;129m@required_args\u001b[39m([\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessages\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m], [\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessages\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[1;32m 818\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcreate\u001b[39m(\n\u001b[1;32m 819\u001b[0m \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 856\u001b[0m timeout: \u001b[38;5;28mfloat\u001b[39m \u001b[38;5;241m|\u001b[39m httpx\u001b[38;5;241m.\u001b[39mTimeout \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m|\u001b[39m NotGiven \u001b[38;5;241m=\u001b[39m NOT_GIVEN,\n\u001b[1;32m 857\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ChatCompletion \u001b[38;5;241m|\u001b[39m Stream[ChatCompletionChunk]:\n\u001b[1;32m 858\u001b[0m validate_response_format(response_format)\n\u001b[0;32m--> 859\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_post\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 860\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/chat/completions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m 861\u001b[0m \u001b[43m \u001b[49m\u001b[43mbody\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmaybe_transform\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 862\u001b[0m \u001b[43m \u001b[49m\u001b[43m{\u001b[49m\n\u001b[1;32m 863\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmessages\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 864\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmodel\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 865\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43maudio\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43maudio\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 866\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfrequency_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfrequency_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 867\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfunction_call\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunction_call\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 868\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfunctions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunctions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 869\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogit_bias\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogit_bias\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 870\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 871\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmax_completion_tokens\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_completion_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 872\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmax_tokens\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 873\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmetadata\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmetadata\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 874\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmodalities\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodalities\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 875\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mn\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 876\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mparallel_tool_calls\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mparallel_tool_calls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 877\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mprediction\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mprediction\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 878\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mpresence_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpresence_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 879\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mreasoning_effort\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mreasoning_effort\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 880\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mresponse_format\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mresponse_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 881\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mseed\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 882\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mservice_tier\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mservice_tier\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 883\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstop\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstop\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 884\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstore\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstore\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 885\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstream\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 886\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstream_options\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 887\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtemperature\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 888\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtool_choice\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtool_choice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 889\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtools\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 890\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_logprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_logprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 891\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_p\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_p\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 892\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43muser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 893\u001b[0m \u001b[43m \u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 894\u001b[0m \u001b[43m \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCompletionCreateParams\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 895\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 896\u001b[0m \u001b[43m \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmake_request_options\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 897\u001b[0m \u001b[43m \u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_query\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_body\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_body\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m 898\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 899\u001b[0m \u001b[43m \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mChatCompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 900\u001b[0m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m 901\u001b[0m \u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mStream\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatCompletionChunk\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 902\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/openai/_base_client.py:1283\u001b[0m, in \u001b[0;36mSyncAPIClient.post\u001b[0;34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[0m\n\u001b[1;32m 1269\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mpost\u001b[39m(\n\u001b[1;32m 1270\u001b[0m \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m 1271\u001b[0m path: \u001b[38;5;28mstr\u001b[39m,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 1278\u001b[0m stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[1;32m 1279\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ResponseT \u001b[38;5;241m|\u001b[39m _StreamT:\n\u001b[1;32m 1280\u001b[0m opts \u001b[38;5;241m=\u001b[39m FinalRequestOptions\u001b[38;5;241m.\u001b[39mconstruct(\n\u001b[1;32m 1281\u001b[0m method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpost\u001b[39m\u001b[38;5;124m\"\u001b[39m, url\u001b[38;5;241m=\u001b[39mpath, json_data\u001b[38;5;241m=\u001b[39mbody, files\u001b[38;5;241m=\u001b[39mto_httpx_files(files), \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions\n\u001b[1;32m 1282\u001b[0m )\n\u001b[0;32m-> 1283\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m cast(ResponseT, \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mopts\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m)\u001b[49m)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/openai/_base_client.py:960\u001b[0m, in \u001b[0;36mSyncAPIClient.request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m 957\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 958\u001b[0m retries_taken \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0\u001b[39m\n\u001b[0;32m--> 960\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 961\u001b[0m \u001b[43m \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 962\u001b[0m \u001b[43m \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 963\u001b[0m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 964\u001b[0m \u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 965\u001b[0m \u001b[43m \u001b[49m\u001b[43mretries_taken\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mretries_taken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 966\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/openai/_base_client.py:996\u001b[0m, in \u001b[0;36mSyncAPIClient._request\u001b[0;34m(self, cast_to, options, retries_taken, stream, stream_cls)\u001b[0m\n\u001b[1;32m 993\u001b[0m log\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mSending HTTP Request: \u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[38;5;124m \u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[38;5;124m\"\u001b[39m, request\u001b[38;5;241m.\u001b[39mmethod, request\u001b[38;5;241m.\u001b[39murl)\n\u001b[1;32m 995\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 996\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_client\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msend\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 997\u001b[0m \u001b[43m \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 998\u001b[0m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_should_stream_response_body\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 999\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 1000\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1001\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m httpx\u001b[38;5;241m.\u001b[39mTimeoutException \u001b[38;5;28;01mas\u001b[39;00m err:\n\u001b[1;32m 1002\u001b[0m log\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEncountered httpx.TimeoutException\u001b[39m\u001b[38;5;124m\"\u001b[39m, exc_info\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m)\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpx/_client.py:914\u001b[0m, in \u001b[0;36mClient.send\u001b[0;34m(self, request, stream, auth, follow_redirects)\u001b[0m\n\u001b[1;32m 906\u001b[0m follow_redirects \u001b[38;5;241m=\u001b[39m (\n\u001b[1;32m 907\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfollow_redirects\n\u001b[1;32m 908\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(follow_redirects, UseClientDefault)\n\u001b[1;32m 909\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m follow_redirects\n\u001b[1;32m 910\u001b[0m )\n\u001b[1;32m 912\u001b[0m auth \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_build_request_auth(request, auth)\n\u001b[0;32m--> 914\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_send_handling_auth\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 915\u001b[0m \u001b[43m \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 916\u001b[0m \u001b[43m \u001b[49m\u001b[43mauth\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mauth\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 917\u001b[0m \u001b[43m \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 918\u001b[0m \u001b[43m \u001b[49m\u001b[43mhistory\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 919\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 920\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m 921\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m stream:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpx/_client.py:942\u001b[0m, in \u001b[0;36mClient._send_handling_auth\u001b[0;34m(self, request, auth, follow_redirects, history)\u001b[0m\n\u001b[1;32m 939\u001b[0m request \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mnext\u001b[39m(auth_flow)\n\u001b[1;32m 941\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[0;32m--> 942\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_send_handling_redirects\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 943\u001b[0m \u001b[43m \u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 944\u001b[0m \u001b[43m \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 945\u001b[0m \u001b[43m \u001b[49m\u001b[43mhistory\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhistory\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 946\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 947\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m 948\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpx/_client.py:979\u001b[0m, in \u001b[0;36mClient._send_handling_redirects\u001b[0;34m(self, request, follow_redirects, history)\u001b[0m\n\u001b[1;32m 976\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m hook \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_event_hooks[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrequest\u001b[39m\u001b[38;5;124m\"\u001b[39m]:\n\u001b[1;32m 977\u001b[0m hook(request)\n\u001b[0;32m--> 979\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_send_single_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 980\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m 981\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m hook \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_event_hooks[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mresponse\u001b[39m\u001b[38;5;124m\"\u001b[39m]:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpx/_client.py:1015\u001b[0m, in \u001b[0;36mClient._send_single_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 1010\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\n\u001b[1;32m 1011\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mAttempted to send an async request with a sync Client instance.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1012\u001b[0m )\n\u001b[1;32m 1014\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m request_context(request\u001b[38;5;241m=\u001b[39mrequest):\n\u001b[0;32m-> 1015\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[43mtransport\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1017\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(response\u001b[38;5;241m.\u001b[39mstream, SyncByteStream)\n\u001b[1;32m 1019\u001b[0m response\u001b[38;5;241m.\u001b[39mrequest \u001b[38;5;241m=\u001b[39m request\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpx/_transports/default.py:233\u001b[0m, in \u001b[0;36mHTTPTransport.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 220\u001b[0m req \u001b[38;5;241m=\u001b[39m httpcore\u001b[38;5;241m.\u001b[39mRequest(\n\u001b[1;32m 221\u001b[0m method\u001b[38;5;241m=\u001b[39mrequest\u001b[38;5;241m.\u001b[39mmethod,\n\u001b[1;32m 222\u001b[0m url\u001b[38;5;241m=\u001b[39mhttpcore\u001b[38;5;241m.\u001b[39mURL(\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 230\u001b[0m extensions\u001b[38;5;241m=\u001b[39mrequest\u001b[38;5;241m.\u001b[39mextensions,\n\u001b[1;32m 231\u001b[0m )\n\u001b[1;32m 232\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m map_httpcore_exceptions():\n\u001b[0;32m--> 233\u001b[0m resp \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_pool\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mreq\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 235\u001b[0m \u001b[38;5;28;01massert\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(resp\u001b[38;5;241m.\u001b[39mstream, typing\u001b[38;5;241m.\u001b[39mIterable)\n\u001b[1;32m 237\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m Response(\n\u001b[1;32m 238\u001b[0m status_code\u001b[38;5;241m=\u001b[39mresp\u001b[38;5;241m.\u001b[39mstatus,\n\u001b[1;32m 239\u001b[0m headers\u001b[38;5;241m=\u001b[39mresp\u001b[38;5;241m.\u001b[39mheaders,\n\u001b[1;32m 240\u001b[0m stream\u001b[38;5;241m=\u001b[39mResponseStream(resp\u001b[38;5;241m.\u001b[39mstream),\n\u001b[1;32m 241\u001b[0m extensions\u001b[38;5;241m=\u001b[39mresp\u001b[38;5;241m.\u001b[39mextensions,\n\u001b[1;32m 242\u001b[0m )\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/connection_pool.py:268\u001b[0m, in \u001b[0;36mConnectionPool.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 266\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m ShieldCancellation():\n\u001b[1;32m 267\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mresponse_closed(status)\n\u001b[0;32m--> 268\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\n\u001b[1;32m 269\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 270\u001b[0m \u001b[38;5;28;01mbreak\u001b[39;00m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/connection_pool.py:251\u001b[0m, in \u001b[0;36mConnectionPool.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 248\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\n\u001b[1;32m 250\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 251\u001b[0m response \u001b[38;5;241m=\u001b[39m \u001b[43mconnection\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 252\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m ConnectionNotAvailable:\n\u001b[1;32m 253\u001b[0m \u001b[38;5;66;03m# The ConnectionNotAvailable exception is a special case, that\u001b[39;00m\n\u001b[1;32m 254\u001b[0m \u001b[38;5;66;03m# indicates we need to retry the request on a new connection.\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 258\u001b[0m \u001b[38;5;66;03m# might end up as an HTTP/2 connection, but which actually ends\u001b[39;00m\n\u001b[1;32m 259\u001b[0m \u001b[38;5;66;03m# up as HTTP/1.1.\u001b[39;00m\n\u001b[1;32m 260\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_pool_lock:\n\u001b[1;32m 261\u001b[0m \u001b[38;5;66;03m# Maintain our position in the request queue, but reset the\u001b[39;00m\n\u001b[1;32m 262\u001b[0m \u001b[38;5;66;03m# status so that the request becomes queued again.\u001b[39;00m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/connection.py:103\u001b[0m, in \u001b[0;36mHTTPConnection.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 100\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_connection\u001b[38;5;241m.\u001b[39mis_available():\n\u001b[1;32m 101\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m ConnectionNotAvailable()\n\u001b[0;32m--> 103\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_connection\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandle_request\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/http11.py:133\u001b[0m, in \u001b[0;36mHTTP11Connection.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 131\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m Trace(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mresponse_closed\u001b[39m\u001b[38;5;124m\"\u001b[39m, logger, request) \u001b[38;5;28;01mas\u001b[39;00m trace:\n\u001b[1;32m 132\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_response_closed()\n\u001b[0;32m--> 133\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m exc\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/http11.py:111\u001b[0m, in \u001b[0;36mHTTP11Connection.handle_request\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 101\u001b[0m \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[1;32m 103\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m Trace(\n\u001b[1;32m 104\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mreceive_response_headers\u001b[39m\u001b[38;5;124m\"\u001b[39m, logger, request, kwargs\n\u001b[1;32m 105\u001b[0m ) \u001b[38;5;28;01mas\u001b[39;00m trace:\n\u001b[1;32m 106\u001b[0m (\n\u001b[1;32m 107\u001b[0m http_version,\n\u001b[1;32m 108\u001b[0m status,\n\u001b[1;32m 109\u001b[0m reason_phrase,\n\u001b[1;32m 110\u001b[0m headers,\n\u001b[0;32m--> 111\u001b[0m ) \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_receive_response_headers\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 112\u001b[0m trace\u001b[38;5;241m.\u001b[39mreturn_value \u001b[38;5;241m=\u001b[39m (\n\u001b[1;32m 113\u001b[0m http_version,\n\u001b[1;32m 114\u001b[0m status,\n\u001b[1;32m 115\u001b[0m reason_phrase,\n\u001b[1;32m 116\u001b[0m headers,\n\u001b[1;32m 117\u001b[0m )\n\u001b[1;32m 119\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m Response(\n\u001b[1;32m 120\u001b[0m status\u001b[38;5;241m=\u001b[39mstatus,\n\u001b[1;32m 121\u001b[0m headers\u001b[38;5;241m=\u001b[39mheaders,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 127\u001b[0m },\n\u001b[1;32m 128\u001b[0m )\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/http11.py:176\u001b[0m, in \u001b[0;36mHTTP11Connection._receive_response_headers\u001b[0;34m(self, request)\u001b[0m\n\u001b[1;32m 173\u001b[0m timeout \u001b[38;5;241m=\u001b[39m timeouts\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mread\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m)\n\u001b[1;32m 175\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[0;32m--> 176\u001b[0m event \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_receive_event\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 177\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(event, h11\u001b[38;5;241m.\u001b[39mResponse):\n\u001b[1;32m 178\u001b[0m \u001b[38;5;28;01mbreak\u001b[39;00m\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_sync/http11.py:212\u001b[0m, in \u001b[0;36mHTTP11Connection._receive_event\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m 209\u001b[0m event \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_h11_state\u001b[38;5;241m.\u001b[39mnext_event()\n\u001b[1;32m 211\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m event \u001b[38;5;129;01mis\u001b[39;00m h11\u001b[38;5;241m.\u001b[39mNEED_DATA:\n\u001b[0;32m--> 212\u001b[0m data \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_network_stream\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 213\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mREAD_NUM_BYTES\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m 214\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 216\u001b[0m \u001b[38;5;66;03m# If we feed this case through h11 we'll raise an exception like:\u001b[39;00m\n\u001b[1;32m 217\u001b[0m \u001b[38;5;66;03m#\u001b[39;00m\n\u001b[1;32m 218\u001b[0m \u001b[38;5;66;03m# httpcore.RemoteProtocolError: can't handle event type\u001b[39;00m\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 222\u001b[0m \u001b[38;5;66;03m# perspective. Instead we handle this case distinctly and treat\u001b[39;00m\n\u001b[1;32m 223\u001b[0m \u001b[38;5;66;03m# it as a ConnectError.\u001b[39;00m\n\u001b[1;32m 224\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m data \u001b[38;5;241m==\u001b[39m \u001b[38;5;124mb\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_h11_state\u001b[38;5;241m.\u001b[39mtheir_state \u001b[38;5;241m==\u001b[39m h11\u001b[38;5;241m.\u001b[39mSEND_RESPONSE:\n", "File \u001b[0;32m~/miniconda3/envs/prompt-migration/lib/python3.10/site-packages/httpcore/_backends/sync.py:126\u001b[0m, in \u001b[0;36mSyncStream.read\u001b[0;34m(self, max_bytes, timeout)\u001b[0m\n\u001b[1;32m 124\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m map_exceptions(exc_map):\n\u001b[1;32m 125\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_sock\u001b[38;5;241m.\u001b[39msettimeout(timeout)\n\u001b[0;32m--> 126\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_sock\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrecv\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmax_bytes\u001b[49m\u001b[43m)\u001b[49m\n", "\u001b[0;31mKeyboardInterrupt\u001b[0m: " ] } ], "source": [ "subset_size = 500\n", "optimizer = dspy.MIPROv2(\n", " metric=benchmark.metric,\n", " auto=\"medium\",\n", " num_threads=NUM_THREADS,\n", " task_model=TASK_MODEL,\n", " prompt_model=PROMPT_MODEL,\n", " max_labeled_demos=FEW_SHOTS,\n", ")\n", "\n", "optimized_program = optimizer.compile(\n", " program,\n", " trainset=trainset[:subset_size],\n", " valset=valset[:subset_size],\n", " requires_permission_to_run=False,\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"BEST PROMPT:\\n\", optimized_program.predict.signature.instructions)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "score, results, all_scores = evaluate(\n", " optimized_program,\n", " devset=testset[:subset_size],\n", " display_table=False,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Heavy Optimization" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "optimizer = dspy.MIPROv2(\n", " metric=benchmark.metric,\n", " auto=\"heavy\",\n", " num_threads=NUM_THREADS,\n", " task_model=TASK_MODEL,\n", " prompt_model=PROMPT_MODEL,\n", " max_labeled_demos=FEW_SHOTS,\n", ")\n", "\n", "optimized_program = optimizer.compile(\n", " program,\n", " trainset=trainset,\n", " valset=valset,\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"BEST PROMPT:\\n\", optimized_program.predict.signature.instructions)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "score, results, all_scores = evaluate(\n", " optimized_program,\n", " devset=testset,\n", " display_table=False,\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.16" } }, "nbformat": 4, "nbformat_minor": 4 }