{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Meta---Logo@1x.jpg](data:image/jpeg;base64,[encoded data, 20312 characters])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# **Using externally-hosted LLMs**\n",
        "Use llama_recipes.inference.llm to perform inference using Llama and other models using third party services. At the moment, three services have been incorporated:\n",
        "- Together.ai\n",
        "- Anyscale\n",
        "- OpenAI\n",
        "\n",
        "An API token for each service must be obtained and provided to the method before running. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from llama_recipes.inference.llm import  TOGETHER, OPENAI, ANYSCALE\n",
        "\n",
        "together_example = TOGETHER(\"togethercomputer/llama-2-7b-chat\",\"09e45...\")\n",
        "print( together_example.query(prompt=\"Why is the sky blue?\"))\n",
        "\n",
        "\n",
        "openai_example = OPENAI(\"gpt-3.5-turbo\",\"sk-LIz9zL3cYp...\")\n",
        "print( openai_example.query(prompt=\"Why is the sky blue?\"))\n",
        "\n",
        "\n",
        "anyscale_example = ANYSCALE(\"meta-llama/Llama-2-7b-chat-hf\",\"esecret_c3u4x7...\")\n",
        "print( anyscale_example.query(prompt=\"Why is the sky blue?\"))"
      ]
    }
  ],
  "metadata": {
    "custom": {
      "cells": [],
      "metadata": {
        "fileHeader": "",
        "fileUid": "9af50647-0f34-423b-936e-6950218a612f",
        "isAdHoc": false,
        "language_info": {
          "name": "plaintext"
        },
        "orig_nbformat": 4
      },
      "nbformat": 4,
      "nbformat_minor": 2
    },
    "indentAmount": 2
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
