{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Meta---Logo@1x.jpg](data:image/jpeg;base64,[encoded data, 20312 characters])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Using externally-hosted LLMs**\n",
    "Use llama_cookbook.inference.llm to perform inference using Llama and other models using third party services. At the moment, three services have been incorporated:\n",
    "- Together.ai\n",
    "- Anyscale\n",
    "- OpenAI\n",
    "\n",
    "An API token for each service must be obtained and provided to the method before running. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_cookbook.inference.llm import  TOGETHER, OPENAI, ANYSCALE\n",
    "\n",
    "together_example = TOGETHER(\"togethercomputer/llama-2-7b-chat\",\"09e45...\")\n",
    "print( together_example.query(prompt=\"Why is the sky blue?\"))\n",
    "\n",
    "\n",
    "openai_example = OPENAI(\"gpt-3.5-turbo\",\"sk-LIz9zL3cYp...\")\n",
    "print( openai_example.query(prompt=\"Why is the sky blue?\"))\n",
    "\n",
    "\n",
    "anyscale_example = ANYSCALE(\"meta-llama/Llama-2-7b-chat-hf\",\"esecret_c3u4x7...\")\n",
    "print( anyscale_example.query(prompt=\"Why is the sky blue?\"))"
   ]
  }
 ],
 "metadata": {
  "custom": {
   "cells": [],
   "metadata": {
    "fileHeader": "",
    "fileUid": "9af50647-0f34-423b-936e-6950218a612f",
    "isAdHoc": false,
    "language_info": {
     "name": "plaintext"
    },
    "orig_nbformat": 4
   },
   "nbformat": 4,
   "nbformat_minor": 2
  },
  "indentAmount": 2,
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
