# API Inference

This module provides a command-line interface for interacting with Llama models through the Llama API.

## Overview

The `api_inference.py` script allows you to:
- Connect to Llama's API using your API key
- Launch a Gradio web interface for sending prompts to Llama models
- Get completions from models like Llama-4-Maverick-17B

## Prerequisites

- Python 3.8 or higher
- A valid Llama API key
- Required Python packages:
  - gradio
  - llama_api_client

## Installation

Ensure you have the required packages installed:

```bash
pip install gradio llama_api_client
```

## Usage

You can run the script from the command line using:

```bash
python api_inference.py [OPTIONS]
```

### Command-line Options

- `--api-key`: Your API key (optional)
  - If not provided, the script will look for the appropriate environment variable based on the provider
- `--provider`: API provider to use (optional, default: "Llama")
  - Available options: "Llama", "OpenAI"

### Setting Up Your API Key

You can provide your API key in one of two ways:

1. **Command-line argument**:
   ```bash
   python api_inference.py --api-key YOUR_API_KEY --provider Llama
   ```

2. **Environment variable**:
   The environment variable name depends on the provider you choose:
   ```bash
   # For Llama (default provider)
   export LLAMA_API_KEY=YOUR_API_KEY

   # For OpenAI
   export OPENAI_API_KEY=YOUR_API_KEY
   ```

   For Windows:
   ```bash
   # Command Prompt (example for Llama)
   set LLAMA_API_KEY=YOUR_API_KEY

   # PowerShell (example for Llama)
   $env:LLAMA_API_KEY="YOUR_API_KEY"
   ```

## Example

1. Run the script:
   ```bash
   # Using Llama (default provider)
   python api_inference.py --api-key YOUR_API_KEY

   # Using a different provider
   python api_inference.py --api-key YOUR_API_KEY --provider OpenAI
   ```

2. The script will launch a Gradio web interface (typically at http://127.0.0.1:7860)

3. In the interface:
   - Enter your prompt in the text box
   - The default model is "Llama-4-Maverick-17B-128E-Instruct-FP8" but you can change it
   - Click "Submit" to get a response from the model

## Troubleshooting

### API Key Issues

If you see an error like:
```
No API key provided and *_API_KEY environment variable not found
```

Make sure you've either:
- Passed the API key using the `--api-key` argument
- Set the appropriate environment variable for your chosen provider (LLAMA_API_KEY)

## Advanced Usage

You can modify the script to use different models or customize the Gradio interface as needed.

## Implementation Notes

- The script uses type hints for better code readability and IDE support:
  ```python
  api_key: Optional[str] = args.api_key
  ```
  This line uses the `Optional` type from the `typing` module to indicate that `api_key` can be either a string or `None`. The `Optional` type is imported from the `typing` module at the beginning of the script.

## License

[Include license information here]