|
@@ -1,5 +1,7 @@
|
|
|
# Running Llama3 8B Instruct on Android with MLC-LLM
|
|
|
|
|
|
+Author: Thierry Moreau - tmoreau@octo.ai
|
|
|
+
|
|
|
# Overview
|
|
|
In this tutorial we'll learn how to deploy Llama3 8B Instruct on an Android-based phone using MLC-LLM.
|
|
|
|
|
@@ -7,6 +9,8 @@ Machine Learning Compilation for Large Language Models (MLC LLM) is a high-perfo
|
|
|
|
|
|
You can read more about MLC-LLM at the following [link](https://github.com/mlc-ai/mlc-llm).
|
|
|
|
|
|
+MLC-LLM is also what powers the Llama3 inference APIs provided by [OctoAI](https://octo.ai/). You can use OctoAI for your Llama3 cloud-based inference needs by trying out the examples under the [following path](../../../llama_api_providers/OctoAI_API_examples/).
|
|
|
+
|
|
|
This tutorial was tested with the following setup:
|
|
|
* MacBook Pro 16 inch from 2021 with Apple M1 Max and 32GB of RAM running Sonoma 14.3.1
|
|
|
* OnePlus 12 Android Smartphone with a Snapdragon 8Gen3 SoC and 12GB or RAM, running OxygenOS 14.0
|