|
|
@@ -4,15 +4,18 @@ The 'llama-recipes' repository is a companion to the [Meta Llama 3](https://gith
|
|
|
|
|
|
<!-- markdown-link-check-enable -->
|
|
|
> [!IMPORTANT]
|
|
|
-> Meta Llama 3 has a new prompt template and special tokens (based on the tiktoken tokenizer).
|
|
|
+> Meta Llama 3.1 has a new prompt template and special tokens.
|
|
|
> | Token | Description |
|
|
|
> |---|---|
|
|
|
-> `<\|begin_of_text\|>` | This is equivalent to the BOS token. |
|
|
|
-> `<\|end_of_text\|>` | This is equivalent to the EOS token. For multiturn-conversations it's usually unused. Instead, every message is terminated with `<\|eot_id\|>` instead.|
|
|
|
-> `<\|eot_id\|>` | This token signifies the end of the message in a turn i.e. the end of a single message by a system, user or assistant role as shown below.|
|
|
|
-> `<\|start_header_id\|>{role}<\|end_header_id\|>` | These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant. |
|
|
|
+> `<\|begin_of_text\|>` | Specifies the start of the prompt. |
|
|
|
+> `<\|eot_id\|>` | This token signifies the end of a turn i.e. the end of the model's interaction either with the user or tool executor. |
|
|
|
+> `<\|eom_id\|>` | End of Message. A message represents a possible stopping point where the model can inform the execution environment that a tool call needs to be made. |
|
|
|
+> `<\|python_tag\|>` | A special tag used in the model’s response to signify a tool call. |
|
|
|
+> `<\|finetune_right_pad_id\|>` | Used for padding text sequences in a batch to the same length. |
|
|
|
+> `<\|start_header_id\|>{role}<\|end_header_id\|>` | These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant and ipython. |
|
|
|
+> `<\|end_of_text\|>` | This is equivalent to the EOS token. For multiturn-conversations it's usually unused, this token is expected to be generated only by the base models. |
|
|
|
>
|
|
|
-> A multiturn-conversation with Meta Llama 3 follows this prompt template:
|
|
|
+> A multiturn-conversation with Meta Llama 3.1 that includes tool-calling follows this structure:
|
|
|
> ```
|
|
|
> <|begin_of_text|><|start_header_id|>system<|end_header_id|>
|
|
|
>
|
|
|
@@ -20,13 +23,16 @@ The 'llama-recipes' repository is a companion to the [Meta Llama 3](https://gith
|
|
|
>
|
|
|
> {{ user_message_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
|
|
>
|
|
|
-> {{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|>
|
|
|
+> <|python_tag|>{{ model_tool_call_1 }}<|eom_id|><|start_header_id|>ipython<|end_header_id|>
|
|
|
>
|
|
|
-> {{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
|
|
+> {{ tool_response }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
|
|
+>
|
|
|
+> {{model_response_based_on_tool_response}}<|eot_id|>
|
|
|
> ```
|
|
|
> Each message gets trailed by an `<|eot_id|>` token before a new header is started, signaling a role change.
|
|
|
>
|
|
|
-> More details on the new tokenizer and prompt template can be found [here](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3#special-tokens-used-with-meta-llama-3).
|
|
|
+> More details on the new tokenizer and prompt template can be found [here](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1).
|
|
|
+
|
|
|
>
|
|
|
> [!NOTE]
|
|
|
> The llama-recipes repository was recently refactored to promote a better developer experience of using the examples. Some files have been moved to new locations. The `src/` folder has NOT been modified, so the functionality of this repo and package is not impacted.
|
albertodepaola