LLamaRecipes/recipes/use_cases/end2end-recipes/raft/evalset.json

[
    {
       "question":"Why is Meta not sharing the training datasets for Llama?",
       "answer":"We believe developers will have plenty to work with as we release our model weights and starting code for pre-trained and conversational fine-tuned versions as well as responsible use resources. While data mixes are intentionally withheld for competitive reasons, all models have gone through Meta’s internal Privacy Review process to ensure responsible data usage in building our products. We are dedicated to the responsible and ethical development of our GenAI products, ensuring our policies reflect diverse contexts and meet evolving societal expectations."
    },
    {
       "question":"Did Meta use human annotators to develop the data for Llama models?",
       "answer":"Yes. There are more details, for example, about our use of human annotators in the Llama 2 research paper."
    },
    {
       "question":"Can I use the output of the models to improve the Llama family of models, even though I cannot use them for other LLMs?",
       "answer":"It's correct that the license restricts using any part of the Llama models, including the response outputs to train another AI model (LLM or otherwise). However, one can use the outputs to further train the Llama family of models. Techniques such as Quantized Aware Training (QAT) utilize such a technique and hence this is allowed."
    },
    {
       "question":"What operating systems (OS) are officially supported if I want to use Llama model?",
       "answer":"For the core Llama GitHub repos (Llama and Llama3) Linux is the only OS currently supported by this repo. Additional OS support is available through the Llama-Recipes repo."
    },
    {
       "question":"Do Llama models provide traditional autoregressive text completion?",
       "answer":"Llama models are auto-regressive language models, built on the transformer architecture. The core language models function by taking a sequence of words as input and predicting the next word, recursively generating text."
    },
    {
       "question":"Do Llama models support logit biases as a request parameter to control token probabilities during sampling?",
       "answer":"This is implementation dependent (i.e. the code used to run the model)."
    },
    {
       "question":"Do Llama models support adjusting sampling temperature or top-p threshold via request parameters?",
       "answer":"The model itself supports these parameters, but whether they are exposed or not depends on implementation."
    },
    {
       "question":"What is llama-recipes?",
       "answer":"The llama-recipes repository is a companion to the Meta Llama 3 models. The goal of this repository is to provide a scalable library for fine-tuning Meta Llama models, along with some example scripts and notebooks to quickly get started with using the models in a variety of use-cases, including fine-tuning for domain adaptation and building LLM-based applications with Meta Llama and other tools in the LLM ecosystem."
    },
    {
       "question":"What is the difference on the tokenization techniques that Meta Llama 3 uses compare Llama 2?",
       "answer":"Llama 2 uses SentencePiece for tokenization, whereas Llama 3 has transitioned to OpenAI’s Tiktoken."
    },
    {
       "question":"How many tokens were used in Meta Llama 3 pretrain?",
       "answer":"Meta Llama 3 is pretrained on over 15 trillion tokens that were all collected from publicly available sources."
    },
    {
       "question":"How many tokens were used in  Llama 2 pretrain?",
       "answer":"Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources."
    },
    {
       "question":"What is the name of the license agreement that Meta Llama 3 is under?",
       "answer":"Meta LLAMA 3 COMMUNITY LICENSE AGREEMENT."
    },
    {
       "question":"What is the name of the license agreement that Llama 2 is under?",
       "answer":"LLAMA 2 COMMUNITY LICENSE AGREEMENT."
    },
    {
       "question":"What is the context length of Llama 2 models?",
       "answer":"Llama 2's context is 4k"
    },
    {
       "question":"What is the context length of Meta Llama 3 models?",
       "answer":"Meta Llama 3's context is 8k"
    },
    {
       "question":"When is Llama 2 trained?",
       "answer":"Llama 2 was trained between January 2023 and July 2023."
    },
    {
       "question":"What is the name of the Llama 2 model that uses Grouped-Query Attention (GQA) ",
       "answer":"Llama 2 70B"
    },
    {
       "question":"What are the names of the Meta Llama 3 model that use Grouped-Query Attention (GQA) ",
       "answer":"Meta Llama 3 8B and Meta Llama 3 70B"
    },
    {
       "question":"what are the goals for Llama 3",
       "answer":"With Llama 3, we set out to build the best open models that are on par with the best proprietary models available today. We wanted to address developer feedback to increase the overall helpfulness of Llama 3 and are doing so while continuing to play a leading role on responsible use and deployment of LLMs. We are embracing the open source ethos of releasing early and often to enable the community to get access to these models while they are still in development."
    },
    {
       "question":"What versions of Meta Llama 3 are available?",
       "answer":"Meta Llama 3 is available in both 8B and 70B pretrained and instruction-tuned versions."
    },
    {
       "question":"What are some applications of Meta Llama 3?",
       "answer":"Meta Llama 3 supports a wide range of applications including coding tasks, problem solving, translation, and dialogue generation."
    },
    {
       "question":"What improvements does Meta Llama 3 offer over previous models?",
       "answer":"Meta Llama 3 offers enhanced scalability and performance, lower false refusal rates, improved response alignment, and increased diversity in model answers. It also excels in reasoning, code generation, and instruction following."
    },
    {
       "question":"How has Meta Llama 3 been trained?",
       "answer":"Meta Llama 3 has been trained on over 15T tokens of data using custom-built 24K GPU clusters. This training dataset is 7x larger than that used for Llama 2 and includes 4x more code."
    },
    {
       "question":"What safety measures are included with Meta Llama 3?",
       "answer":"Meta Llama 3 includes updates to trust and safety tools such as Llama Guard 2 and Cybersec Eval 2, optimized to support a comprehensive set of safety categories published by MLCommons."
    },
    {
       "question":"What is Meta Llama 3?",
       "answer":"Meta Llama 3 is a highly advanced AI model that excels at language nuances, contextual understanding, and complex tasks like translation and dialogue generation."
    },
    {
       "question":"What are the pretrained versions of Meta Llama 3 available?",
       "answer":"Meta Llama 3 is available with both 8B and 70B pretrained and instruction-tuned versions."
    },
    {
       "question":"What is the context length supported by Llama 3 models?",
       "answer":"Llama 3 models support a context length of 8K, which doubles the capacity of Llama 2."
    },
    {
        "question":"What is the Prompt engineering?",
        "answer":"It is a technique used in natural language processing (NLP) to improve the performance of the language model by providing them with more context and information about the task in hand."
     },
     {
        "question":"What is the Zero-Shot Prompting?",
        "answer":"Large language models like Meta Llama are capable of following instructions and producing responses without having previously seen an example of a task. Prompting without examples is called 'zero-shot prompting'."
     },
      {
        "question":"What are the supported quantization modes in PyTorch?",
        "answer":"Post-Training Dynamic Quantization, Post-Training Static Quantization and Quantization Aware Training (QAT)"
     },
     {
        "question":"What is the LlamaIndex?",
        "answer":"LlamaIndex is mainly a data framework for connecting private or domain-specific data with LLMs, so it specializes in RAG, smart data storage and retrieval, while LangChain is a more general purpose framework which can be used to build agents connecting multiple tools."
     },
     {
       "question":"What is the LangChain?",
       "answer":"LangChain is an open source framework for building LLM powered applications. It implements common abstractions and higher-level APIs to make the app building process easier, so you don't need to call LLM from scratch. "
    }
 ]