Llama 2 (7b)
Model name: Llama 2 (7 Billion Parameters Version)
Developer: Meta
Overview
Llama 2 is part of a series of advanced language models developed by Meta, with this particular variant harnessing 7 billion parameters. It has been specifically fine-tuned for dialogue use cases, offering a step-up in capabilities compared to generic language models.
Base Model
The base model is an auto-regressive language model that utilizes an optimized transformer architecture. It's designed to take text as input and generate text as output, making it highly versatile in language processing tasks.
Unique Features
- Performance: This model outperforms its peers in various standard academic benchmarks such as commonsense reasoning, world knowledge, and reading comprehension.
- Scale: The model is part of a series of models that span from 7 billion to 70 billion parameters, allowing users to choose the scale that best fits their use case.
- Safety Measures: The model uses supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to better align with human preferences for safety and helpfulness.
Training Data
Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over one million new human-annotated examples. Neither the pretraining nor the fine-tuning datasets include Meta user data. The pretraining data has a cutoff of September 2022, but some tuning data is more recent, up to July 2023.
Commercial License
The Llama 2 Community License Agreement permits non-exclusive, worldwide, non-transferable, and royalty-free limited use of Llama 2. Users are allowed to use, reproduce, distribute, and modify the Llama Materials. However, if a user or their affiliated entity surpasses 700 million monthly active users, they must request a license from Meta for commercial use. Meta reserves the right to grant such a license at its discretion. It's essential to note that this license allows for commercial use, but Meta's permission is required under certain conditions.
Additional resources
For more information, here are some helpful resources: