Meta could release the next version of its large language model Llama 3 as early as next week, according to reports.
The Information claims a small version of Llama 3 will get an early release with the full open source model still coming out in July and able to compete with Claude 3 and GPT-4.
The Instagram owner is spending billions on building advanced AI systems including buying hundreds of thousands of H100 GPUs from Nvidia to train Llama and other models.
What is Llama 3?
Llama 3 is a large language model that will come in a range of sizes from very small to compete with the likes of Claude Haiku or Gemini Nano, to larger fully responses and reasoning capable models like GPT-4 or Claude Opus.
Little is known about Llama 3 beyond the fact it is expected to be open source like its predecessor and is likely to be multimodal, capable of understanding visual as well as text inputs.
There will likely be a range of versions and sizes for Llama 3 ranging from as small as 7 billion parameters and as large as more than 100 billion parameters. This would still be smaller than the trillion-plus parameters used to train GPT-4.
Llama 3 is also likely to be less cautious than its predecessor, which drew criticism for over the top moderation controls and overly strict guardrails.
Why only a small version of Llama 3?
Meta released Llama 2 in July last year and it is likely as simple as wanting to stick to a consistent release schedule.
Launching a small version of the upcoming AI early will help build hype about its capabilities. Some of the functionality of Anthropic small model Claude 3 Haiku on on-par with OpenAI's massive model GPT-4.
The AI model space is growing fast and becoming competitive, including in the open source space with new models from DataBricks, Mistral and StabilityAI.
Smaller models are also becoming increasingly valuable for businesses as they are cheaper to run, easier to fine-tune and in some cases can even run on local hardware.
