Meta is launching its next generation artificial intelligence model Llama 3 this July. It will be a fully open source alternative to the dominance of OpenAI’s GPT-4 and Google’s Gemini.
First reported by The Information, the new version of the popular Llama family of models has been in training since last year and is part of Meta’s push to create a superintelligent AI.
According to the report Llama 3 will provide better responses to questions on contentious subjects such as race or equality than the closed systems built by OpenAI and Google or than Llama 2, the version of the model released by Meta last year.
How far Meta will take this isn’t currently clear but according to the report senior leadership feel that the guardrails imposed on the earlier version made it "too safe".
What will make Llama 3 different?
Llama 3 is a large language model, much like Gemini, Mistral Large or GPT-4. What we don’t know is exactly what capabilities the new model will have or its size.
What is likely from previous versions of Llama is Meta will offer a range of sizes and make it easy for third-party developers to re-train, fine tune and build on top of the base model.
However, it will still have base guardrails. Not just because of the potential impact on Meta’s reputation if it goes completely rogue, but also because of increasing pressure from regulators and national governments over AI safety — including the European Union's new AI Act.
FAIR's mission is to develop the science and technology for human-level AI assistants: machines that understand the world, can perceive, remember, reason, plan, and act.In the not-to-distant future, all of our interactions with the digital world will be mediated by AI… https://t.co/qxa8zYsKCfJanuary 18, 2024
According to the Information article Meta researchers are working on ways to "loosen up" Llama 3 compared to previous generations while still maintaining overall safety.
This runs counter to some efforts from other companies to add strict guardrails around controversial topics, including Google which has been forced to disable Gemini’s ability to create images of people after it over interpreted the diversity instructions.
Meta is apparently looking to at least provide context around the query rather than ignore it or tell off the user for asking the question.
Why is Llama a big deal?
Llama 2, the previous generation large language model from Meta, is used in the company's own products to power conversational AI tools. As well as in a range of other platforms including on GroqChat as it is open source.
Being an open model also means it can be run locally on a laptop or even a phone. There are tools like Ollama or Pinokio that make this relatively easy to accomplish and you can interact with it, running entirely on your machine, like you would ChatGPT — but offline.
For Meta, Llama is crucial. It is part of the social media giant's ambitions to make AI more useful, including expanding the Meta AI assistant and building superintelligent models capable of understanding the real world and how we interact with it.
It is expected to be much larger than Llama 2 with more than 100 billion parameters, but it isn't clear yet whether it will be multimodal and so able to understand and analyze images and video as well as text — something already possible with both Gemini and GPT-4.
How is Meta tackling the refusal issue?
Meta is planning to hire someone to oversee the tone and safety training of Llama before release. This won't be to completely stop it responding, but rather help it become more nuanced in its responses and ensure it does more than say "I can't help you with that query."
This isn't just to solve the most controversial topics, but also other subjects of conversation. I asked Llama 2 via GroqChat how I could get out of going to school and it refused to respond, saying it won't tell me to lie or fake illness.
The timeline of release could still change but July seems to make sense based on previous schedules. We're also expecting a new version of OpenAI's GPT-4 this year and possibly the arrival of GPT-5.
Google is also working on new versions of Gemini and has already unveiled Gemini Pro 1.5 with a massive context window. So its going to be a big year for AI - again!
