The arms-race between technology companies regarding AI models has heated up significantly in the last two weeks. OpenAI revealed GPT-4o immediately before Google announced a number of Gemini improvements. That was followed by Microsoft announcing a bunch of Copilot+ PCs along with AI improvements of its own.
But while all that was going on, Meta was taking care of AI business of its own. The company quietly released a research paper on its efforts in the multi-modal AI space. Spotted by Venture Beat, the paper shows that Meta is working on a multi-modal large language model called Chameleon.
That's not to be confused with the generative AI model CM3leon (pronounced Chameleon) that Meta AI revealed last summer. Meta AI did say in its blog post that the CM3leon model would lead to improvements on future LLMs.
The research paper claims that Chameleon is state-of-the-art and beats or competes evenly with other models like Gemini, GPT-4 and Meta’s own Llama-2.
Similar to Google’s Gemini, Chameleon is built on an “early-fusion token-based mixed-modal” architecture. This means that model was built to learn from the beginning from a combination of images, code, text and other inputs, and it uses that content to create sequences.
The other way to build a multi-modal architecture is to sew together a number of models that are trained on a single modality. This is called “late fusion.” In essence, the AI system takes individual models and fuses them together to make inferences. Late fusion apparently works well but can be limiting in the ability for the AI to integrate information.
In the paper, the authors say that Chameleon is most similar to Google’s Gemini which was built in a similar fashion. Unlike Gemini, though, the researchers say that Chameleon is an end-to-end model.
If the paper’s claims and testing are true and can be replicated, it does appear that the Chameleon model matches or exceeds many of the AI models that are available today.
An interesting wrinkle to the claims made about Chameleon, is that Mark Zuckerberg and Meta have been pushing open source as the future for much of this year. The already existing Llama 3 is open source and expected to receive big update in July. Meta also just opened up the Quest headset operating system to hardware manufacturers.
The paper does not indicate if or when Meta might release this new model. Publicly, Meta has been hard at work on the latest iteration of the Llama assistant. Llama 3 just went live on Facebook, Instagram and WhatsApp in late April.
