A new artificial intelligence model that is open source, can run "on-device" and is free to install is performing as well as ChatGPT on some key tests.
Developed by French startup Mistral AI, the model is called Mixtral 8x7B and has been shown to perform well on science, math, coding and reasoning benchmarks.
Mistral AI says it is committed to open models making this available through a torrent link and putting it on the open-source AI platform Hugging Face. It has also been made available in third-party AI aggregators like Poe and made available for installion on local devices.
What is Mixtral and why is it important?
Mixtral 8x7B is the second model from Mistral AI, replacing 7B which in itself performed well on standard tests. The new AI is a “mixture of experts model” that uses multiple levels of decision-making to improve responses and accuracy.
As well as being available to download and use freely it is also an open-weights model, meaning it can be re-trained and used in other formats. Weights are bits of information that represent the knowledge the AI learned during training runs.
This is part of a growing class of open-source AI models that includes StableLM from Stable Diffusion, Llama 2 from Meta and BLOOM from Hugging Face.
How Mixtral compares
It was trained on data extracted from the open web and is a sparse model allowing for more parameters while controlling the size and number of parameters per token. While it is a 46.7 billion parameter model, it only uses 12.9 billion per token rather than the full count.
This efficiency means they can achieve GPT-level abilities on a fraction of the size. On the most common benchmarks, pitted against Llama with 70 billion parameters and GPT-3.5 with 175 billion, it outperforms both on four of them and matches them on others.
Other tests have shown it also performs well in terms of accuracy, beating Llama 2 on the TruthfulQA benchmark by 20 percentage points. Mistral AI claims it is also less biased than other models, available in a range of languages including French, Spanish and English.
What is Mistral AI
Very excited to release our second model, Mixtral 8x7B, an open weight mixture of experts model.Mixtral matches or outperforms Llama 2 70B and GPT3.5 on most benchmarks, and has the inference speed of a 12B dense model. It supports a context length of 32k tokens. (1/n) https://t.co/WdGTSARKC1 pic.twitter.com/LQhXSlZ9jODecember 11, 2023
Mistral AI is quickly becoming a major player in the generative AI space, signing partnership deals with Google, and releasing open-source models outperforming the major players.
However, the company didn’t exist a year ago when ChatGPT first launched, going from an idea on paper to one of the fastest-growing AI labs in under a year.
The French startup raised €105 million ($115m) in its first funding round just seven months ago, €500 million ($549m) last month and another €365 million ($400m) last week. It is now valued at €2 billion ($2.19bn), which is twice the required threshold for unicorn status.
The rapid growth of Mistral and other open model providers like StabilityAI and open platform Hugging Face shows the growing importance of AI models that can be run locally, controlled independently of a big company and can be adapted as needed.
