On Tuesday, Meta announced Llama 2, a new version of its large language model.
Lots of attention has been paid to LLMs like OpenAI’s GPT-4 which powers ChatGPT and Google’s PaLM 2 which powers Bard, but don’t sleep on Meta’s efforts to compete in the generative AI arena. Unlike GPT-4 and PaLM 2, which are closed models, Llama 2 is open-source (more on that below). It is already scoring similarly on certain benchmarks, and will probably only get better with input from the public.
Here’s what to know about Meta’s LLM sleeper-hit:
What is Claude 2? How to access this ChatGPT competitor.
Llama 2 is open-source, unlike its major competitors
Meta has made Llama 2 open-source and free for research and commercial use, because it gives the public more opportunity to shape and benefit from the transformative technology.
“Giving businesses, startups, entrepreneurs, and researchers access to tools developed at a scale that would be challenging to build themselves, backed by computing power they might not otherwise access, will open up a world of opportunities for them to experiment, innovate in exciting ways, and ultimately benefit from economically and socially,” said the blog post announcing the release.
Google and OpenAI haven’t shared details about how their proprietary LLMs are developed, such as specifics on the number of parameters or what data was used to train the models. Conversely, Llama 2 is freely accessible through Hugging Face, Amazon Web Services, and Microsoft Azure, meaning developers have access to its code and data, which will enable them to build and improve on the model. Llama 2 has three different pre-trained model sizes: 7 billion, 13 billion, and 70 billion parameters, which can all be accessed.
Microsoft Azure is Llama 2’s ‘preferred partner’
Alongside Meta’s release of Llama 2, it also announced a partnership with Microsoft’s cloud computing platform Azure. As of this week, Azure customers can use Llama 2 as an option for building applications. Yes, you can download Llama 2 directly, but through Azure’s AI platform, you get the fine-tuning, safety, and inference features that are specially designed for working with LLMs.
This partnership aligns with the two tech giants’ public stance of democratizing AI. Azure is a paid subscription, but Microsoft has made AI-powered Bing and Edge tools free to use. And those tools are powered by OpenAI’s GPT-4, which has received heavy investment from Microsoft. All that’s to say, Microsoft is going hard on generative AI and leveraging powerful partnerships to make it happen. For Meta, the Microsoft partnership is a valuable channel for Azure developers to use Llama 2.
How Llama 2 stacks up against ChatGPT
Enough with the background, can Llama 2 actually dethrone ChatGPT as the best LLM? The short answer is not yet, but maybe eventually.
In Meta’s research paper, it compared Llama 2’s performance on various academic benchmarks to other models, including OpenAI’s GPT-3.5 and GPT-4 and Google’s PaLM and PaLM 2. Llama 2 didn’t score as high on datasets that measure general knowledge, grade school math, or coding abilities, but Meta conducted another study using human evaluations that had interesting results. Humans scored 4,000 prompts on “helpfulness” comparing various models. In this, Llama 2 beat ChatGPT, earning 35.9 percent “wins” against ChatGPT’s 32.5 percent.
These scores are measured against closed models, but when it came to benchmark comparisons of other open-source models, Llama 2 had them beat. Meta’s LLM outperformed other open-sourced models (MPT and Falcon) across the board.
Roughly speaking, Llama 2’s intelligence is rated as similar to GPT-3.5, and it’s a rated slightly more helpful than ChatGPT in chatbot form. Theoretically, with efforts from the developer community, Llama 2 will only get better from here.
