BrandWikiReturn

Llama (language model)

Llama ("Large Language Model Meta AI" serving as a backronym) is a family of large language models (LLMs) released by Meta AI starting in February 2023.[3]

Llama models come in different sizes, ranging from 1 billion to 2 trillion parameters. Initially only a foundation model, starting with Llama 2, Meta AI released instruction fine-tuned versions alongside foundation models.

Model weights for the first version of Llama were only available to researchers on a case-by-case basis, under a non-commercial license.[4] Unauthorized copies of the first model were shared via BitTorrent.[5] Subsequent versions of Llama were made accessible outside academia and released under licenses that permitted some commercial use.[6]

Alongside the release of Llama 3, Meta rolled out Meta AI, an AI assistant built on Llama. Meta AI has a dedicated website and is available on Facebook and WhatsApp.[7] The latest version is Llama 4, released in April 2025.[8]

In April 2026, Meta Superintelligence Labs released Muse Spark as a replacement for Llama.[9]

Background

After the release of large language models such as GPT-3, a focus of research was up-scaling models, which in some instances showed major increases in emergent capabilities.[10] The release of ChatGPT and its surprise success caused an increase in attention to large language models.[11]

Compared with other responses to ChatGPT, Meta's Chief AI scientist Yann LeCun stated that large language models are best for aiding with writing.[12][13][14]

Versions

Initial release

The first version of Llama (stylized as LLaMA and sometimes referred to as Llama 1) was announced on February 24, 2023, via a blog post and a paper describing the model's training, architecture, and performance. The inference code used to run the model was publicly released under the open-source GPLv3 license. Access to the model's weights was managed by an application process, with access to be granted "on a case-by-case basis to academic researchers; those affiliated with organizations in government, civil society, and academia; and industry research laboratories around the world".

Llama was trained on only publicly available information, and was trained at various model sizes, with the intention to make it more accessible to different hardware. The model was exclusively a foundation model, although the paper contained examples of instruction fine-tuned versions of the model.

Meta AI reported the 13B parameter model performance on most NLP benchmarks exceeded that of the much larger GPT-3 (with 175B parameters), and the largest 65B model was competitive with state of the art models such as PaLM and Chinchilla.[15]

Leak

On March 3, 2023, a torrent containing Llama's weights was uploaded, with a link to the torrent shared on the 4chan imageboard and subsequently spread through online AI communities. That same day, a pull request on the main Llama repository was opened, requesting to add the magnet link to the official documentation.[16][17] On March 4, a pull request was opened to add links to HuggingFace repositories containing the model.[18][16] On March 6, Meta filed takedown requests to remove the HuggingFace repositories linked in the pull request, characterizing it as "unauthorized distribution" of the model. HuggingFace complied with the requests.[19] On March 20, Meta filed a DMCA takedown request for copyright infringement against a repository containing a script that downloaded Llama from a mirror, and GitHub complied the next day.[20]

Reactions to the leak varied. Some speculated that the model would be used for malicious purposes, such as more sophisticated spam. Some have celebrated the model's accessibility, as well as the fact that smaller versions of the model can be run relatively cheaply, suggesting that this will promote the flourishing of additional research developments. Multiple commentators, such as Simon Willison, compared Llama to Stable Diffusion, a text-to-image model which, unlike comparably sophisticated models which preceded it, was openly distributed, leading to a rapid proliferation of associated tools, techniques, and software.

Leak

On March 3, 2023, a torrent containing Llama's weights was uploaded, with a link to the torrent shared on the 4chan imageboard and subsequently spread through online AI communities. That same day, a pull request on the main Llama repository was opened, requesting to add the magnet link to the official documentation.[16][17] On March 4, a pull request was opened to add links to HuggingFace repositories containing the model.[18][16] On March 6, Meta filed takedown requests to remove the HuggingFace repositories linked in the pull request, characterizing it as "unauthorized distribution" of the model. HuggingFace complied with the requests.[19] On March 20, Meta filed a DMCA takedown request for copyright infringement against a repository containing a script that downloaded Llama from a mirror, and GitHub complied the next day.[20]

Reactions to the leak varied. Some speculated that the model would be used for malicious purposes, such as more sophisticated spam. Some have celebrated the model's accessibility, as well as the fact that smaller versions of the model can be run relatively cheaply, suggesting that this will promote the flourishing of additional research developments. Multiple commentators, such as Simon Willison, compared Llama to Stable Diffusion, a text-to-image model which, unlike comparably sophisticated models which preceded it, was openly distributed, leading to a rapid proliferation of associated tools, techniques, and software.

Llama 2

On July 18, 2023, in partnership with Microsoft, Meta announced Llama 2 (stylized as LLaMa 2), the next generation of Llama. Meta trained and released Llama 2 in three model sizes: 7, 13, and 70 billion parameters.[21] The model architecture remains largely unchanged from that of Llama 1 models, but 40% more data was used to train the foundational models.[22]

Llama 2 includes foundation models and models fine-tuned for chat. In a further departure from the original version of Llama, all models are released with weights and may be used for many commercial use cases. Because Llama's license enforces an acceptable use policy that prohibits Llama from being used for some purposes, it is not open source. Meta's use of the term open-source to describe Llama has been disputed by the Open Source Initiative (which maintains The Open Source Definition) and others.[23][24]

Code Llama is a fine-tune of Llama 2 with code specific datasets. 7B, 13B, and 34B versions were released on August 24, 2023, with a 70B version released on January 29, 2024.[25] Starting with the foundation models from Llama 2, Meta AI would train an additional 500B tokens of code datasets, before an additional 20B token of long-context data, creating the Code Llama foundation models. This foundation model was further trained on 5B instruction following token to create the instruct fine-tune. Another foundation model was created for Python code, which trained on 100B tokens of Python-only code, before the long-context data.[26]

Llama 3

On April 18, 2024, Meta released Llama 3 with two sizes: 8B and 70B parameters. The models have been pre-trained on approximately 15 trillion tokens of text gathered from “publicly available sources” with the instruct models fine-tuned on “publicly available instruction datasets, as well as over 10M human-annotated examples". Meta AI's testing showed in April 2024 that Llama 3 70B was beating Gemini Pro 1.5 and Claude 3 Sonnet on most benchmarks. Meta also announced plans to make Llama 3 multilingual and multimodal, better at coding and reasoning, and to increase its context window.[27][28]

Regarding scaling laws, Llama 3 models empirically showed that when a model is trained on data that is more than the "Chinchilla-optimal" amount, the performance continues to scale log-linearly. For example, the Chinchilla-optimal dataset for Llama 3 8B is 200 billion tokens, but performance continued to scale log-linearly to the 75-times larger dataset of 15 trillion tokens.[29]

During an interview with Dwarkesh Patel, Mark Zuckerberg said that the 8B version of Llama 3 was nearly as powerful as the largest Llama 2. Compared to previous models, Zuckerberg stated the team was surprised that the 70B model was still learning even at the end of the 15T tokens training. The decision was made to end training to focus GPU power elsewhere.[30]

Llama 3.1 was released on July 23, 2024, with three sizes: 8B, 70B, and 405B parameters.[31][32]

Llama 4

[[File:A Representation of Meta AI and Llama (Meta AI Imagine 2025).webp|alt=An AI-generated image of a glowing neon orb and a llama|thumb|Example of an image generated by Meta AI Imagine, powered by Llama 4. Prompt:

External links

References

  1. Llama 4 Community License Agreement 5 April 2025, retrieved 15 July 2025^
  2. Llama 4 Acceptable Use Policy Meta Llama, retrieved 15 July 2025^
  3. Kif Leswing. Mark Zuckerberg announces Meta's new large language model as A.I. race heats up CNBC, 2023-02-24, retrieved 2025-04-10^
  4. Yuvraj Malik, Katie Paul. Meta heats up Big Tech's AI arms race with new language model Reuters, 25 February 2023, retrieved 6 August 2025^
  5. Alex Hern. TechScape: Will Meta's massive leak democratise AI – and at what cost? The Guardian, 2023-03-07, retrieved 2025-04-10^
  6. Emilia David. Meta's AI research head wants open source licensing to change The Verge, 30 October 2023, retrieved 20 October 2024^
  7. Alex Heath. Meta's battle with ChatGPT begins now The Verge, 2024-04-18, retrieved 2025-04-10^
  8. Carl Franzen. Meta defends Llama 4 release against 'reports of mixed quality,' blames bugs VentureBeat, 2025-04-08, retrieved 2025-04-10^
  9. Meta debuts new AI model in first test of costly ‘superintelligence’ team The Guardian, 2026-04-09, retrieved 2026-04-11^
  10. Examining Emergent Abilities in Large Language Models hai.stanford.edu, 13 September 2022^
  11. The inside story of how ChatGPT was built from the people who made it MIT Technology Review, retrieved 2024-10-20^
  12. Tiernan Ray. ChatGPT is 'not particularly innovative,' and 'nothing revolutionary', says Meta's chief AI scientist ZDNET, 23 January 2023^
  13. Nik Badminton. Meta's Yann LeCun on auto-regressive Large Language Models (LLMs) Futurist.com, 13 February 2023, retrieved 20 October 2024^
  14. Yann LeCun on LinkedIn: My unwavering opinion on current (auto-regressive) LLMs LinkedIn, retrieved 2024-10-20^
  15. Matthias Bastian. Metas "LLaMA" language model shows that parameters are not everything The Decoder, 2023-02-25, retrieved 2025-06-21^
  16. Anirudh VK. Meta's LLaMA Leaked to the Public, Thanks To 4chan Analytics India Magazine, 6 March 2023, retrieved 17 March 2023^
  17. Save bandwidth by using a torrent to distribute more efficiently by ChristopherKing42 · Pull Request #73 · facebookresearch/llama GitHub, retrieved 25 March 2023^
  18. Download weights from hugging face to help us save bandwidth by Jainam213 · Pull Request #109 · facebookresearch/llama GitHub, retrieved 17 March 2023^
  19. Joseph Cox. Facebook's Powerful Large Language Model Leaks Online Vice, 7 March 2023, retrieved 17 March 2023^
  20. github/dmca - Notice of Claimed Infringement via Email GitHub, 21 March 2023, retrieved 25 March 2023^
  21. Meta and Microsoft Introduce the Next Generation of LLaMA Meta, 18 July 2023, retrieved 21 July 2023^
  22. Hugo Touvron, Louis Martin. LLaMA-2: Open Foundation and Fine-Tuned Chat Models 18 Jul 2023^
  23. Benj Edwards. Meta launches LLaMA-2, a source-available AI model that allows commercial applications Ars Technica, 2023-07-18, retrieved 2023-08-08^
  24. Prasanth Aby Thomas. Meta offers Llama AI to US government for national security CIO, 5 November 2024, retrieved 9 December 2024^
  25. Introducing Code Llama, a state-of-the-art large language model for coding ai.meta.com, retrieved 2024-10-20^
  26. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu. Code Llama: Open Foundation Models for Code 2024-01-31^
  27. Kyle Wiggers. Meta releases Llama 3, claims it's among the best open models available TechCrunch, 18 April 2024, retrieved 20 October 2024^
  28. Tobias Mann. Meta debuts third-generation Llama large language model The Register, April 19, 2024, retrieved October 20, 2024^
  29. Introducing Meta Llama 3: The most capable openly available LLM to date ai.meta.com, April 18, 2024, retrieved 2024-04-21^
  30. Dwarkesh Patel. Mark Zuckerberg - Llama 3, Open Sourcing $10b Models, & Caesar Augustus www.dwarkeshpatel.com, 2024-07-24, retrieved 2024-08-01^
  31. Meta's Llama 3.1 is open-source, kind of. Here's how it could reshape the AI race Fast Company, 2024-07-23^
  32. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten. The Llama 3 Herd of Models 2024-07-31^
  33. The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation ai.meta.com, retrieved 2025-04-05^
  34. Kylie Robison. Meta got caught gaming AI benchmarks The Verge, 8 April 2025, retrieved 8 April 2025^
  35. Kyle Wiggers. Meta's benchmarks for its new AI models are a bit misleading TechCrunch, 6 April 2025, retrieved 8 April 2025^
  36. Carl Franzen. Meta defends Llama 4 release against 'reports of mixed quality,' blames bugs VentureBeat, 8 April 2025, retrieved 8 April 2025^
  37. Llama Models www.llama.com, retrieved April 20, 2025^
  38. The Falcon has landed in the Hugging Face ecosystem huggingface.co, retrieved 2023-06-20^
  39. llama/MODEL_CARD.md at main · meta-llama/llama GitHub, retrieved 2024-05-28^
  40. Andrej Karpathy (Apr 18, 2024), The model card has some more interesting info too X (formerly Twitter), retrieved October 20, 2024^
  41. llama3/MODEL_CARD.md at main · meta-llama/llama3 GitHub, retrieved 2024-05-28^
  42. llama-models/models/llama3_1/MODEL_CARD.md at main · meta-llama/llama-models GitHub, retrieved 2024-07-23^
  43. Kylie Robison. Meta releases its first open AI model that can process images The Verge, 2024-09-25, retrieved 2024-09-25^
  44. Kyle Wiggers. Meta's Llama AI models get multimodal TechCrunch, 2024-09-25, retrieved 2024-09-25^
  45. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models ai.meta.com, retrieved 2024-09-26^
  46. meta-llama/Llama-4-Maverick-17B-128E · Hugging Face huggingface.co, 2025-04-05, retrieved 2025-04-06^
  47. Noam Shazeer. GLU Variants Improve Transformer 2020-02-01^
  48. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, Yunfeng Liu. RoFormer: Enhanced Transformer with Rotary Position Embedding 2021-04-01^
  49. Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton. Layer Normalization 2016-07-01^
  50. Biao Zhang, Rico Sennrich. Root Mean Square Layer Normalization 2019-10-01^
  51. Sharon Goldman. RedPajama replicates LLaMA dataset to build open source, state-of-the-art LLMs VentureBeat, 2023-04-18, retrieved 2025-06-21^
  52. Kyle Wiggers. Mark Zuckerberg gave Meta's Llama team the OK to train on copyrighted works, filing claims Techcrunch, January 9, 2025, retrieved January 12, 2025^
  53. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, Tatsunori B. Hashimoto. Alpaca: A Strong, Replicable Instruction-Following Model Stanford Center for Research on Foundation Models, 13 March 2023^
  54. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi. Self-Instruct: Aligning Language Models with Self-Generated Instructions 2022^
  55. Stanford CRFM crfm.stanford.edu, retrieved 2023-03-20^
  56. Katyanna Quach. Stanford takes costly, risky Alpaca AI model offline www.theregister.com^
  57. Stanford Researchers Take Down Alpaca AI Over Cost and Hallucinations Gizmodo, 21 March 2023, retrieved 20 October 2024^
  58. Meditron: An LLM suite for low-resource medical settings leveraging Meta Llama ai.meta.com^
  59. Tanya Petersen. EPFL's new Large Language Model for Medical Knowledge 28 November 2023, retrieved 20 October 2024^
  60. epfLLM/meditron epfLLM, 11 May 2024, retrieved 20 October 2024^
  61. How Companies Are Using Meta Llama Meta, 7 May 2024, retrieved 20 October 2024^
  62. How dependent is China on US artificial intelligence technology? Reuters, May 9, 2024^
  63. Benj Edwards. You can now run a GPT-3-level AI model on your laptop, phone, and Raspberry Pi Ars Technica, 2023-03-13, retrieved 2024-01-04^
  64. GGUF huggingface.co, retrieved 9 May 2024^
  65. Maxime Labonne. Quantize Llama models with GGUF and llama.cpp Medium, Towards Data Science, 29 November 2023, retrieved 9 May 2024^
  66. Matthew Connatser. Llamafile LLM driver project boosts performance on CPU cores www.theregister.com, retrieved 10 May 2024^
  67. Large Language Models in Space—and Beyond Booz Allen Hamilton, 18 April 2025, retrieved 7 August 2025^
  68. Hayden Field. Meta and Booz Allen partner on 'Space Llama' AI program with Nvidia and HPE CNBC, 2025-04-25, retrieved 2025-08-10^
  69. Sunny Cheung. PRC Adapts Meta's Llama for Military and Security AI Applications Jamestown Foundation, October 31, 2024, retrieved 2024-11-03^
  70. James Pomfret, Jessie Pang. Chinese researchers develop AI model for military use on back of Meta's Llama Reuters, November 1, 2024, retrieved November 1, 2024^
  71. Matthew S. Smith. Meta Opens Its AI Model for the U.S. Military IEEE Spectrum, 17 November 2024, retrieved 9 December 2024^
  72. Stefano Maffulli. Meta's LLaMa license is not Open Source Open Source Initiative, 20 July 2023, retrieved 10 July 2025^
  73. Jordan Maris. Meta's LLaMa license is still not Open Source Open Source Initiative, 18 February 2025, retrieved 10 July 2025^
  74. Rosalie Chan, Kali Hays. Meta's new 'open source' Llama 2 AI model isn't so open after all Business Insider, 19 July 2023, retrieved 10 July 2025^
  75. Pascale Davies. Why Meta's 'open source' AI isn't all it seems Euronews, 28 October 2024, retrieved 10 July 2025^
  76. Kylie Robison. Open-source AI must reveal its training data, per new OSI definition The Verge, 28 October 2024, retrieved 10 July 2025^
  77. Krzysztof Siewicz. Llama 3.1 Community License is not a free software license Free Software Foundation, 24 January 2025, retrieved 10 July 2025^
  78. Various Licenses and Comments about Them Free Software Foundation, retrieved 10 July 2025^
  79. Michael Nolan. Llama and ChatGPT Are Not Open-Source IEEE Spectrum, 27 July 2023, retrieved 10 July 2025^
  80. David Gray Widder, Meredith Whittaker, Sarah Myers West. Why 'open' AI systems are actually closed, and why this matters Nature, 27 November 2024^
  81. Will Knight. Meta's Open Source Llama 3 Is Already Nipping at OpenAI's Heels Wired, retrieved 2024-10-20^
  82. Meta's amped-up AI agents confusing Facebook users ABC News, 19 April 2024, retrieved 2024-10-20^
  83. Will Knight. Meta's New Llama 3.1 AI Model Is Free, Powerful, and Risky Wired, retrieved 2024-08-04^
  84. Richard Waters. Meta under fire for 'polluting' open-source Financial Times, October 17, 2024^
  85. Introducing LLaMA: A foundational, 65-billion-parameter large language model Meta AI, 24 February 2023, retrieved 16 March 2023^
  86. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal. LLaMA: Open and Efficient Foundation Language Models 2023^
  87. James Vincent. Meta's powerful AI language model has leaked online — what happens now? The Verge, 8 March 2023, retrieved 16 March 2023^
  88. llama GitHub, retrieved 16 March 2023^
  89. Simon Willison. Large language models are having their Stable Diffusion moment Simon Willison's Weblog, 11 March 2023, retrieved 16 March 2023^
  90. alpaca-lora GitHub, retrieved 5 April 2023^
  91. Jay Peters, James Vincent. Meta has a new machine learning language model to remind you it does AI too The Verge, 24 February 2023^