AI2 Releases OLMo 2: A New Open Language Model to Rival Meta’s Llama

AI2, the nonprofit AI research organization co-founded by the late Paul Allen, has released OLMo 2, its second Open Language Model (OLMo). OLMo 2 is fully transparent and accessible, meeting the Open Source Initiative’s stringent open-source AI definition.

What Makes OLMo 2 Unique?

OLMo 2 was built using publicly accessible tools and data, ensuring reproducibility and openness at every stage of its creation. According to AI2’s announcement, the model includes open-source training code, reproducible recipes, intermediate checkpoints, transparent evaluations, and a comprehensive data set. This approach aims to empower the open-source community to innovate and build on AI2’s advancements.

The OLMo 2 family features two models:

• OLMo 7B: With 7 billion parameters
• OLMo 13B: With 13 billion parameters

Parameters, which serve as indicators of a model’s problem-solving capacity, highlight the sophistication of these models. Higher parameter counts often equate to better performance on complex tasks.

Competitive Performance

It learns from a massive database of 5 trillion tokens sourced from high-quality websites, academic papers, forums, and synthetic and human-generated math workbooks. A AI2 claims OLMo 7B outperforms Meta’s Llama 3.1 8B model in performance, a significant step forward for fully open-source models.

Open Source for All

Distributed under the Apache 2.0 license, OLMo 2 is available for commercial use. This aligns with AI2’s mission of reducing concentrated power in AI development and promoting equitable access to advanced technologies.

The Debate on Open-Source AI

The release of OLMo 2 comes amid concerns over the misuse of open-source AI, especially after reports of Llama models being used for military applications. Dirk Groeneveld, an AI2 engineer, acknowledged these concerns but emphasized the benefits of open models, such as fostering technical advancements, enabling reproducibility, and democratizing AI access.

With OLMo 2, AI2 aims to provide a robust and transparent tool for innovation while addressing the challenges of ethical AI deployment.