OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs (coming soon), and details involved in training these models. This model has been converted from allenai/OLMo-1B for the Hugging Face Transformers format. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B | 3 Trillion |16 | 2048 | 16 | 2048 | | OLMo 7B | 2.5 Trillion | 32 | 4096 | 32 | 2048 | | OLMo 7B Twin 2T | 2 Trillion | 32 | 4096 | 32 | 2048 | We are releasing many checkpoints for these models, for every 1000 training steps. These have not yet been converted into Hugging Face Transformers format, but are available in allenai/OLMo-1B.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
We introduce OLMo 2 32B, the largest model in the OLMo 2 family. OLMo 2 was pre-trained on OLMo-mix-1124 and uses Dolmino-mix-1124 for mid-training. OLMo 2 is the latest in a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. We have released all code, checkpoints, logs, and associated training details on GitHub. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
OLMo 2 32B Instruct March 2025 is post-trained variant of the OLMo-2 32B March 2025 model, which has undergone supervised finetuning on an OLMo-specific variant of the Tülu 3 dataset, further DPO training on this dataset, and final RLVR training on this dataset. Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. Check out the OLMo 2 paper or Tülu 3 paper for more details! OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs, and associated training details. OLMo 2 will be supported in the next version of Transformers, and you need to install it from the main branch using:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
We introduce OLMo 2 1B, the smallest model in the OLMo 2 family. OLMo 2 was pre-trained on OLMo-mix-1124 and uses Dolmino-mix-1124 for mid-training. OLMo 2 is the latest in a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. We have released all code, checkpoints, logs, and associated training details on GitHub. If using vLLM, you will need to install from the main branch until v0.7.4 is released. Please
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
OLMo 2 1B DPO April 2025 is post-trained variant of the allenai/OLMo-2-0425-1B-SFT model, which has undergone supervised finetuning on an OLMo-specific variant of the Tülu 3 dataset and further DPO training on this dataset. Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. Check out the OLMo 2 paper or Tülu 3 paper for more details! OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs, and associated training details. *NOTE: This is different than previous OLMo 2 and Tülu 3 models due to a minor change in configuration. It does NOT have the bos token before the rest. Our other models have at the beginning of the chat template.*
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
分类待补充
输入 待核验→ 输出 待核验
OLMo 2 1B Instruct April 2025 is post-trained variant of the allenai/OLMo-2-0425-1B-RLVR1 model, which has undergone supervised finetuning on an OLMo-specific variant of the Tülu 3 dataset, further DPO training on this dataset, and final RLVR training on this dataset. Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. Check out the OLMo 2 paper or Tülu 3 paper for more details! OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs, and associated training details. - **Model type:** A model trained on a mix of publicly available, synthetic and human-created datasets. - **Language(s) (NLP):** Primarily English - **License:** Apache 2.0 - **Finetuned from model:** allenai/OLMo-2-0425-1B-RLVR1
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
OLMo 2 1B SFT April 2025 is post-trained variant of the allenai/OLMo-2-0425-1B model, which has undergone supervised finetuning on an OLMo-specific variant of the Tülu 3 dataset. Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. Check out the OLMo 2 paper or Tülu 3 paper for more details! OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs, and associated training details. *NOTE: This is different than previous OLMo 2 and Tülu 3 models due to a minor change in configuration. It does NOT have the bos token before the rest. Our other models have at the beginning of the chat template.*
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
We introduce OLMo 2, a new family of 7B and 13B models trained on up to 5T tokens. These models are on par with or better than equivalently-sized fully-open models, and competitive with open-weight models from Meta and Mistral on English academic benchmarks. OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The core models released in this batch include the following: OLMo 2 will be supported in the next version of Transformers, and you need to install it from the main branch using:
译文排队中 · 当前显示来源原文
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
Upon the initial release of OLMo-2 models, we realized the post-trained models did not share the pre-tokenization logic that the base models use. As a result, we have trained new post-trained models. The new models are available under the same names as the original models, but we have made the old models available with a postfix "-preview". See OLMo 2 Preview Post-trained Models for the colleciton of the legacy models. OLMo 2 13B Instruct November 2024 is post-trained variant of the OLMo-2 13B November 2024 model, which has undergone supervised finetuning on an OLMo-specific variant of the Tülu 3 dataset, and finally RLVR training using this data. Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. Check out the OLMo 2 paper or Tülu 3 paper for more details! OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The core models released in this batch include the following:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
We introduce OLMo 2, a new family of 7B and 13B models featuring a 9-point increase in MMLU, among other evaluation improvements, compared to the original OLMo 7B model. These gains come from training on OLMo-mix-1124 and Dolmino-mix-1124 datasets and staged training approach. OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details. OLMo 2 will be supported in the next version of Transformers, and you need to install it from the main branch using:
译文排队中 · 当前显示来源原文
分类待补充
输入 待核验→ 输出 待核验
